Recent Inquiries and Investigations of the CRU Emails and 

the IPCC Fourth Assessment Report 

1. The disclosure of climate data from the Climate Research Unit at the University of 

 East Anglia – UK House of Commons, Science and Technology Committee...................2 

2. Report of the International Panel set up by the University of East Anglia to examine the 

research of the Climate Research Unit – University of East Anglia, Oxburgh 

 Investigative Panel.........................................................................................................63 

3. RA-10 Final Investigation Report Involving Dr. Michael E. Mann – The Pennsylvania 

State University.............................................................................................................72 

4. Assessing an IPCC assessment – An analysis of statements on projected regional impact 

in the 2007 report – Netherlands Environmental Assessment Agency ............................91 

5. The Independent Climate Change E-mails Review – University of East Anglia, 

 Russell Investigative Panel ..........................................................................................191 

Page numbers refer to PDF file pages and not the pages of the individual reports.
HC 387-I 

Published on 31 March 2010 

by authority of the House of Commons 

London: The Stationery Office Limited 

£0.00 

House of Commons 

Science and Technology 

Committee 

The disclosure of 

climate data from the 

Climatic Research Unit 

at the University of 

East Anglia 

Eighth Report of Session 2009–10 

Report, together with formal minutes 

Ordered by the House of Commons 

to be printed 24 March 2010 
The Science and Technology Committee 

The Science and Technology Committee is appointed by the House of Commons to examine the 

expenditure, administration and policy of the Government Office for Science. Under 

arrangements agreed by the House on 25 June 2009 the Science and Technology Committee 

was established on 1 October 2009 with the same membership and Chairman as the former 

Innovation, Universities, Science and Skills Committee and its proceedings were deemed to have 

been in respect of the Science and Technology Committee. 

Current membership 

Mr Phil Willis (Liberal Democrat, Harrogate and Knaresborough)(Chair)

Dr Roberta Blackman-Woods (Labour, City of Durham)

Mr Tim Boswell (Conservative, Daventry)

Mr Ian Cawsey (Labour, Brigg & Goole)

Mrs Nadine Dorries (Conservative, Mid Bedfordshire)

Dr Evan Harris (Liberal Democrat, Oxford West & Abingdon)

Dr Brian Iddon (Labour, Bolton South East)

Mr Gordon Marsden (Labour, Blackpool South) 

Dr Doug Naysmith (Labour, Bristol North West)

Dr Bob Spink (Independent, Castle Point)

Ian Stewart (Labour, Eccles) 

Graham Stringer (Labour, Manchester, Blackley)

Dr Desmond Turner (Labour, Brighton Kemptown)

Mr Rob Wilson (Conservative, Reading East) 

Powers 

The Committee is one of the departmental Select Committees, the powers of which are set out 

in House of Commons Standing Orders, principally in SO No.152. These are available on the 

Internet via www.parliament.uk. 

Publications 

The Reports and evidence of the Committee are published by The Stationery Office by Order of 

the House. All publications of the Committee (including press notices) are on the Internet at 

http://www.parliament.uk/science. A list of reports from the Committee in this Parliament is 

included at the back of this volume. 

Committee staff 

The current staff of the Committee are: Glenn McKee (Clerk); Richard Ward (Second Clerk); Dr 

Christopher Tyler (Committee Specialist); Xameerah Malik (Committee Specialist); Andy Boyd 

(Senior Committee Assistant); Camilla Brace (Committee Assistant); Dilys Tonge (Committee 

Assistant); Melanie Lee (Committee Assistant); Jim Hudson (Committee Support Assistant); and 

Becky Jones (Media Officer). 

Contacts 

All correspondence should be addressed to the Clerk of the Science and Technology Committee, 

Committee Office, 7 Millbank, London SW1P 3JA. The telephone number for general inquiries 

is: 020 7219 2793; the Committee’s e-mail address is: scitechcom@parliament.uk. 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 1

Contents 

Report Page

Summary 3

1 Introduction 5

The Climatic Research Unit at UEA 5

The disclosure of climate data 5

The aftermath 6

The independent inquiries set up by UEA 7

Our inquiry 8

Our Report 9

2 Datasets 10

Climate science 10

Context 11

Complaints and accusations 11

Transparency 12

Dishonesty 19

Perverting the peer review process 21

3 Freedom of information issues 24

Freedom of Information legislation 24

Alleged breaches of the Freedom of Information Act 2000 26

The e-mails 26

Correspondence with the Deputy Information Commissioner 28

Volume of requests 33

4 Independent inquiries 36

The Independent Climate Change Email Review 36

Terms of reference 36

The Review team 38

Transparency 40

Scientific Appraisal Panel 41

Public view of the climate science 42

Need for a single review 44

5 Conclusions 46

Conclusions and recommendations 47

Formal Minutes 52

Witnesses 55

List of written evidence 55
2 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

List of unprinted evidence 57

List of Reports from the Committee during the current Parliament 58
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 3

Summary 

The disclosure of climate data from the Climatic Research Unit (CRU) at the University of 

East Anglia (UEA) in November 2009 had the potential to damage the reputation of the 

climate science and the scientists involved. 

We believe that the focus on CRU and Professor Phil Jones, Director of CRU, in particular, 

has largely been misplaced. Whilst we are concerned that the disclosed e-mails suggest a 

blunt refusal to share scientific data and methodologies with others, we can sympathise 

with Professor Jones, who must have found it frustrating to handle requests for data that he 

knew—or perceived—were motivated by a desire simply to undermine his work. 

In the context of the sharing of data and methodologies, we consider that Professor Jones’s 

actions were in line with common practice in the climate science community. It is not 

standard practice in climate science to publish the raw data and the computer code in 

academic papers. However, climate science is a matter of great importance and the quality 

of the science should be irreproachable. We therefore consider that climate scientists 

should take steps to make available all the data that support their work (including raw data) 

and full methodological workings (including the computer codes). Had both been 

available, many of the problems at UEA could have been avoided. 

We are content that the phrases such as “trick” or “hiding the decline” were colloquial 

terms used in private e-mails and the balance of evidence is that they were not part of a 

systematic attempt to mislead. Likewise the evidence that we have seen does not suggest 

that Professor Jones was trying to subvert the peer review process. Academics should not 

be criticised for making informal comments on academic papers. 

In the context of Freedom of Information (FOIA), much of the responsibility should lie 

with UEA. The disclosed e-mails appear to show a culture of non-disclosure at CRU and 

instances where information may have been deleted, to avoid disclosure. We found prima 

facie evidence to suggest that the UEA found ways to support the culture at CRU of 

resisting disclosure of information to climate change sceptics. The failure of UEA to grasp 

fully the potential damage to CRU and UEA by the non-disclosure of FOIA requests was 

regrettable. UEA needs to review its policy towards FOIA and re-assess how it can support 

academics whose expertise in this area is limited. 

The Deputy Information Commissioner has given a clear indication that a breach of the 

Freedom of Information Act 2000 may have occurred but that a prosecution was timebarred; however no investigation has been carried out. In our view it is unsatisfactory to 

leave the matter unresolved. We conclude that the matter needs to be resolved 

conclusively—either by the Independent Climate Change Email Review or by the 

Information Commissioner. 

We accept the independence of the Climate Change E-mail Review and recommend that 

the Review be open and transparent, taking oral evidence and conducting interviews in 

public wherever possible. 

On 22 March UEA announced the Scientific Appraisal Panel to be chaired by Lord 
4 Optional header 

Oxburgh. This Panel should determine whether the work of CRU has been soundly built 

and it would be premature for us to pre-judge its work. 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 5

1 Introduction 

1. On Friday 20 November 2009 it was reported across the world that hackers had targeted 

a “leading climate research unit”1

 and that e-mails from the University of East Anglia’s 

(UEA) Climatic Research Unit (CRU), one of the world’s foremost centres of climate 

science, had been published in the internet.2

 The story of the substantial file of private emails, documents and data that had been leaked helped ignite the global warming debate in 

the run up to the Copenhagen climate change conference in December 2009. As reported 

by the press, exchanges on the internet alleged that data had been manipulated or deleted, 

in order to support evidence on global warming. 

The Climatic Research Unit at UEA 

2. UEA was founded in 1963 and in 1972 UEA established CRU.3 CRU’s website describes 

the Unit as being “widely recognised as one of the world’s leading institutions concerned 

with the study of natural and anthropogenic [human caused] climate change”.4 CRU has a 

staff of around thirty research scientists and students.5

 But as we heard in oral evidence, it 

is in fact “a very small Unit [with only] three full-time members of academic staff”.6

 

3. CRU has developed a number of the datasets widely used in climate research, including 

the global temperature record used to monitor the state of the climate system, as well as 

statistical software packages and climate models. In its written submission to the inquiry 

UEA outlined CRU’s “pioneering role” in the science of understanding the world’s 

changing climate. CRU’s contributions included the compilation of a global land 

temperature record and the development of increasingly sophisticated methods by which 

to represent the average temperature of the globe and changes in that average over time.7

Professor Edward Acton, the Vice-Chancellor of UEA, indicated that he was “immensely 

proud of what they have done; [as] without them humanity would be vastly less able to 

understand climate change.”8

The disclosure of climate data 

4. In mid November 2009 it appeared that a server used by CRU had been accessed with 

160 MB of data containing more than 1,000 e-mails and 3,000 other documents being 

1 “Hackers target leading climate research unit”, BBC News website, 20 November 2009 

news.bbc.co.uk/1/hi/sci/tech/8370282.stm 

2 For example: “Hacked E-Mail Is New Fodder for Climate Dispute”, New York Times website, 21 November 2009 

www.nytimes.com/2009/11/21/science/earth/21climate.html?_r=4 and “Hackers leak emails, stoking climate debate”, 

Sydney Morning Herald website, 23 November 2009, www.smh.com.au/technology/technology-news/hackers-leakemails-stoking-climate-debate-20091123-iu6u.html 

3 Ev 17, paras 1.2 and 1.5 

4 “About the Climatic Research Unit”, CRU website, www.cru.uea.ac.uk/cru/about/ 

5 As above

6 Q 92 

7 Ev 17, paras 1.5-1.6 

8 Q 152 
6 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

copied.9 A UEA spokeswoman confirmed that the information was not available on a 

server that could be easily accessed and could not have been inadvertently released.10 It is 

not known exactly when the breach occurred; the RealClimate website, “a commentary site 

on climate science by working climate scientists for the interested public and journalists”,11

indicated that UEA had been notified of the possible security breach on 17 November.12

The following was posted anonymously on the climate-sceptic blog, The Air Vent: 

November 17, 2009 at 9:57 pm 

We feel that climate science is, in the current situation, too important to be kept 

under wraps. 

We hereby release a random selection of correspondence, code, and documents. 

Hopefully it will give some insight into the science and the people behind it.13

From here the debate was “blown wide open”.14 The Guardian ran the story on 20 

November with the headline: “Climate sceptics claim leaked e-mails are evidence of 

collusion among scientists”.15

5. UEA issued a statement on 20 November: “This information has been obtained and 

published without our permission and we took immediate action to remove the server in 

question from operation. We are undertaking a thorough internal investigation and we 

have involved the police in this inquiry.”16 The e-mails contained technical and routine 

aspects of climate research, including data analysis and details of scientific conferences. 

The controversy has focused on a small number of e-mails, particularly those sent to, or 

written by, climatologist Professor Phil Jones, the Director of CRU. 

The aftermath 

6. Condemnation of alleged malpractices found within the leaked CRU e-mails was quickly 

disseminated on the internet. Contributors to climate change debate websites and written 

submissions to us claimed that these e-mails showed a deliberate and systematic attempt by 

leading climate scientists to manipulate climate data, arbitrarily adjusting and “cherrypicking” data that supported their global warming claims and deleting adverse data that 

questioned their theories.17 It was alleged that UEA may not have complied with the 

requirements of the Freedom of Information Act 2000, that inappropriate statistical 

methods and defective computer programmes may have been used to analyse data and that 

9 RealClimate website archive, November 2009, www.realclimate.org/index.php/archives/2009/11/the-cru-hack 

10 “Scotland Yard call in to probe climate data leak from UEA in Norwich”, Norwich Evening News, 1 December 2009 

11 RealClimate website ‘about’ page, www.realclimate.org 

12 RealClimate website archive, November 2009, www.realclimate.org/index.php/archives/2009/11/the-cru-hack; the 

data may have been downloaded on to the RealClimate—see paragraph 12. 

13 The Air Vent website, November 2009 archive, noconsensus.wordpress.com/2009/11/page/3/ 

14 As above

15 “Climate sceptics claim leaked emails are evidence of collusion among scientists”, The Guardian, 20 November 2009 

16 “Sceptics publish climate e-mails ‘stolen from East Anglia University’”, The Times, 21 November 2009 

17 For examples see Ev 85 [Roger Helmer MEP], Ev 92 [Godfrey Bloom MEP], and Ev 144 [Stephen McIntyre] 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 7

CRU may have attempted to abuse the process of peer review to prevent the publication of 

research papers with conflicting opinions about climate change.18 

7. In a statement released on 24 November, Professor Trevor Davies, UEA pro-ViceChancellor with responsibility for research, rejected calls for Professor Jones’s resignation: 

“We see no reason for Professor Jones to resign and, indeed, we would not accept his 

resignation. He is a valued and important scientist.”19 He also contested several of the 

claims of malpractice: “It is well known within the scientific community and particularly 

those who are sceptical of climate change that over 95% of the raw station data has been 

accessible through the Global Historical Climatology Network for several years. We are 

quite clearly not hiding information which seems to be the speculation on some blogs and 

by some media commentators”. He added: 

There is nothing in the stolen material which indicates that peer-reviewed 

publications by CRU, and others, on the nature of global warming and related 

climate change are not of the highest-quality of scientific investigation and 

interpretation. CRU’s peer-reviewed publications are consistent with, and have 

contributed to, the overwhelming scientific consensus that the climate is being 

strongly influenced by human activity.20

8. On 1 December, Professor Jones announced that he would step aside from the Director’s 

role during the course of the independent review.21

The independent inquiries set up by UEA 

9. On 3 December UEA announced that an independent review—the Independent Climate 

Change Email Review—into the allegations made against CRU would be carried out by Sir 

Muir Russell.22 Professor Acton explained in a letter to us why Sir Muir was chosen to head 

the review: 

Sir Muir is extremely experienced in public life, has an understanding of the conduct 

of universities and research, and is entirely independent of any association with this 

University and with the climate change debate.23

10. Alongside the Independent Climate Change E-Mails Review, UEA decided on a 

separate scientific assessment of CRU’s key scientific publications; an external reappraisal 

of the science itself. The Royal Society agreed to assist UEA in identifying assessors with 

the requisite experience, standing and independence.24 UEA announced on 22 March that 

Lord Oxburgh FRS would “chair an independent Scientific Assessment Panel to examine 

18 For examples see Ev 90 [Phillip Bratby]; Ev 115 [David Holland], para 2; Ev 144 [Stephen McIntyre]; Ev 194 [Peabody 

Energy Company], para 24. 

19 “Climate scientist at centre of leaked email row dismisses conspiracy claims”, The Guardian, 24 November 2009 

20 UEA, “CRU update 2”, 24 November 2009, www.uea.ac.uk/mac/comm/media/press/2009/nov/CRUupdate 

21 UEA, “CRU update 3”, 1 December 2009, www.uea.ac.uk/mac/comm/media/press/2009/nov/CRUupdate 

22 “Sir Muir Russell to head the Independent Review into the allegations against the Climatic Research Unit (CRU)”, 

UEA Press Release, 3 December 2009, www.uea.ac.uk/mac/comm/media/press/2009/dec/CRUreview 

23 Ev 16 

24 Ev 18, para 2.3 
8 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

important elements of the published science of the Climatic Research Unit (CRU) at the 

University of East Anglia”.25

Our inquiry 

11. We were concerned by the press reports and on 1 December 2009 the Chair of the 

Committee wrote to the Vice-Chancellor of UEA. The letter explained that we took a close 

interest in academic integrity and the systems in place to ensure the quality of evidence 

from research and evidence-based policy making. The letter requested a note on the recent 

events setting out: 

a) what had taken place; 

b) the steps that had been taken to investigate the allegations and to test the integrity of 

the data held and used by CRU; 

c) how CRU justified its commitment to academic transparency; and 

d) how the Vice-Chancellor proposed to restore confidence in CRU and its handling of 

data. 

We also asked for an assurance that none of the data referred to in the e-mails that had 

been publicised had been destroyed.26

12. UEA replied on 10 December 2009. It explained that “a significant amount of material 

including emails and documents appears to have been accessed illegally from a back-up 

server in CRU and downloaded in whole, or possibly in part, on to the RealClimate 

website.”27 This incident was the subject of a police enquiry and the Norfolk Constabulary 

investigation was expected to take some time. UEA was keen to stress that this “episode is 

being treated very seriously” and announced that it had set up the independent inquiry, 

headed by Sir Muir Russell, to investigate the allegations against CRU. UEA said that “none 

of the adjusted station data referred to in the emails that have been published has been 

destroyed.”28

13. In the light of the gravity of the allegations against CRU, the growing weight of 

damaging press coverage, on-going concerns about the deletion of data and the serious 

implications for UK science we decided to hold an inquiry into the disclosure of the data at 

CRU. On 22 January 2010 we therefore announced the inquiry inviting submissions on 

three key issues: 

• What were the implications of the disclosures for the integrity of scientific research? 

• Were the terms of reference and scope of the Independent Review announced on 3 

December 2009 by UEA adequate? 

25 “CRU Scientific Assessment Panel announced”, UEA Press Release, 22 March 2010, 

www.uea.ac.uk/mac/comm/media/press/CRUstatements/SAPannounce 

26 House of Commons Science and Technology Committee Press Notice 04, 7 December 2009, Session 2009–10 

27 Ev 16 

28 Ev 17 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 9

• How independent were the other two international data sets (see paragraph 23)? 

14. If there had been more time available before the end of this Parliament we would have 

preferred to carry out a wider inquiry into the science of global warming itself. In response 

to enquiries we issued a statement on 1 February making it clear that the inquiry would 

focus on the terms of reference announced on 22 January and that this was not an inquiry 

into global warming.29

15. We set a deadline of 10 February for the submission of memoranda and we have 

received 58 submissions, not including supplementary memoranda. We held one oral 

evidence session on 1 March, when we took evidence from five panels: 

a) Rt Hon Lord Lawson of Blaby, Chairman, and Dr Benny Peiser, Director, Global 

Warming Policy Foundation; 

b) Richard Thomas CBE, former Information Commissioner; 

c) Professor Edward Acton, Vice-Chancellor, UEA and Professor Phil Jones, Director of 

CRU; 

d) Sir Muir Russell, Head of the Independent Climate Change E-Mails Review; and 

e) Professor John Beddington, Government Chief Scientific Adviser, Professor Julia 

Slingo OBE, Chief Scientist, Met Office, and Professor Bob Watson, Chief Scientist, 

Department for Environment, Food and Rural Affairs. 

16. We would like to thank everyone who contributed to the inquiry through written 

submissions or oral evidence. We also received unsolicited copies of a number of books 

challenging anthropogenic global warming and reviewing events at CRU and the disclosed 

e-mails.30

Our Report 

17. In the time left before the end of this Parliament we will not be able to cover all the 

issues raised by the events at UEA, nor cover all the ground that would be covered by the 

Independent Climate Change Email Review and the Scientific Appraisal Panel. We have 

therefore concentrated on what we believe to be key issues. Of central concern is the 

accuracy and availability of CRU’s data, datasets and computer programming, which we 

address in Chapter 2 of this Report; and related to the data and methodology is the 

question of access, or the withholding of access, under the Freedom of Information Act 

2000 which we cover in Chapter 3. Finally, in Chapter 4 we comment on the independent 

reviews that UEA has announced. 

29 House of Commons Science and Technology Committee Press Notice 11, 1 February 2010, Session 2009–10 

30 The Committee received the following books: 

Christopher Booker, The Real Global Warming Disaster, Continuum, 2009 

A.W. Montford, The Hockey Stick Illusion, Stacey International, 2010 

Steven Mosher and Tom Fuller, Climategate, St Matthew Publishing, 2010 

Ian Plimer, Heaven and Earth, Quartet Books Limited, 2009 
10 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

2 Datasets 

Climate science 

18. Climate is distinct from weather: it is the average of weather conditions over a number 

of years. Climatologists study climates in different parts of the world and for the Earth as a 

whole. CRU, according to its website: “has developed a number of the data sets widely used 

in climate research, including the global temperature record used to monitor the state of 

the climate system, as well as statistical software packages and climate models”.31

19. The process of calculating the Earth’s average global temperatures (past, present and 

future) is complicated and lengthy. Data from thousands of weather stations all around the 

world, on land and at sea, must be collected, checked for quality, adjusted for 

inconsistencies and error margins, and then mapped onto a series of grids on the Earth’s 

surface. The methods, results and conclusions are then presented to the academic world, 

first by passing the peer review process prior to publication, and second, after presentation, 

the scrutiny of the wider academic community. 

20. Climate science, like any other science, uses the scientific method to make its 

assessments of past and present climate and predictions about the future climate. The key 

characteristics of the scientific method can be described as: characterisations, hypotheses, 

predictions, and experiments. 

• Characterisations: consideration of a problem, and examination of whether or not an 

explanation exists for it. 

• Hypotheses: if no such explanation exists, a new explanation is stated. 

• Predictions: what consequences follow from a new explanation? 

• Experiments: is the outcome consistent with the predicted consequences? 

Each of these is subject to peer review prior to the formal sharing of knowledge through 

publication. Through peer review scientists allow their views and methods to be critically 

appraised expertly and externally. 

• Replication and verification 

To have the results and conclusions survive criticism or scepticism and be part of the 

accepted canon of scientific knowledge, most experiments will have to be demonstrably 

replicable (by the same group) to pass peer review and will often need to be verified by 

other independent researchers taking similar approaches. 

21. Therefore climatologists are, like other scientists, required to test their theories—such 

as global warming and the causes of warming—against observational data. They must also 

replicate and verify their experiments, by holding independent datasets and conducting 

independent analyses of these datasets, and by publishing their full methods and results for 

31 www.cru.uea.ac.uk/cru/about 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 11

scrutiny. Ultimately, these ideas are put up to the threat of falsification by other scientists 

working in the field. 

22. In this Chapter we discuss some aspects of this process. 

Context 

23. There are three main international climate datasets, which have been built up from 

direct temperature measurements on land and sea at weather stations all around the world: 

a) the National Climatic Data Center (NCDC) of the National Oceanographic and 

Atmospheric Administration (NOAA) in Asheville, North Carolina, USA; 

b) the Goddard Institute of Space Studies (GISS), part of the National Aeronautic and 

Space Administration (NASA) in New York, USA; and 

c) CRUTEM3, at CRU, UEA.32

24. In addition, there are two others, one in Russia and one in Japan, that use similar 

methods.33 There are also two that use satellite observations, by the University of Alabama 

at Huntsville and by Remote Sensing Systems, California.34

25. Professor Jones, commenting on the different climate research groups around the 

world in the UK, US, Russia and Japan,35 told us that: 

we are all working independently so we may be using a lot of common data but the 

way of going from the raw data to a derived product of gridded temperatures and 

then the average for the hemisphere and the globe is totally independent between the 

different groups.36

26. What sets the CRU dataset apart is its comprehensiveness: 

The CRU dataset, which forms the land surface component of the HadCRUT global 

temperature record, was compiled with the aim of comprehensiveness. The majority 

of the data in it are derived from the same freely-available raw data sets used by 

NOAA and NASA. However, it also includes data derived from station data that 

were obtained directly from countries, institutions and scientists on the 

understanding that they would not be passed on.37

Complaints and accusations 

27. The complaints and accusations made against CRU in relation to the scientific process 

come under two broad headings. The first is transparency: that CRU failed to abide by best 

32 Ev 21, para 4.2 

33 Q 78 

34 Ev 104 [D.R. Keiller], para 2 

35 Q 79 

36 Q 80 

37 Ev 64 [John Beddington and Julia Slingo] 
12 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

scientific practice by refusing to share its raw data and detailed methods. The second is 

honesty: that CRU has deliberately misrepresented the data, in order to produce results 

that fit its preconceived views about the anthropogenic warming of the climate. We take 

each of these complaints and accusations in turn. 

Transparency 

Raw data 

28. Warwick Hughes, a “freelance earth scientist from Australia”,38 had asked Professor 

Jones for CRU’s raw data. He received the following reply: 

I should warn you that some data we have we are not supposed [to] pass on to 

others. We can pass on the gridded data—which we do. Even if WMO [World 

Meteorological Organization] agrees, I will still not pass on the data. We have 25 or 

so years invested in the work. Why should I make the data available to you, when 

your aim is to try and find something wrong with it.39

29. On the face of it, this looks like an unreasonable response to a reasonable request. As 

Lord Lawson put it: “Ask any decent scientist and they will say the keystone for integrity in 

scientific research is full and transparent disclosure of data and methods”.40 However, 

Professor Jones, while confessing that he has sent some “awful” e-mails,41 defended his 

position. 

30. First, in answer to the question of whether the raw data are accessible and verifiable, 

Professor Jones told us that: 

The simple answer is yes, most of the same basic data are available in the United 

States in something called the Global Historical Climatology Network. They have 

been downloadable there for a number of years so people have been able to take the 

data, do whatever method of assessment of the quality of the data and derive their 

own gridded product and compare that with other workers.42

31. In addition, of course, there are the sources of the data, the weather stations, to which 

any individual is free to go and collect the data in the same way that CRU did. This is 

feasible because the list of stations that CRU used was published in 2008.43

32. Even if CRU had wanted to, it would have been unable to publish all of these data 

because, as Professor Acton explained, some of the data are bound by commercial 

agreements with different national meteorological organisations: 

38 www.warwickhughes.com 

39 Ev 158, Appendix 1 

40 Q 9 

41 Q 103 

42 Q 78 

43 Q 98 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 13

Unfortunately, several of these countries impose conditions and say you are not 

allowed to pass [on the data]. Seven countries have said “No, you cannot”, half the 

countries have not yet answered, Canada and Poland are amongst those who have 

said, “No you cannot publish it” and also Sweden. Russia is very hesitant. We are 

under a commercial promise, as it were, not to; we are longing to publish it because 

what science needs is the most openness.44

(The issue with Sweden has since been resolved. The Swedish Meteorological and 

Hydrological Institute gave permission for CRU to publish its Swedish data on the UEA 

website on 8 March 2010.45) 

33. Second, as UEA explained in its submission, it is: 

sometimes necessary to adjust temperature data because changes in station location, 

instrument or observation time, or in the methods used to calculate monthly average 

temperatures can introduce false trends. These have to be removed or adjusted, or 

else the overall series of values will be incorrect. In the early 1980s, CRU 

painstakingly examined the long-term homogeneity of each station temperature 

series which it acquired. As a result, data were adjusted for about 11% of the sites, 

that is approximately 314 sites out of a then-total of some 3,276. This was in 

complete accordance with standard practice, and all adjustments were documented.46

34. Professor Jones added, when he gave oral evidence: 

It is all documented [...] what [adjustments we made to the data] in the 1980s and 

since then we have obviously added more station data as more has become available, 

as countries have digitised more data; we have added that in and we have reported 

on that in our peer review publications in 2003 and 2006.47

35. These kinds of adjustments to raw data take a lot of time. That is why, in the words of 

Professor Jones, “Most scientists do not want to deal with the raw station data, they would 

rather deal with a derived product”.48

36. A third point was made by Professor Acton that CRU should not be under any 

obligation to provide raw data: 

May I also point out that it is not a national archive, it is not a library, it is a research 

unit. It has no special duty to conserve and its data is the copy of data provided by 

over 150 countries, whose national meteorological stations turn the data into the 

average for a month.49

44 Q 94 

45 Ev 39, para B 

46 Ev 18, para 3.4 

47 Q 81 

48 Q 107 

49 Q 92 
14 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

37. CRU’s refusal to release the raw data gave some the impression that it was deliberately 

keeping its work private so that its studies could not “be replicated and critiqued”.50 The 

Peabody Energy Company said of CRU that “they appeared to be particularly concerned 

that putting their information in the public domain would expose their work to 

criticism”.51 Even an effort to conduct a simple quality check was said to be thwarted by 

CRU’s unwillingness to share the data it had used.52 In contrast, NASA has been able to 

make all its raw data available as well as its programmes.53

38. We recognise that some of the e-mails suggest a blunt refusal to share data, even 

unrestricted data, with others. We acknowledge that Professor Jones must have found it 

frustrating to handle requests for data that he knew—or perceived—were motivated by 

a desire simply to seek to undermine his work. But Professor Jones’s failure to handle 

helpfully requests for data in a field as important and controversial as climate science 

was bound to be viewed with suspicion. He was obviously frustrated by other workers 

in the field trying to “undermine” his work, but his actions were inevitably 

counterproductive. Professor Jones told us that the published e-mails represented only 

“one tenth of 1%” of his output, which amounts to one million e-mails, and that we 

were only seeing the end of a protracted series of e-mail exchanges. We consider that 

further suspicion could have been allayed by releasing all the e-mails. In addition, we 

consider that had the available raw data been available online from an early stage, these 

kinds of unfortunate e-mail exchanges would not have occurred. In our view, CRU 

should have been more open with its raw data and followed the more open approach of 

NASA to making data available.

39. We are not in a position to set out any further the extent, if any, to which CRU 

should have made the data available in the interests of transparency, and we hope that 

the Independent Climate Change Email Review will reach specific conclusions on this 

point. However, transparency and accountability are of are increasing importance to 

the public, so we recommend that the Government reviews the rules for the accessibility 

of data sets collected and analysed with UK public money.

Methods 

40. The Royal Society of Chemistry in its submission made it clear that: 

It is essential that the public and all non-specialists remain truly confident in the 

scientific method to provide a sound scientific evidence-base on which strong 

decisions can be made.54

There have been criticisms that Professor Jones and colleagues have not shared their 

methodologies. Andrew Montford, author of The Hockey Stick Illusion,

55 pointed out in his 

memorandum that: 

50 Ev 194 [Peabody Energy Company], para 20 

51 As above

52 Ev 152 [Steven Mosher], para 8 

53 Q 150 [Professor Jones] 

54 Ev 170, summary 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 15

The scientific method demands that findings be subject to testing and verification by 

others. The refusal of CRU scientists to release information to those who they felt 

might question or threaten their findings have led many to conclude that the CRU’s 

work is not trustworthy.56

41. Professor Jones contested these claims. According to him, “The methods are published 

in the scientific papers; they are relatively simple and there is nothing that is rocket science 

in them”.57 He also noted: “We have made all the adjustments we have made to the data 

available in these reports58; they are 25 years old now”.59 He added that the programme that 

produced the global temperature average had been available from the Met Office since 

December 2009.60

42. On this basis, he argued, it was unnecessary to provide the exact codes that he used to 

produce the CRUTEM3 chart. The Met Office had released its code and it produced 

exactly the same result.61

43. In answer to the charge that the computer codes that were stolen from CRU’s computer 

network were defective,62 Professor Jones pointed out that: 

Those codes are from a much earlier time, they are from the period about 2000 to 

2004. [They] do not relate to the production of the global and hemispheric 

temperature series. They are nothing to do with that, they are to do with a different 

project [...] that was funded by the British Atmospheric Data Centre, which is run by 

NERC, and that was to produce more gridded temperature data and precipitation 

data and other variables. A lot of that has been released on a Dutch website and also 

the BADC website.63

44. CRU’s alleged refusal to disclose its assumptions and methodologies gave credence to 

the view that exposure to “independent scrutiny would have undermined the AGW 

[anthropogenic global warming] hypothesis”.64 However, the failure to publish the 

computer code for CRUTEM3 left CRU vulnerable when concerns emerged that other 

codes it used had faults. John Graham-Cumming, a professional computer programmer, 

told us that: 

 

55 Andrew Montford, The Hockey Stick Illusion: Climategate and the corruption of science, Stacey International, 2010 

56 Ev 159, para 4 

57 Q 92 

58 Raymond Bradley, Mick Kelly, Phil Jones and others, A Climatic Data Bank for Northern Hemisphere Land Areas, 

1851-1980, US DoE, Technical Report TRO17, 1985, p 335; Phil Jones, Sarah Raper, Ben Santer, and others, A Grid 

Point Surface Air Temperature Data Set for the Northern Hemisphere, DoE Technical Report No. TR022, US 

Department of Energy, 1985, p 251; Phil Jones, Sarah Raper, Claire Goodess, and others, A Grid Point Surface Air 

Temperature Data Set for the Southern Hemisphere, 1851-1984, DoE Technical Report No. TR027, US Department of 

Energy, 1986, 73 

59 Q 97 

60 As above

61 Qq 139-42 

62 Ev 32, Q 137; Ev 196 [John Graham-Cumming] 

63 Qq 137-38 

64 Ev 94 [Clive Menzies], para 1.5 
16 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

the organization writing the [other] code did not adhere to standards one might find 

in professional software engineering. The code had easily identified bugs, no visible 

test mechanism, was not apparently under version control and was poorly 

documented. It would not be surprising to find that other code written at the same 

organization was of similar quality. And given that I subsequently found a bug in the 

actual CRUTEM3 code only reinforces my opinion.65

45. The conspiracy claims were fuelled by CRU’s refusal to share the most detailed aspects 

of its methodologies, for example, the computer codes for producing global temperature 

averages. We note that the research passed the peer review process of some highly 

reputable journals. However, we note that CRU could have been more open at that time 

in providing the detailed methodological working on its website. We recommend that 

all publicly funded research groups consider whether they are being as open as they can 

be, and ought to be, with the details of their methodologies.

Repeatability and verification 

46. These complaints and concerns surrounding transparency cut to the heart of the 

scientific process. It has been argued that without access to the raw data and detailed 

methodology it is not possible to check the results of CRU’s work. The Institute of Physics 

pointed out that: 

Published reconstructions may represent only a part of the raw data available and 

may be sensitive to the choices made and the statistical techniques used. Different 

choices, omissions or statistical processes may lead to different conclusions. This 

possibility was evidently the reason behind some of the (rejected) requests for further 

information.66

47. This has substance if one considers CRU’s work in isolation. But science is more than 

individual researchers or research groups. One should put research in context and ask the 

question: what would one hope to find by double checking the processing of the raw data? 

If this were the only dataset in existence, and Professor Jones’s team had been the only 

team in the world to analyse it, then it might make sense to double check independently 

the processing of the raw data and the methods. But there are other datasets and other 

analyses that have been carried out as Professor Jones explained: 

There are two groups in America that we [CRU] compare with and there are also 

two additional groups, one in Russia and one in Japan, that also produce similar 

records to ourselves and they all show pretty much the same sort of course of 

instrumental temperature change since the nineteenth century compared to today.67

[...] we are all working independently so we may be using a lot of common data but 

the way of going from the raw data to a derived product of gridded temperatures and 

65 Ev 196 

66 Ev 167, para 4 

67 Q 78 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 17

then the average for the hemisphere and the globe is totally independent between the 

different groups.68

48. In its memorandum UEA explained the differences between the methodologies used by 

three basic datasets for land areas of the world, NOAA, NASA and CRU/UEA: 

All these datasets rely on primary observations recorded by NMSs [National 

Meteorological Services] across the globe.69

GISS[70] and NCDC[71] each use at least 7,200 stations. CRUTEM3 uses fewer. In 

CRUTEM3, each monthly temperature value is expressed as a departure from the 

average for the base period 1961–90. This “anomaly method” of expressing 

temperature records demands an adequate amount of data for the base period; this 

limitation reduces the number of stations used by CRUTEM3 to 4,348 (from the 

dataset total of 5,121). The latest NCDC analysis [...] has now moved to the “anomaly 

method” though with different refinements from those of CRU.72

NCDC and GISS use different approaches to the problem of “absolute temperature” 

from those of CRUTEM3. The homogeneity procedures undertaken by GISS and 

NCDC are completely different from those adopted for CRUTEM3. NCDC has an 

automated adjustment procedure [...], whilst GISS additionally makes allowances for 

urbanization effects at some stations.73

49. In our call for evidence we asked for submissions on the question of how independent 

the other international data sets are. We have established to the extent that a limited 

inquiry of this nature can, that the NCDC/NOAA and GISS/NASA data sets measuring 

temperature changes on land and at sea have arrived at similar conclusions using similar 

data to that used by CRU, but using independently devised methodologies. We have 

further identified that there are two other data sets (University of Alabama and Remote 

Sensing Systems), using satellite observations that use entirely different data than that used 

by CRU. These also confirm the findings of the CRU work. We therefore conclude that 

there is independent verification, through the use of other methodologies and other 

sources of data, of the results and conclusions of the Climate Research Unit at the 

University of East Anglia.

50. The fact that all the datasets show broadly the same sort of course of instrumental 

temperature change since the nineteenth century compared to today was why Professor 

John Beddington, the Government Chief Scientific Adviser, had the confidence to say that 

68 Q 80 

69 Ev 21, para 4.3 

70 Dataset held by the Goddard Institute for Space Studies (GISS, USA) part of the National Aeronautic and Space 

Administration (NASA) 

71 Global Historical Climatology Network (GHCN) dataset held by National Climatic Data Center (NCDC), the National 

Oceanographic and Atmospheric Administration (NOAA, USA) 

72 Ev 21, para 4.4 

73 Ev 21, para 4.5 
18 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

human induced global warming was, in terms of the evidence to support that hypothesis, 

“unchallengeable”:74

I think in terms of datasets, of the way in which data is analysed, there will always be 

some degree of uncertainty but when you get a series of fundamentally different 

analyses on the basic data and they come up with similar conclusions, you get a [...] 

great deal of certainty coming out of it.75

51. Even if the data that CRU used were not publicly available—which they mostly 

are—or the methods not published—which they have been—its published results would 

still be credible: the results from CRU agree with those drawn from other international 

data sets; in other words, the analyses have been repeated and the conclusions have 

been verified.

52. That is probably part of why it has not been practice in the climate science community 

to publish all the data and computer codes with the academic papers. We got to the crux of 

the issue during an interesting exchange with Professor Jones: 

Graham Stringer: You are saying that every paper that you have produced, the 

computer programmes, the weather stations, all the information, the codes, have 

been available to scientists so that they could test out how good your work was. Is 

that the case on all the papers you have produced? 

Professor Jones: That is not the case. 

Graham Stringer: Why is it not? 

Professor Jones: Because it has not been standard practice to do that. 

Graham Stringer: That takes me back to the original point, that if it is not standard 

practice how can the science progress? 

Professor Jones: Maybe it should be standard practice but it is not standard practice 

across the subject.76

53. Another reason why data and the codes were not published may be that norms for 

publication evolved in a period when the journals were only published in hard copy. In 

such circumstances it is understandable why an editor would not want to publish raw 

climate data (extremely long lists of numbers) and code for the computer programmes that 

analyse the data (which run to hundreds of thousands of lines of code). However, in the age 

of the internet, these kinds of products can be made available more easily, and we are 

minded to agree with Professor Jones observation on this point that: “Maybe it should be 

standard practice”.77 

74 Q 191 

75 Qq 191–92 

76 Qq 100–02 

77 Q 102 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 19

54. It is not standard practice in climate science and many other fields to publish the 

raw data and the computer code in academic papers. We think that this is problematic 

because climate science is a matter of global importance and of public interest, and 

therefore the quality and transparency of the science should be irreproachable. We 

therefore consider that climate scientists should take steps to make available all the data 

used to generate their published work, including raw data; and it should also be made 

clear and referenced where data has been used but, because of commercial or national 

security reasons is not available. Scientists are also, under Freedom of Information laws 

and under the rules of normal scientific conduct, entitled to withhold data which is due 

to be published under the peer-review process.78 In addition, scientists should take steps 

to make available in full their methodological workings, including the computer codes. 

Data and methodological workings should be provided via the internet. There should 

be enough information published to allow verification.

Dishonesty 

55. Of all the e-mails released, one dated 16 November 1999 has caused particular concern: 

I’ve just completed Mike’s Nature trick of adding in the real temps to each series for 

the last 20 years (ie from 1981 onwards) amd [sic] from 1961 for Keith’s to hide the 

decline.79

56. The word “trick” and the phrase “hide the decline” have been taken by some to 

demonstrate intent on the part of Professor Jones to “falsify data” and to “exaggerate 

warming”.80

“Trick” 

57. In his submission, Peter Taylor, author of Chill,

81 states that: 

The tree ring data did not match the model expectation (ie the ‘hockey stick’ pattern 

of a sudden rise at the end of the period). Rather than admit this, the team-workers 

discuss using Michael Mann’s ‘trick’ of replacing the offending tree-ring data and 

using instrumental data in its place in a spliced graph.82 

58. UEA interpreted the use of the word “trick” differently: 

as for the (now notorious) word ‘trick’, so deeply appealing to the media, this has 

been richly misinterpreted and quoted out of context. It was used in an informal 

email, discussing the difficulties of statistical presentation. It does not mean a ‘ruse’ 

or method of deception. In context it is obvious that it is used in the informal sense 

78 See paragraph 78 and following; section 22 of the FOIA provides an exemption from disclosure where the requested 

information is intended for future (but imminent) publication. 

79 E-mail from Phil Jones to Ray Bradley, 16 November 1999 

80 Ev 93 [Godfrey Bloom MEP], para 4 

81 Peter Taylor, Chill, A Reassessment of Global Warming Theory: Does Climate Change Mean the World is Cooling, and 

If So What Should We Do About It?, Clairview Books, 2009 

82 Ev 188, para 22 
20 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

of ‘the best way of doing something’. In this case it was ‘the trick or knack’ of 

constructing a statistical illustration which would combine the most reliable proxy 

and instrumental evidence of temperature trends.83

59. These interpretations of the colloquial meaning of “trick” have been accepted by even 

the staunchest of critics: 

Lord Lawson of Blaby: The sinister thing is not the word ‘trick’. In their [UEA’s] 

own evidence they say that what they mean by ‘trick’ is the best way of doing 

something. 

Chairman: You accept that? 

Lord Lawson of Blaby: I accept that.84

60. Critics of CRU have suggested that Professor Jones’s use of the word “trick” is 

evidence that he was part of a conspiracy to hide evidence that did not fit his view that 

recent global warming is predominately caused by human activity. The balance of 

evidence patently fails to support this view. It appears to be a colloquialism for a “neat” 

method of handling data.

“Hide the decline” 

61. Lord Lawson did, however, describe CRU’s treatment of the data as “reprehensible”,85

because, in his view, Professor Jones deliberately hid data that demonstrated a decline in 

temperatures.86

62. The data that he believed to be “hidden” are a set of tree ring data that disagree with 

other data sources regarding temperature trends. Lord Lawson said: “when the proxy series 

[...] departed from the measured temperature series, a normal person will say maybe that 

means the proxy series is not all that reliable”.87 In that context he made two specific 

claims: 

• that the tree ring data were flawed because “for a long period before 1421 they relied on 

one single pine tree”;88 and 

• that the divergence problem was not just for data after the 1960s, “it is not a good fit in 

the latter half of the nineteenth century either”.89

63. It is outside the remit of the terms of reference of this inquiry to make a detailed 

assessment of the science, but it is worth noting that Professor Jones had a very different 

perspective. On the first point, he commented: 

83 Ev 19, para 3.5.6 

84 Qq 25–26 

85 Q 26 

86 Qq 26–28 

87 Q 26 

88 As above

89 Q 28 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 21

That particular reconstruction went back to 1400, or just after 1400, and that is 

because there are insufficient trees to go back before that, there are more than just 

one. We have criteria to determine how far you can go back in terms of the number 

of trees you have at a certain number of sites.90

64. On the second point, he told us: 

One of the curves was based on tree ring data which showed a very good relationship 

between the tree rings and the temperature from the latter part of the nineteenth 

century through to 1960, and after that there was a divergence where the trees did 

not go up as much as the real temperatures had.91

65. Professor Jones has published on this issue on several occasions, including a 1998 

Nature paper92 and subsequent papers.93 He contested the view that he was trying to hide 

the decline in the sense that he was trying to pretend that these data did not exist and 

thereby exaggerate global warming: “We do not accept it was hidden because it was 

discussed in a paper[94] the year before and we have discussed it in every paper we have 

written on tree rings and climate”.95 Rather, what was meant by “hide the decline” was 

remove the effects of data known to be problematic in the sense that the data were known 

to be misleading. UEA made it clear in its written submission that: 

CRU never sought to disguise this specific type of tree-ring “decline or divergence”. 

On the contrary, CRU has published a number of pioneering articles that illustrate, 

suggest reasons for, and discuss the implications of this interesting phenomenon.96

66. Critics of CRU have suggested that Professor Jones’s use of the words “hide the 

decline” is evidence that he was part of a conspiracy to hide evidence that did not fit his 

view that recent global warming is predominantly caused by human activity. That he 

has published papers—including a paper in Nature—dealing with this aspect of the 

science clearly refutes this allegation. In our view, it was shorthand for the practice of 

discarding data known to be erroneous. We expect that this is a matter the Scientific 

Appraisal Panel will address.

Perverting the peer review process 

67. The main allegations on the suppression or distortion of others’ findings concern the 

role of CRU in the operation of the peer review process. It has been alleged that scientists at 

CRU abused the peer review process to prevent those with dissenting views on climate 

change the opportunity in getting papers published. There are three key accusations. First, 

90 Q 125 

91 Q 122 

92 Q 122; Keith Briffa and others, “Reduced sensitivity of recent tree-growth to temperature at high northern 

latitudes”, Nature, vol 391 (1998), pp 678-82 

93 For example: Edward Cook, Paul Krusic and Phil Jones, “Dendroclimatic signals in long tree-ring chronologies from 

the Himalayas of Nepal”, International Journal of Climatology, Vol 23 (2003), pp 707-32 

94 Keith Briffa and others, “Trees tell of past climates: but are they speaking less clearly today?”, Philosophical 

Transactions of the Royal Society of London Series B-Biological Sciences, vol 353 (1998), pp 65-73 

95 Q 124 

96 Ev 19, para 3.5.5 
22 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

David Holland, an author of several FOIA requests that were mentioned in the leaked emails, claimed that climate scientists at CRU corrupted the IPCC process: 

The emails show that a group of influential climate scientists colluded to subvert the 

peer-review process of the IPCC and science journals, and thereby delay or prevent 

the publication and assessment of research by scientists who disagreed with the 

group’s conclusions about global warming. They manufactured pre-determined 

conclusions through the corruption of the IPCC process and deleted procedural and 

other information hoping to avoid its disclosure under freedom-of-information 

requests.97

68. In one e-mail, Professor Jones appeared to suggest that he and another scientist would 

deliberately try to “keep out” two papers from the IPCC’s Fourth Assessment Report.98

69. The second is that climate scientists tried to suppress a paper on research fraud. As Dr 

Benny Peiser, Director of the Global Warming Policy Foundation, put it: 

The CRU e-mails under investigation suggest that climate scientists (not only at 

CRU but also elsewhere) have actively sought to prevent a paper on alleged research 

fraud from being published in violation of principles of academic integrity.99

70. The third allegation is made by Dr Sonja Boehmer-Christiansen, a former peer 

reviewer for the IPCC, editor of the journal, Energy & Environment, and Reader Emeritus 

97 Ev 115, para 2 

98 www.eastangliaemails.com 

99 Ev 164, para 2 

From: Phil Jones <p.jones@xxxxxxxxx.xxx>

To: "Michael E. Mann" <mann@xxxxxxxxx.xxx> 

Subject: HIGHLY CONFIDENTIAL 

Date: Thu Jul 8 16:30:16 2004 

Mike, 

Only have it in the pdf form. FYI ONLY - don’t pass on. Relevant paras are the last 

2 in section 4 on p13. As I said it is worded carefully due to Adrian knowing Eugenia for years. 

He knows the’re wrong, but he succumbed to her almost pleading with him to tone it down as 

it might affect her proposals in the future ! 

I didn’t say any of this, so be careful how you use it - if at all. Keep quiet also that you have the 

pdf. The attachment is a very good paper - I’ve been pushing Adrian over the last weeks to get 

it submitted to JGR or J. Climate. The main results are great for CRU and also for ERA-40. The 

basic message is clear - you have to put enough surface and sonde obs into a model to produce 

Reanalyses. The jumps when the data input change stand out so clearly. NCEP does many odd 

things also around sea ice and over snow and ice. The other paper by MM is just garbage - as 

you knew. De Freitas again. Pielke is also losing all credibility as well by replying to the mad 

Finn as well - frequently as I see it. I can’t see either of these papers being in the next IPCC 

report. Kevin and I will keep Them out somehow - even if we have to redefine what the peerreview literature is ! 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 23

at Hull University, who stated in her memorandum that her journal became the focus of 

attacks from CRU scientists: 

As editor of a journal which remained open to scientists who challenged the 

orthodoxy, I became the target of a number of CRU manoeuvres. The hacked emails 

revealed attempts to manipulate peer review to E&E’s disadvantage, and showed that 

libel threats were considered against its editorial team. Dr Jones even tried to put 

pressure on my university department. The emailers expressed anger over my 

publication of several papers that questioned the ‘hockey stick’ graph and the 

reliability of CRU temperature data. The desire to control the peer review process in 

their favour is expressed several times. [...] CRU clearly disliked my journal and 

believed that “good” climate scientists do not read it.100

71. When we asked Professor Jones about these accusations, he contested each of them. 

• On the claim that he tried to keep two papers out of the IPCC report, he explained that 

the papers were already published and that “I was just commenting that I did not think 

those papers were very good”.101

• On the claim by he tried to suppress papers that alleged research fraud, he told us: 

Dr Benny Peiser [...] was editing a series of papers in Energy & Environment. He 

asked me to comment on a particular paper and I sent him some views back that I 

did not think the paper was very good. It was not a formal review, he was just asking 

me for my views.102

• On the claims made by Dr Boehmer-Christiansen, he noted: “I was sending an email to 

the head of department about a complaint that she had made about me to the UK 

Climate Impacts Programme, so I was just responding there”.103

72. In summary, Professor Jones argued: 

I do not think there is anything in those emails that really supports any view that I or 

CRU have been trying to pervert the peer review process in any way. I have just been 

giving my views on specific papers.104

73. The evidence that we have seen does not suggest that Professor Jones was trying to 

subvert the peer review process. Academics should not be criticised for making 

informal comments on academic papers. The Independent Climate Change Email 

Review should look in detail at all of these claims.

100 Ev 125, paras 4.1–4.3 

101 Q 154 

102 Q 157 

103 As above

104 Q 159 
24 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

3 Freedom of information issues 

74. We are not a tribunal reviewing whether breaches of the Freedom of Information Act 

2000 (FOIA) have taken place but see as our role in this inquiry as considering whether: 

(a) the arrangements for examining whether CRU breached FOIA are adequate; 

(b) whether the six-month time limit on the initiation of a prosecution where a 

public authority acts so as to prevent intentionally the disclosure of requested 

information needs to be revised; and 

(c) whether UEA ensured that CRU was able to meet the requirements of the 

legislation when it received FOIA requests. 

Freedom of Information legislation 

75. The FOIA creating new rights of access to information came into operation on 1 

January 2005. CRU, as part of UEA, is classed as a “public authority” for the purposes of 

the FOIA. In his submission Richard Thomas, who was Information Commissioner from 

2002 until June 2009, explained the application of the FOIA to scientific data held by UK 

universities: 

the public must be satisfied that publicly-funded universities, as with any other 

public authority in receipt of public funding, are properly accountable, adopt systems 

of good governance and can inspire public trust and confidence in their work and 

operations [...] The fact that the FOIA requests relate to complex scientific data does 

not detract from this proposition or excuse non-compliance.105

76. When he gave oral evidence, we asked Mr Thomas if the legislation drew a distinction 

between, on the one hand, scientific data and modelling and, on the other hand, 

administrative records. He replied: 

the broad answer [...] is no [...] First of all, the legislation applies to information held 

by the public authority, and information is not elaborated in that sense. [...] It is not 

ownership. The legislation uses the word “held”, and in the Environmental 

Information Regulations [EIR] that phrase “held” is slightly elaborated. If I can quote 

the regulation for you there, “It is held by a public authority if the information: (a) is 

in the authority’s possession and has been produced or received by the authority, or 

(b) is held by another person on behalf of the authority.” So that is an elaboration of 

the concept of “held”. It is not ownership.106

77. Mr Thomas considered that the issues in this case which were most relevant to the 

information law appeared to be: 

(a) the relevance and impact of the information laws on scientific and academic 

research conducted within universities; 

105 Ev 8, para 3.2 

106 Qq 59–60 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 25

(b) the adequacy of section 77 of FOIA to deal with suggestions that CRU 

researchers deleted information, not in the course of normal work, but to 

frustrate FOIA/EIR107 requests; 

(c) the handling of a large number of FOIA/EIR requests by UEA relating 

especially to climate change research which (within CRU) it “held”; and 

(d) whether this case illustrates that there is scope to extend the “proactive” 

disclosure provisions of FOIA as they relate to universities.108

78. Parliament has created a presumption in favour of disclosure but there are 

exclusions.109 Mr Thomas explained: 

There are over 20 exemptions to the fundamental duty to disclose requested 

information in FOIA.[...] Eight of the main exemptions are absolute and 16 are 

qualified. Qualified means that there is a “public interest override,” which means 

that, even where the exemption applies, the public interest considerations must be 

considered. In formal terms, there must still be disclosure—even though the 

qualified exemption applies—unless the public interest in the exemption outweighs 

the public interest in disclosure. 

Mr Thomas added that: 

The exemptions are similar to those found in other Freedom of Information laws in 

force in the world. I am not aware which exemptions were considered by the 

University as potentially applicable to some or all of the requests to CRU. I can 

speculate that some or all of the following [...] might have been considered: 

(a) Section 22—where the requested information is intended for future (but 

imminent) publication; 

(b) Section 40—where disclosure of personal data would breach any of the data 

protection principles; 

(c) Section 41—where the information had been obtained from elsewhere in such 

circumstances that its disclosure would constitute an actionable breach of 

confidence under common law; 

(d) Section 43 (qualified)—where disclosure would, or would be likely to, 

prejudice the commercial interests of any person, including the public 

authority; 

(e) Section 44—where disclosure is prohibited by another enactment or 

inconsistent with an EU obligation (which may include some intellectual 

property restrictions); and 

107 EIR: Environmental Information Regulations 2004. Deriving from European Directive 2003/4/EC these give rights of 

public access to environmental information held by public authorities. 

108 Ev 8, para 2.2 

109 Ev 9, para 3.6 
26 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

(f) Section 14 (not an exemption, strictly speaking)—where the request is 

vexatious.110

79. We were grateful to Mr Thomas for explaining the operation of the FOIA and EIR. He 

did, however, point out that he did not have detailed knowledge of events at UEA since 

leaving the Information Commissioner’s Office: 

I have no idea at all what has happened inside my former office. I cannot say because 

this is a serious matter. It depends a great deal on the circumstances of the particular 

case, the evidence. I have had no direct contact with the office as to how this case is 

being handled.111

Alleged breaches of the Freedom of Information Act 2000 

The e-mails 

80. Some of the hacked e-mails appear to reveal scientists encouraging their colleagues to 

resist disclosure and to delete e-mails, apparently to prevent them from being revealed to 

people making FOIA requests. Below are examples, in chronological order, of e-mails sent 

by Professor Jones which address FOIA and requests for information. 

E-mail: 1107454306 [Extract] 

At 09:41 AM 2/2/2005, Phil Jones wrote: 

Mike,[...]Just sent loads of station data to Scott. Make sure he documents everything better this 

time! And don’t leave stuff lying around on ftp sites - you never know who is trawling them. The 

two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of 

Information Act now in the UK, I think I’ll delete the file rather than send to anyone. Does your 

similar act in the US force you to respond to enquiries within 20 days? - our does ! The UK works 

on precedents, so the first request will test it. We also have a data protection act, which I will hide 

behind. Tom Wigley has sent me a worried email when he heard about it - thought people could 

ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR 

should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say 

we must adhere to it !. [...] 

E-mail: 1219239172 [Extract] 

From: Phil Jones <p.jones@xxxxxxxxx.xxx> 

To: Gavin Schmidt <gschmidt@xxxxxxxxx.xxx> 

Subject: Re: Revised version the Wengen paper 

Date: Wed Aug 20 09:32:52 2008 

[...] Keith/Tim still getting FOI requests as well as MOHC and Reading. All our FOI officers have 

been in discussions and are now using the same exceptions not to respond - advice they got from 

the Information Commissioner. As an aside and just between us, it seems that Brian Hoskins has 

withdrawn himself from the WG1 Lead nominations. It seems he doesn’t want to have to deal with 

 110 Ev 9, para 3.7 

111 Q 58 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 27

this hassle. 

The FOI line we’re all using is this. IPCC is exempt from any countries FOI – the Sceptics have been 

told this. Even though we (MOHC, CRU/UEA) possibly hold relevant info the IPCC is not part our 

remit (mission statement, aims etc) therefore we don’t have an obligation to pass it on. 

Cheers 

Phil 

E-mail: 1228330629 

From: Phil Jones <p.jones@xxxxxxxxx.xxx> 

To: santer1@xxxxxxxxx.xxx, Tom Wigley <wigley@xxxxxxxxx.xxx> 

Subject: Re: Schles suggestion 

Date: Wed Dec 3 13:57:09 2008 

Cc: mann <mann@xxxxxxxxx.xxx>, Gavin Schmidt <gschmidt@xxxxxxxxx.xxx>, Karl Taylor 

<taylor13@xxxxxxxxx.xxx>, peter gleckler gleckler1@xxxxxxxxx.xxx 

Ben, 

When the FOI requests began here, the FOI person said we had to abide by the requests. It took a 

couple of half hour sessions - one at a screen, to convince them otherwise showing them what CA 

was all about. Once they became aware of the types of people we were dealing with, everyone at 

UEA (in the registry and in the Environmental Sciences school - the head of school and a few 

others) became very supportive. I’ve got to know the FOI person quite well and the Chief Librarian 

- who deals with appeals. The VC is also aware of what is going on - at least for one of the requests, 

but probably doesn’t know the number we’re dealing with. We are in double figures. 

One issue is that these requests aren’t that widely known within the School. So I don’t know who 

else at UEA may be getting them. CRU is moving up the ladder of requests at UEA though - we’re 

way behind computing though. We’re away of requests going to others in the UK - MOHC, 

Reading, DEFRA and Imperial College. So spelling out all the detail to the LLNL management 

should be the first thing you do. I hope that Dave is being supportive at PCMDI. The inadvertent 

email I sent last month has led to a Data Protection Act request sent by a certain Canadian, saying 

that the email maligned his scientific credibility with his peers! 

If he pays 10 pounds (which he hasn’t yet) I am supposed to go through my emails and he can get 

anything I’ve written about him. About 2 months ago I deleted loads of emails, so have very little - if 

anything at all. This legislation is different from the FOI - it is supposed to be used to find put why 

you might have a poor credit rating ! In response to FOI and EIR requests, we’ve put up some data - 

mainly paleo data. Each request generally leads to more - to explain what we’ve put up. Every time, 

so far, that hasn’t led to anything being added - instead just statements saying read what is in the 

papers and what is on the web site! Tim Osborn sent one such response (via the FOI person) earlier 

this week. We’ve never sent programs, any codes and manuals. 

In the UK, the Research Assessment Exercise results will be out in 2 weeks time. 

These are expensive to produce and take too much time, so from next year we’ll be moving onto a 

metric based system. The metrics will be # and amounts of grants, papers and citations etc. I did 

flippantly suggest that the # of FOI requests you get should be another. 

When you look at CA, they only look papers from a handful of people. They will start on another 

coming out in The Holocene early next year. Gavin and Mike are on this with loads of others. I’ve 

told both exactly what will appear on CA once they get access to it! 

Cheers 
28 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

Phil 

E-mail: 1237496573 [Extract] 

From: Phil Jones <p.jones@xxxxxxxxx.xxx> 

To: santer1@xxxxxxxxx.xxx 

Subject: Re: See the link below 

Date: Thu Mar 19 17:02:53 2009 

[...] CRU has had numerous FOI requests since the beginning of 2007. The Met Office, Reading, 

NCDC and GISS have had as well – many related to IPCC involvement. I know the world changes 

and the way we do things changes, but these requests and the sorts of simple mistakes, should not 

have an influence on the way things have been adequately dealt with for over a century. 

Cheers 

Phil 

81. In his submission Andrew Montford stated that: 

Research materials should be made available to outsiders as a requirement of the 

scientific method. That scientists have failed to do so is reprehensible, but the fact 

that they have apparently also resorted to breaches of the Freedom of Information 

Act in order to do so requires urgent attention from policymakers.112

82. As we explained in the previous chapter, David Holland was the author of several FOIA 

requests that were mentioned in the leaked e-mails. In his submission he pointed out that 

on 9 May [2008] in e-mail 1210367056, Professor Jones sent “my formal information 

request to ‘team’ members Mann, Hughes and Ammann” writing: 

You can delete this attachment if you want. Keep this quiet also, but this is the 

person who is putting in FOI requests for all emails Keith and Tim have written and 

received re Ch 6 of AR4.113 We think we’ve found a way around this.114 

83. Mr Holland also drew attention to e-mail 1212063122 dated 29 May 2008 in which 

Professor Jones asked Professor Mann: 

Can you delete any emails you may have had with Keith re AR4? Keith will do 

likewise. Can you also email [Eu]Gene [Wahl] and get him to do the same? I don’t 

have his new email address. We will be getting Caspar [Ammann] to do likewise.115 

Correspondence with the Deputy Information Commissioner 

84. On 22 January 2010, when the Deputy Information Commissioner, Graham Smith, 

issued a statement which suggested that at l east some of the requested information should 

112 Ev 159, para 6 

113 Intergovernmental Panel on Climate Change: Fourth Assessment Report 

114 Ev 117, para 23 

115 Ev 118, para 32 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 29

have been disclosed in the absence of applicable exemptions, it gave support to the 

criticisms of CRU’s handling of FOIA requests. Mr Smith said: 

The FOI Act makes it an offence for public authorities to act so as to prevent 

intentionally the disclosure of requested information. Mr Holland’s FOI requests 

were submitted in 2007/8, but it has only recently come to light that they were not 

dealt with in accordance with the Act. The legislation requires action within six 

months of the offence taking place, so by the time the action came to light the 

opportunity to consider a prosecution was long gone.116

85. Mr Thomas commented that this was “clearly a reference to section 77 of the Act 

and/or the near-identical Regulation 19 of EIR”.117 Section 77 of the FOIA provides: 

1. Where: 

(a) a request for information has been made to a public authority, 

(b) under section 1 of this Act or section 7 of the Data Protection Act 1998, the 

applicant would have been entitled (subject to payment of any fee) to 

communication of any information in accordance with that section, 

any person to whom this subsection applies is guilty of an offence if he alters, defaces, 

blocks, erases, destroys or conceals any record held by the public authority, with the 

intention of preventing the disclosure by that authority of all, or any part, of the 

information to the communication of which the applicant would have been entitled. 

2. Subsection (1) applies to the public authority and to any person who is employed 

by, is an officer of, or is subject to the direction of, the public authority. 

3. A person guilty of an offence under this section is liable on summary conviction to 

a fine not exceeding level 5 on the standard scale.118

86. Mr Thomas added that the Deputy Commissioner also appeared “to have in mind” 

section 127(1) of the Magistrates Court Act 1980, which provides that 

a magistrates’ court shall not try an information or hear a complaint unless the 

information was laid, or the complaint made, within 6 months from the time when 

the offence was committed, or the matter of complaint arose.119

Mr Thomas confirmed in oral evidence that 

because of the interaction with the Magistrates Court Act, any prosecution must be 

brought within six months of the offence being committed.120

87. In its memorandum to our inquiry, UEA defended its actions: 

116 Ev 9, para 4.1 

117 Ev 10 

118 Ev 10, para 4.1 

119 Ev 10, para 4.2 

120 Q 56 
30 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

CRU has been accused of refusing to release data requested under the FOIA. There 

are many obstacles outside CRU’s control surrounding the release of data provided 

by NMSs [National Meteorological Services]. Many FOIA requests made to CRU 

related to primary data provided by the NMSs. Some of these data are subject to 

formal non-publication agreements between the NMS and CRU. Other primary data 

had been provided to CRU on an individual-to-individual basis, with accompanying 

verbal agreements that they may be used within the gridded dataset, but should not 

be passed on to others. CRU responded to the FOIA requests for primary data by 

pointing out that approximately 90% of the stations in the CRU dataset are available 

from other sources, particularly GHCN.121

88. On 29 January there was an exchange between UEA and Mr Smith, the Deputy 

Commissioner. Brian Summers, the Registrar and Secretary of UEA responded forcibly to 

Mr Smith’s 22 January press statement, which asserted that UEA had not dealt with FOIA 

requests “as they should have been under the legislation”.122 He did not consider it was 

“acceptable that such a statement which has led to an extremely damaging commentary on 

the University [was] first communicated to the University by a journalist”.123 His letter goes 

on to defend UEA’s actions in detail and to ask that, if the Information Commissioner’s 

Office (ICO) cannot retract the 22 January statement, it issue a clarification regarding the 

alleged breaches of the FOIA. A response from the ICO was issued the same day. It did not 

retract the original statement but offered clarification: 

1. [No] decision notice has yet been issued and no alleged breaches have yet been 

put to the University for comment. That matter has yet to be addressed, but it 

will be over coming months. 

2. The fact that the elements of a section 77 offence may have been found here, but 

cannot be acted on because of the elapsed time, is a very serious matter. The ICO 

is not resiling from its position on this. 

3. The ICO’s position is as stated in point 2 above. The statement may be read to 

indicate that.124 Under section 77, an offence may be committed by an individual, 

not necessarily the public authority itself. 

4. Errors like this are frequently made in press reports and the ICO cannot be 

expected to correct them, particularly when the ICO has not itself referred to 

penalties or sanctions in its own statement.125

121 Ev 20, para 3.7.2 

122 “Scientists in stolen e-mail scandal hid climate data”, The Times, 28 January 2010 

123 Registrar and Secretary to Deputy Information Commissioner - 29 January 2010, UEA website, Correspondence 

between University of East Anglia and the Information Commissioner’s Office, 

www.uea.ac.uk/mac/comm/media/press/CRUstatements/ICOcorrespondence 

124 UEA had asked the Deputy Commissioner to confirm that “your statement cannot be taken to mean that there has 

been a demonstrable breach of Section 77, which is a breach of the FOI which can result in prosecution”; Registrar 

and Secretary to Deputy Information Commissioner, 29 January 2010, UEA website, Correspondence between 

University of East Anglia and the Information Commissioner’s Office, 

www.uea.ac.uk/mac/comm/media/press/CRUstatements/ICOcorrespondence 

125 Deputy Information Commissioner to Registrar and Secretary - 29 January 2010, UEA website, Correspondence 

between University of East Anglia and the Information Commissioner’s Office, 

www.uea.ac.uk/mac/comm/media/press/CRUstatements/ICOcorrespondence 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 31

89. UEA responded on 1 February thanking the ICO for the clarification but setting out its 

concerns relating to the press coverage of the ICO’s original statement: 

Your clarification that the press cannot infer from your statement to the Sunday 

Times that it has been established that the University (or indeed any individual 

associated with the University) has breached the terms of the Freedom of 

Information Act is welcome. [UEA’s] reputation which has been subjected to these 

damaging and incorrect assertions claiming to be based on your statement and we 

must take some steps to put this right. We will be writing to the media which carried 

reports based on your statement, pointing out the inaccuracies and asking them to 

rectify the position.126

90. In his oral evidence Professor Acton questioned the ICO statement of 22 January: 

our principle is that prima facie evidence is evidence which on the face of it and 

without investigation suggests that there is a case to answer. To my mind if there is 

prima facie evidence; why did I set up the Muir Russell independent review? Prima 

facie evidence is not the same as, you have been found to breach. [...] If it is sub 

judice, if, as we had in the letter ten days ago from the ICO, the investigation has not 

even begun, I am puzzled how we could have been found to breach if there has been 

no investigation.127

91. The ICO’s most recent letter, dated 3 March, in UEA’s view, “makes plain that there is 

no assumption by the ICO, prior to investigation, that UEA has breached the Act; and that 

no investigation has yet been completed.”128 The ICO’s letter confirmed that the “ICO is 

not pursuing any investigation under section 77 of the Act. That matter is closed as far as 

the ICO is concerned, given the statutory time limits for action”. It added that: 

The ICO acknowledges your concern about the statement made and the subsequent 

media and blog reports. Given that the Deputy Commissioner has already been 

publicly associated with the matter, any Decision Notice will be reviewed and signed 

off by another authorised signatory.129

We regret that the ICO made a statement to the press that went beyond that which it 

could substantiate and that it took over a month for the ICO properly to put the record 

straight. We recommend that the ICO develop procedures to ensure that its public 

comments are checked and that mechanisms exist to swiftly correct any mis-statements 

or misinterpretations of such statements.

92. The disclosed e-mails appear to show a culture of non-disclosure at CRU and instances 

where information (disclosable or otherwise) may have been deleted, to avoid disclosure. 

The Deputy Information Commissioner’s letter of 29 January gives a clear indication that a 

126 Registrar and Secretary to Deputy Information Commissioner - 1 February 2010, UEA website, Correspondence 

between University of East Anglia and the Information Commissioner’s Office, 

www.uea.ac.uk/mac/comm/media/press/CRUstatements/ICOcorrespondence 

127 Q130 

128 Ev 39, para A 

129 Ev 39, annex 
32 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

breach of the FOIA may have occurred but that a prosecution was time-barred.130 As, 

however, UEA pointed out, no investigation has been carried out. 

93. It seems to us that both sides have a point. There is prima facie evidence that CRU has 

breached the Freedom of Information Act 2000. It would, however, be premature, 

without a thorough investigation affording each party the opportunity to make 

representations, to conclude that UEA was in breach of the Act. In our view, it is 

unsatisfactory to leave the matter unresolved simply because of the operation of the sixmonth time limit on the initiation of prosecutions. Much of the reputation of CRU 

hangs on the issue. We conclude that the matter needs to be resolved conclusively—

either by the Independent Climate Change Email Review or by the Information 

Commissioner.

94. On the question of the six-month time limit on the initiation of prosecutions, Mr 

Thomas pressed for a revision of the law. He pointed out that apart from in the most 

blatant cases “it will usually be impossible for the ICO to detect an offence within 6 months 

of its occurrence” and thus to be able to initiate a prosecution.131 He drew attention to a 

recent debate in the House of Lords on a proposal to amend the time limit. In reply, in the 

debate the Parliamentary Under-Secretary of State at the Ministry of Justice said that: 

The Freedom of Information Act 2000 came into force only in 2005, and [...] we have 

no evidence at present that the current six-month time limit presents a systemic 

problem for the Information Commissioner or any other prosecutor in taking action 

under Section 77. [...] We will listen to the views of the Information Commissioner 

and other interested parties on this point, and if there is evidence that the current 

legislation is causing systemic difficulties, we will look for ways to address the matter, 

if necessary by means of an alternative legislative vehicle in the future. However, I 

cannot go further than that today on behalf of the Government.132

No change was made to the legislation. 

95. We consider that events at CRU throw light on the operation of the Freedom of 

Information Act 2000 and, in particular, whether there is a need to amend the time limit 

on prosecutions from six months from the time the alleged offence was committed. If the 

Minister was correct to assert in July 2009 that the Government had no evidence that 

the current six-month time limit presents a systemic problem, then it is now clear that 

such evidence exists. Irrespective of whether or not CRU breached the Freedom of 

Information Act 2000, we recommend that the Government review the operation of 

section 77 of the 2000 Act and the six month limit on the initiation of prosecutions 

provided by section 127(1) of the Magistrates Court Act 1980.

130 UEA website, Correspondence between University of East Anglia and the Information Commissioner’s Office, 

www.uea.ac.uk/mac/comm/media/press/CRUstatements/ICOcorrespondence 

131 Ev 10, para 4.3 

132 HL Deb, 21 July 2009, col 1571 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 33

Volume of requests 

96. In the face of allegations of poor handling of FOIA requests, one of the explanations 

offered by UEA was that in: 

July 2009 UEA received an unprecedented, and frankly administratively 

overwhelming, deluge of FOIA requests related to CRU. These amounted to 61 

requests out of a 2009 total of 107 related to CRU, compared to annual totals of 2 in 

2008 and 4 in 2007 (University totals for those years were 204, 72 and 44 

respectively).133

97. At the oral evidence session Lord Lawson commented on the increase in the volume of 

FOIA requests: 

what had happened was there had been a very, very small number of FOI Act 

requests to begin with and it was in response to those that there was all the evasion, 

the lack of disclosure and all the other things which we have seen in the emails: 

discussions about possibly destroying evidence and so on. All that came well before 

the 2009 flood of stuff. The 2009 flood, if you look at the sequence of events, was a 

response to the refusal to give disclosure of various things before. That was what 

came first.134

98. There are two issues here: the adequacy of CRU’s handling of the FOIA requests and 

whether the increase in the number of requests in July 2009 was a deluge. On the latter, Mr 

Thomas said that, whilst agreeing that UEA had faced a significant rise in FOIA requests in 

July 2009, he did not consider that a total of 61 was a “huge number”.135 

99. On handling, CRU claimed that it could not cope with the significant rise in FOIA 

requests because it only had three full-time academic staff.136 We therefore wrote to UEA 

on 2 March 2010 to ask what extra resources were provided to assist CRU cope with these 

requests. UEA responded that: 

additional support was provided to the University’s Information Policy Compliance 

Manager (IPCM) who handles FOI requests. This included rescheduling workloads 

to allow him to concentrate on the CRU FOI requests and diverting secretarial 

support to provide additional resource. Given the high volume of requests received, 

the Director of Information Services (DoIS) also took an active role in the first stage 

of a number of requests, thus providing additional support to the IPCM. (Should any 

cases where the DoIS was directly involved in the first stage be appealed then we have 

arranged for the PVC Academic to adjudicate to ensure impartiality). ISD also fasttracked the merging of the Security Policy and Compliance team to ensure that a 

fully trained back-up to the IPCM was available.137 

133 Ev 20, para 3.7.4 

134 Q 9 

135 Q 68 

136 Q 92 [Professor Acton], Ev 20, para 3.7.4; Ev 37, Q 1 

137 Ev 37, para 1 
34 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

100. The Science Faculty also provided additional administrative support, including that of 

the Director of Faculty Administration, the most senior member of the Faculty’s 

administrative staff. UEA pointed out that many of the requests were of a very technical 

nature and: 

required scientific knowledge and understanding of the subject area in order to 

provide the details. Despite the additional administrative resources provided, the 

requirement to respond to the 61 requests received in July 2009 impacted 

considerably upon the work of CRU.138

101. We also asked UEA to outline what legal advice and guidance on handling had been 

offered to CRU in handling these FOIA requests. UEA confirmed that the: 

IPCM provided advice to CRU on the requirements of the Act both generally, and in 

relation to any applicable sections, exemptions or exceptions pertaining to the 

specific request. In this latter role, the IPCM set out the requirements of any possible 

exemption or exception, inclusive of the public interest test, and elicited from CRU 

staff whether the public interest test had been met. Additional advanced training was 

provided to the ‘FOI Contact’ for the Faculty of Science, the Director of Faculty 

Administration. In this role, the FOI contact acted as a support to CRU in the 

location and retrieval of information and provided assistance to the IPCM in 

exploring the application of the Act to the specific requests.139

102. On the evidence we took we have concerns about the handling of FOIA requests by 

CRU. First, the disclosed e-mails betray an attitude to freedom of information that was 

antipathetic to the spirit of disclosure in the legislation. Mr Thomas pointed out that: 

the simplest approach, particularly where requests tend to generate either a defensive 

attitude or place a great burden on the public authority, is proactive disclosure in the 

first place.[...] Public authorities ought to decide what really has to be kept away from 

the public. If it is particularly sensitive or there is a good reason for withholding it, 

fair enough, but where there is no good reason for withholding information, then 

why not proactively disclose it and avoid the hassle of large numbers of requests?140

103. Whether or not CRU liked it, those making FOIA requests were entitled to have their 

requests dealt with in accordance with the legislation and, if the information sought did not 

fall within one of the exclusions provided by the FOIA, it should have been disclosed. We 

have already recommended in paragraph 54 above that in future information, 

including data and methodology, should be published proactively on the internet 

wherever possible. However, a culture of withholding information—from those 

perceived by CRU to be hostile to global warming—appears to have pervaded CRU’s 

approach to FOIA requests from the outset. We consider this to be unacceptable.

104. In the face of such an unhelpful approach we are not surprised that FOIA requests 

multiplied. When the surge in FOIA requests hit CRU in July 2009 UEA provided extra 

138 Ev 37, para 1 

139 Ev 37, para 2 

140 Q 70 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 35

resources but because of their technical nature the same small group of staff at CRU had a 

pivotal role in handling the requests. We are not clear that the culture changed. We cannot 

reach a firm conclusion on the basis of the evidence we took but we must put on record 

our concern about the manner in which UEA allowed CRU to handle FOIA requests. 

Further, we found prima facie evidence to suggest that the UEA found ways to support 

the culture at CRU of resisting disclosure of information to climate change sceptics. 

The failure of UEA to grasp fully the potential damage to CRU and UEA by the nondisclosure of FOIA requests was regrettable. UEA needs to review its policy towards 

FOIA and re-assess how it can support academics whose expertise in this area is 

limited. 
36 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

4 Independent inquiries 

105. There are two reviews underway: the Independent Climate Change Email Review led 

by Sir Muir Russell; and a scientific assessment panel reviewing CRU’s key scientific 

publications. The Vice-Chancellor explained to us in oral evidence on 1 March 2010 that 

the reviews would focus on different matters: 

Muir Russell’s independent review is not looking at the science, it is looking at 

allegations about malpractice. As for the science itself, I have not actually seen any 

evidence of any flaw in the science but I am hoping, later this week, to announce the 

chair of a panel to reassess the science and make sure there is nothing wrong.141 

In the event the announcement was not made until 22 March. 

The Independent Climate Change Email Review 

106. The Independent Climate Change Email Review is being conducted by a team, led by 

Sir Muir Russell. According to the Review’s website the team has more than 100 years’ 

collective expertise of scientific research methodology and a wide range of scientific 

backgrounds. None have any links to the Climatic Research Unit, or the United Nations’ 

Intergovernmental Panel on Climate Change (IPCC).142

Terms of reference 

107. The Review’s terms of reference are as follows: 

The Independent Review will investigate the key allegations that arose from a series 

of hacked e-mails from the University of East Anglia’s Climatic Research Unit 

(CRU). The review will: 

1.1. Examine the hacked e-mail exchanges, other relevant e-mail exchanges and any 

other information held at CRU to determine whether there is any evidence of the 

manipulation or suppression of data which is at odds with acceptable scientific 

practice and may therefore call into question any of the research outcomes. 

1.2. Review CRU’s policies and practices for acquiring, assembling, subjecting to 

peer review and disseminating data and research findings, and their compliance or 

otherwise with best scientific practice. 

1.3. Review CRU’s compliance or otherwise with the University’s policies and 

practices regarding requests under the Freedom of Information Act (‘the FOIA’) 

and the Environmental Information Regulations (‘the EIR’) for the release of data. 

141 Q 129 

142 www.cce-review.org/About.php 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 37

1.4. Review and make recommendations as to the appropriate management, 

governance and security structures for CRU and the security, integrity and release 

of the data it holds.143

108. Sir Muir has discretion to amend or add to the terms of reference if he feels necessary, 

devise his own methods of working, and call on appropriate expertise, in order to 

investigate the allegations fully. UEA has asked for the Review to be completed by Spring 

2010 and this will be made public along with UEA’s response.144

109. Lord Lawson, in both his written submission and his oral evidence, considered that 

the terms of reference “may be a bit too CRU-centric”145 and “needed to be extended to 

include more fully the issue of the dissenting scientists”.146 These points were echoed in 

written submissions to us. Andrew Montford suggested that: 

The independence of the review is not assured. Sir Muir Russell was appointed to 

head the review by the vice-chancellor of the University of East Anglia, [...] Edward 

Acton. However, the emails disclosed implicate [his] predecessor in an apparent 

breach of the Freedom of Information Act and there is therefore a prime-facie case 

that the review is not sufficiently independent. [...] The review must take evidence 

from sceptics. At time of writing it appears that no prominent sceptic has been 

contacted by Sir Muir with a view to providing evidence. Without complainants 

being able to make their case to the review, it is unlikely that the findings will be 

sound or accepted by the sceptic community.147

Mike Haseler, creator of the Number 10 Petition regarding the CRU, was also critical of the 

Review saying that it “seems to serve no real purpose except the PR of the University to 

appear to be doing something.”148 

110. Others offered amendments to the terms of reference. Professor Ross McKitrick, a 

professor of environmental economics, recommended that the terms of reference “should 

consider whether CRU scientists whose responsibilities include providing climate data to 

the IPCC should not serve as IPCC Lead Authors (or Coordinating Lead Authors) on any 

Report or Chapter that assesses evidence for or against its quality for climatic research 

purposes.”149 

111. The Royal Society of Chemistry considered the terms of reference “adequate”150 and 

Professor John Beddington suggested that they “give sufficient scope for the issue to be 

investigated in full”.151 Professor Peter Cox, a former lead author on the last IPCC Working 

143 Ev 39 

144 “Sir Muir Russell to head the Independent Review into the allegations against the Climatic Research Unit (CRU)” 

UEA Press Release, 3 December 2009, www.uea.ac.uk/mac/comm/media/press/2009/dec/CRUreview 

145 Q 5, Ev 1, annex containing letter dated 26 January 2010 from the Foundation to Sir Muir Russell (not printed) 

146 Q 3 

147 Ev 161, paras 22 and 24 

148 Ev 139, para 27 

149 Ev 140, para 3.2 

150 Ev 172, para 12 

151 Ev 45, para 7 
38 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

Group, suggested that the “Inquiry should hear evidence on the reviewing of scientific 

papers and the exclusion of papers from the IPCC report. It will be critical to determine 

whether these decisions were carried out on the basis of scientific merit alone”.152

112. In response to criticisms Sir Muir pointed out that the review “is not actually about 

the big science of global warming and making forecasts for the next hundred years”.153 He 

said that “it will not be window dressing”, and UEA had “not interfered at all”.154

113. We accept the assurances that Sir Muir Russell has given about the independence 

of the Independent Climate Change Email Review and we expect him to be scrupulous 

in preserving its impartiality. We see no reason why the Review’s conclusions and 

UEA’s response have to be published together. Indeed, it could give the impression that 

UEA was being given an advantage when it comes to responding. We consider that the 

Review’s conclusions and recommendations should not be conveyed to UEA in advance 

of publication. 

114. With regards to the terms of reference of the Review, we consider that as well as 

measuring CRU against current acceptable scientific practice, the Review should also 

make recommendations on best practice to be followed by CRU in the future. We invite 

Sir Muir Russell to respond formally to our Report to the extent that he sets out 

whether, on the basis of its contents, he finds the Terms of Reference of his inquiry 

need to be changed.

The Review team 

115. The Review Team membership, as announced, consisted of: 

Sir Muir Russell 

Professor Geoffrey Boulton 

Dr Philip Campbell [subsequently resigned] 

Professor Peter Clarke 

Mr David Eyton 

Professor Jim Norton.155

116. Sir Muir and the Review team held a press briefing at the Science Media Centre in 

London on 11 February 2010 to announce its membership, publish its workplan and issue 

a call for submissions from interested parties. Almost immediately it was beset by claims of 

partiality. On the same day as the launch Sir Muir Russell accepted the resignation of Dr 

Philip Campbell, Editor of Chief of Nature, after a recording of an interview given by Dr 

Campbell to China Radio International in December 2009 was alleged to raise doubts over 

his impartiality. Dr Campbell said: 

I made the remarks in good faith on the basis of media reports of the leaks. As I have 

made clear subsequently, I support the need for a full review of the facts behind the 

152 Ev 132, para 2 

153 Q 163 

154 Q 166 

155 Ev 40 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 39

leaked e-mails. There must be nothing that calls into question the ability of the 

independent Review to complete this task, and therefore I have decided to withdraw 

from the team.156

117. Sir Muir said “I have spoken to Philip Campbell, and I understand why he has 

withdrawn. I regret the loss of his expertise, but I respect his decision."157 Further 

allegations arose on 12 February that Professor Geoffrey Boulton’s background and views 

affected his ability to be a member of the Review.158 These have been rejected by Sir Muir 

Russell and by Professor Boulton. Professor Boulton said: 

At the Review press conference (on February 11), I pointed out that I had worked 

full-time in the School of Environmental Sciences at UEA from its inception in 1968 

to 1980, and that I had a part-time appointment between 1980 and 1986, whilst 

working primarily in the University of Amsterdam. Since then, I have had no 

professional contact with the University of East Anglia or the Climatic Research 

Unit. I was equally clear that although my research is not in the field of modern or 

recent climate change, I am familiar with its scientific basis and uncertainties 

surrounding it. I declared my current view of the balance of evidence: that the earth 

is warming and that human activity is implicated. These remain the views of the vast 

majority of scientists who research on climate change in its different aspects. They 

are based on extensive work worldwide, not that of a single institution. As a sceptical 

scientist, I am prepared to change those views if the evidence merits it. They certainly 

do not prevent me from being heavily biased against poor scientific practice, 

wherever it arises.159

Sir Muir Russell said: 

This Review must determine if there is evidence of poor scientific practice, as well as 

investigate allegations around the manipulation and suppression of data. As others 

have pointed out, it would be impossible to find somebody with the qualifications 

and experience we need who has not formed an opinion on climate change. I am 

completely confident that each member of the Review team has the integrity, the 

expertise, and the experience to complete our work impartially.160

118. In his oral evidence Sir Muir outlined his approach in choosing the team: 

156 “Dr Philip Campbell withdraws from the Review”, Independent Climate Change Email Review News release, 12 

February 2010, www.cce-review.org/News.php 

157 As above

158 There has been pressure on Professor Boulton to step down. The Scotsman reported: “Dr Benny Peizer, [sic] director 

of the Global Warming Policy Foundation, a think tank which claims the debate on climate change has become 

distorted, called for Prof Boulton to step down, too. He said: ‘Prof Boulton obviously is a very distinguished 

geologist. The problem is, he is a very outspoken campaigner on this issue and he’s given talks calling for 

galvanising public opinion. He also worked at the very institution that he is now going to be investigating. That, we 

think, is a conflict of interest.’” ( “Senior Scots scientist in climate probe row”, The Scotsman, 13 February 2010) Sir 

Muir has rejected the call. (“Allegations of bias against Review member rejected”, Independent Climate Change 

Email Review News release, 15 February 2010) 

159 “Allegations of bias against Review member rejected”, Independent Climate Change Email Review News release, 15 

February 2010,www.cce-review.org/News.php 

160 As above
40 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

You can see as you look at the composition of the team that I needed to be looking at 

climate science in general but not somebody who was associated with this particular 

stream of work but would understand what was going on. There were going to be 

huge data handling issues, there was a lot of work on computing and data security 

and so on and that the work was going to have a resonance out there in the real 

world and around the world. Really on that basis I came up with this set of names 

that you can see. In relation to Dr Campbell, the others that I had got together 

thought that it would be extremely important to have somebody who knew about 

peer review and that was really the qualification that brought him in.161

119. It is unfortunate that the Independent Review got off to a bad start with the 

necessary resignation of Dr Campbell. The question of the operation of peer review is 

going to be a critical issue in the inquiry and the Review Team needs to take steps to 

ensure the insight and experience he would have brought are replaced.

Transparency 

120. Contributors to our inquiry have suggested the importance that the Independent 

Review is open and transparent. Lord Lawson, in his oral evidence, said that he was: 

concerned about the openness and transparency, [...] there should be public 

hearings, like you are having here—I think that is very, very important—and I regret 

the fact that it appears that they do not intend to do this.162

Andrew Montford commented: 

The review must be held in public. Sir Muir Russell has stated that he wants to retain 

the confidence of global warming sceptics. However, in his letter to Mr Willis of 10 

December 2009, [...] the vice-chancellor of UEA, states that Sir Muir will present his 

findings to [him], who will in turn present a report to the council of the university. 

We are asked to believe that Sir Muir will properly investigate [the ViceChancellor’s] role in the alleged FoI breaches, and that [he] will pass on the findings 

that Sir Muir makes on this subject to the university council.163

121. When answering our question on transparency Sir Muir indicated that the Review 

team “plans to put on its website the evidence that we receive”.164 When pressed on the 

question of holding public evidence sessions Sir Muir responded that: 

all my predispositions and those of the fellow team members are to do it that way 

[via written evidence] rather than to do it in a hearing of perhaps this kind or in a 

series of one-to-one interviews or whatever. Where we have interviews with people 

in CRU or elsewhere, those will be written up and they will be part of the record but 

at the moment I am not really sure that getting to the stage of putting people in a 

161 Q 160 

162 Q 3 

163 Ev 161, para 23 

164 Q 172 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 41

hearing context is going to be a particularly effective way of adding value to the 

objective evidence that we want to get our hands on.165 

122. We agree that the Review must be open and transparent. We conclude that, when the 

Independent Review holds oral hearings or interviews, they should be carried out in 

public wherever possible and that it should publish all the written evidence it receives 

on its website as soon as possible.

Scientific Appraisal Panel 

123. In its evidence to us the Independent Climate Change Email Review stated that its 

remit does not invite it to re-appraise the scientific work of CRU. That re-appraisal is being 

separately commissioned by UEA, with the assistance of the Royal Society.166 In a 

statement released on 11 February UEA said that: 

The Royal Society will assist the University in identifying assessors with the requisite 

expertise, standing and independence. “Published papers from CRU have gone 

through the rigorous and intensive peer review process which is the keystone for 

maintaining the integrity of scientific research,” said Professor Trevor Davies, the 

University’s Pro-Vice-Chancellor for Research, Enterprise and Engagement. “That 

process and the findings of our researchers have been the subject of significant 

debate in recent months. Colleagues in CRU have strenuously defended their 

conduct and the published work and we believe it is in the interests of all concerned 

that there should be an additional assessment considering the science itself.” 

The independent reassessment will complement Sir Muir Russell’s Review of the key 

allegations about the handling of data arising from the publication of a series of emails hacked from CRU. Sir Muir’s Review is expected to announce its finding in 

Spring 2010. 

The reassessment of CRU’s key publications will be completed at the earliest date the 

assessors can manage. The findings will be made public.167

124. Details of the panel were announced on 22 March. It will be headed by Lord Oxburgh. 

His appointment was made on the recommendation of the Royal Society, which was also 

consulted on the choice of the six scientists on the panel: Professor Huw Davies, Professor 

of Physics at the Institute for Atmospheric and Climate Science at ETH Zürich; Professor 

Kerry Emanuel, Professor of Meteorology at Massachusetts Institute of Technology; 

Professor Lisa Graumlich, Director of the School of Natural Resources and the 

Environment at The University of Arizona; Professor David Hand, Professor of Statistics in 

the Department of Mathematics at Imperial College; Professor Herbert Huppert, Professor 

of Theoretical Geophysics at the University of Cambridge; and Professor Michael Kelly, 

Prince Philip Professor of Technology at the University of Cambridge. The panel will have 

165 Q 176 

166 Ev 40, para 4 

167 UEA, 11 February 2010, 

www.uea.ac.uk/mac/comm/media/press/CRUstatements/New+scientific+assessment+of+climatic+research+publicatio

ns+announced 
42 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

access to any publications or materials it requests, and all information considered will be 

listed in the Report. UEA, in consultation with the Royal Society, has suggested that the 

panel looks in particular at key publications, from the body of CRU’s research referred to 

in the UEA submission to our inquiry. According to the announcement on 22 March, the 

panel will meet in Norwich in April and will have the opportunity to see original data and 

speak to those who did the work and it comprises of scientists who use techniques similar 

to those used in CRU but who largely apply them to other areas of research, as well as those 

with experience in climate or related research.168 

125. Announcing the Panel, Professor Trevor Davies, UEA’s Pro-Vice-Chancellor for 

Research, said that: 

Our concern has been to bring together a distinguished group of independent 

scientists who understand the difference between assertion and evidence, and are 

familiar with using the latter to judge the validity of conclusions arising from science 

research. The panel members have the right mix of skills to understand the complex 

nature of climate research and the discipline-based expertise to scrutinise CRU’s 

research. How they do this will be entirely down to the panel. 

The choice of scientists is sure to be the subject of discussion, and experience would 

suggest that it is impossible to find a group of eminent scientists to look at this issue 

who are acceptable to every interest group which has expressed a view in the last few 

months. Similarly it is unlikely that a group of people who have the necessary 

experience to assess the science, but have formed no view of their own on global 

warming, could be found.169

Public view of the climate science 

126. There is no doubt that the e-mail disclosure from CRU in November 2009, and 

especially the extensive media coverage that has followed it ever since, has affected the 

general public view of climate science, both in the UK and further afield. Professor Bob 

Watson, Defra’s Chief Scientific Adviser, told us that “the media has certainly portrayed 

the UEA issue as a crisis, so I think to the public it has been portrayed as a crisis”.170

Professor Peter Cox, a climate scientist and a lead-author on the last IPCC171 Working 

Group, in his written submission to us, said as much: “I am concerned that public 

confidence in the science of climate change has been undermined by the email leak”.172 In 

its submission the Royal Society of Chemistry said that the: 

true nature of science dictates that research is transparent and robust enough to 

survive scrutiny. A lack of willingness to disseminate scientific information may infer 

that the scientific results or methods used are not robust enough to face scrutiny, 

even if this conjecture is not well-founded. This has far-reaching consequences for 

168 “CRU Scientific Assessment Panel announced”, UEA Press Release, 22 March 2010, 

www.uea.ac.uk/mac/comm/media/press/CRUstatements/SAPannounce 

169 As above 

170 Q 198 

171 Intergovernmental Panel on Climate Change 

172 Ev 132, para 1 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 43

the reputation of science as a whole, with the ability to undermine the public’s 

confidence in science.173

127. The majority of submissions submitted to our inquiry has been from those who stated 

that the disclosed e-mails confirmed their worries that the climate change orthodoxy has 

serious flaws and the actions of CRU seriously impugned the integrity of climate change 

research.174 A representative example was the memorandum from Dr Phillip Bratby, “a 

semi-retired energy consultant”, who said that having examined the disclosures: 

It is concluded that over at least a period of 20 years, climate science has been 

seriously compromised by the actions of a small group of scientists who have 

attempted to control the debate about climate change. The effects of this are 

potentially profound. For example a generation of work may have been corrupted 

and may be unreliable. A generation of students may have been corrupted and their 

work may be unreliable.175 

128. Others offered a different perspective. Dr Timothy Osborn, a full-time member of 

staff at CRU, defended CRU: 

It is impossible to draw firm conclusions from the hacked documents and emails. 

They do not represent the complete record, and they are not a random selection 

from the complete record. They are clearly selected with a purpose in mind and it is 

easy for people to fall into the traps set by those who did the selection.176

129. Beyond CRU, Professor Hans von Storch and Dr Myles Allen, professional statistical 

climatologists, agreed that the publication of the hacked e-mails had initiated an intense 

debate about the credibility of climate science and that “unfortunately, this debate 

sometimes goes so far as to question a key result of climate science”,177 and the 

language used in some of these e-mails has created concern, among both scientists 

and the public, about the openness and integrity of the scientific process. But at the 

same time it is critical to point out that no grounds have arisen to doubt the validity 

of the thermometer-based temperature record since 1850, nor any results based 

upon it.178

130. We put the concerns about the threat to the reputation of science to the fifth panel 

who gave oral evidence: Professor John Beddington, Government Chief Scientific Adviser, 

Professor Julia Slingo, Chief Scientist, Met Office, and Professor Bob Watson, Chief 

Scientist, Department for Environment, Food and Rural Affairs. Professor Beddington did 

173 Ev 171, para 4 

174 For examples, see Ev 68 [Richard S Courtney]; Ev 77 [Walter Radtke]; Ev 78 [Geoffrey Sherrington]; and Ev 93 [Clive 

Menzies] 

175 Ev 92, para 21 

176 Ev 130, para 3 

177 Ev 172, para 1 

178 As above
44 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

not consider that “UK science has been damaged”.179 The Met Office, in its written 

submission stated that 

the UK enjoys a reputation for strong and robust science on the international stage. 

In the field of climate research the Met Office is widely acknowledged as world 

leading.180 

Professor Slingo confirmed in oral evidence that she has “absolute confidence in the 

science that we produce at the Met Office”,181 and Professor Watson, looking at the wider 

situation, attested that “there is absolutely no adverse effect on any of the conclusions of the 

IPCC.”182 

131. In our view, reputation has to be built on the solid foundation of excellent, peerreviewed science. The review of the science to be carried out by the Scientific Appraisal 

Panel, which UEA announced on 22 March, should determine whether the work of 

CRU has been soundly built and it would be premature for us to pre-judge that review. 

132. Reputation does not, however, rest solely on the quality of work as it should. It also 

depends on perception. It is self-evident that the disclosure of CRU e-mails has 

damaged the reputation of UK climate science and, as views on global warming have 

become polarised, any deviation from the highest scientific standards will be pounced 

on. As we explained in chapter 2, the practices and methods of climate science are a key 

issue. If the practices of CRU are found to be in line with the rest of climate science, the 

question would arise whether climate science methods of operation need to change. In 

this event we would recommend that the scientific community should consider 

changing those practices to ensure greater transparency.

Need for a single review 

133. The final issue is whether the best interests of science are served by having two reviews 

or inquiries. We found this difficult to evaluate as details of the Scientific Appraisal Panel 

were released in a late stage in our inquiry. When we asked Sir Muir whether it would be 

better to have a single inquiry, he responded: 

It would have been possible, obviously, to have constructed an inquiry that looked at 

both aspects of that, and that was not what I was asked to do. Whether I would have 

been the right person to be asked to do it I do not know but certainly it obviously 

became clear to the Vice Chancellor that there was this different issue about the 

confidence that one should have not in all the methodological and handling issues 

but in the higher level set of conclusions about what was actually happening.183

134. The process of two reviews or inquiries is underway. In our view there is the potential 

for overlap between the two inquiries—for example, the question of the operation of peer 

179 Q 194 

180 Ev 46, para 1 

181 Q 197 

182 Q 198 

183 Q 181 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 45

review needs to examine both methodology and quality of the science subject to review. 

The two reviews or inquiries need to map their activities to ensure that there are no 

unmanaged overlaps or gaps. If there are, the whole process could be undermined.
46 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

5 Conclusions 

135. Consideration of the complaints and accusations made against CRU has led us to 

three broad conclusions. 

136. Conclusion 1 The focus on Professor Jones and CRU has been largely 

misplaced. On the accusations relating to Professor Jones’s refusal to share raw data 

and computer codes, we consider that his actions were in line with common practice in 

the climate science community. We have suggested that the community consider 

becoming more transparent by publishing raw data and detailed methodologies. On 

accusations relating to Freedom of Information, we consider that much of the 

responsibility should lie with UEA, not CRU. 

137. Conclusion 2 In addition, insofar as we have been able to consider accusations 

of dishonesty—for example, Professor Jones’s alleged attempt to “hide the decline”—

we consider that there is no case to answer. Within our limited inquiry and the evidence 

we took, the scientific reputation of Professor Jones and CRU remains intact. We have 

found no reason in this unfortunate episode to challenge the scientific consensus as 

expressed by Professor Beddington, that “global warming is happening [and] that it is 

induced by human activity”.184 It was not our purpose to examine, nor did we seek 

evidence on, the science produced by CRU. It will be for the Scientific Appraisal Panel 

to look in detail into all the evidence to determine whether or not the consensus view 

remains valid. 

138. Conclusion 3 A great responsibility rests on the shoulders of climate science: to 

provide the planet’s decision makers with the knowledge they need to secure our future. 

The challenge that this poses is extensive and some of these decisions risk our standard 

of living. When the prices to pay are so large, the knowledge on which these kinds of 

decisions are taken had better be right. The science must be irreproachable.

184 Q 191 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 47

Conclusions and recommendations 

Datasets 

1. We recognise that some of the e-mails suggest a blunt refusal to share data, even 

unrestricted data, with others. We acknowledge that Professor Jones must have 

found it frustrating to handle requests for data that he knew—or perceived—were 

motivated by a desire simply to seek to undermine his work. But Professor Jones’s 

failure to handle helpfully requests for data in a field as important and controversial 

as climate science was bound to be viewed with suspicion. He was obviously 

frustrated by other workers in the field trying to “undermine” his work, but his 

actions were inevitably counterproductive. Professor Jones told us that the published 

e-mails represented only “one tenth of 1%” of his output, which amounts to one 

million e-mails, and that we were only seeing the end of a protracted series of e-mail 

exchanges. We consider that further suspicion could have been allayed by releasing 

all the e-mails. In addition, we consider that had the available raw data been available 

online from an early stage, these kinds of unfortunate e-mail exchanges would not 

have occurred. In our view, CRU should have been more open with its raw data and 

followed the more open approach of NASA to making data available. (Paragraph 38) 

2. We are not in a position to set out any further the extent, if any, to which CRU 

should have made the data available in the interests of transparency, and we hope 

that the Independent Climate Change Email Review will reach specific conclusions 

on this point. However, transparency and accountability are of are increasing 

importance to the public, so we recommend that the Government reviews the rules 

for the accessibility of data sets collected and analysed with UK public money. 

(Paragraph 39) 

3. We note that the research passed the peer review process of some highly reputable 

journals. However, we note that CRU could have been more open at that time in 

providing the detailed methodological working on its website. We recommend that 

all publicly funded research groups consider whether they are being as open as they 

can be, and ought to be, with the details of their methodologies. (Paragraph 45) 

4. We therefore conclude that there is independent verification, through the use of 

other methodologies and other sources of data, of the results and conclusions of the 

Climate Research Unit at the University of East Anglia. (Paragraph 49) 

5. Even if the data that CRU used were not publicly available—which they mostly are—

or the methods not published—which they have been—its published results would 

still be credible: the results from CRU agree with those drawn from other 

international data sets; in other words, the analyses have been repeated and the 

conclusions have been verified. (Paragraph 51) 

6. It is not standard practice in climate science and many other fields to publish the raw 

data and the computer code in academic papers. We think that this is problematic 

because climate science is a matter of global importance and of public interest, and 

therefore the quality and transparency of the science should be irreproachable. We 
48 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

therefore consider that climate scientists should take steps to make available all the 

data used to generate their published work, including raw data; and it should also be 

made clear and referenced where data has been used but, because of commercial or 

national security reasons is not available. Scientists are also, under Freedom of 

Information laws and under the rules of normal scientific conduct, entitled to 

withhold data which is due to be published under the peer-review process. In 

addition, scientists should take steps to make available in full their methodological 

workings, including the computer codes. Data and methodological workings should 

be provided via the internet. There should be enough information published to allow 

verification. (Paragraph 54) 

7. Critics of CRU have suggested that Professor Jones’s use of the word “trick” is 

evidence that he was part of a conspiracy to hide evidence that did not fit his view 

that recent global warming is predominately caused by human activity. The balance 

of evidence patently fails to support this view. It appears to be a colloquialism for a 

“neat” method of handling data. (Paragraph 60) 

8. Critics of CRU have suggested that Professor Jones’s use of the words “hide the 

decline” is evidence that he was part of a conspiracy to hide evidence that did not fit 

his view that recent global warming is predominantly caused by human activity. That 

he has published papers—including a paper in Nature—dealing with this aspect of 

the science clearly refutes this allegation. In our view, it was shorthand for the 

practice of discarding data known to be erroneous. We expect that this is a matter 

the Scientific Appraisal Panel will address. (Paragraph 66) 

9. The evidence that we have seen does not suggest that Professor Jones was trying to 

subvert the peer review process. Academics should not be criticised for making 

informal comments on academic papers. The Independent Climate Change Email 

Review should look in detail at all of these claims. (Paragraph 73) 

Freedom of Information issues 

10. We regret that the ICO made a statement to the press that went beyond that which it 

could substantiate and that it took over a month for the ICO properly to put the 

record straight. We recommend that the ICO develop procedures to ensure that its 

public comments are checked and that mechanisms exist to swiftly correct any misstatements or misinterpretations of such statements. (Paragraph 91) 

11. There is prima facie evidence that CRU has breached the Freedom of Information 

Act 2000. It would, however, be premature, without a thorough investigation 

affording each party the opportunity to make representations, to conclude that UEA 

was in breach of the Act. In our view, it is unsatisfactory to leave the matter 

unresolved simply because of the operation of the six-month time limit on the 

initiation of prosecutions. Much of the reputation of CRU hangs on the issue. We 

conclude that the matter needs to be resolved conclusively—either by the 

Independent Climate Change Email Review or by the Information Commissioner. 

(Paragraph 93) 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 49

12. If the Minister was correct to assert in July 2009 that the Government had no 

evidence that the current six-month time limit presents a systemic problem, then it is 

now clear that such evidence exists. Irrespective of whether or not CRU breached the 

Freedom of Information Act 2000, we recommend that the Government review the 

operation of section 77 of the 2000 Act and the six month limit on the initiation of 

prosecutions provided by section 127(1) of the Magistrates Court Act 1980. 

(Paragraph 95) 

13. We have already recommended in paragraph 54 above that in future information, 

including data and methodology, should be published proactively on the internet 

wherever possible. However, a culture of withholding information—from those 

perceived by CRU to be hostile to global warming—appears to have pervaded CRU’s 

approach to FOIA requests from the outset. We consider this to be unacceptable. 

(Paragraph 103) 

14. We cannot reach a firm conclusion on the basis of the evidence we took but we must 

put on record our concern about the manner in which UEA allowed CRU to handle 

FOIA requests. Further, we found prima facie evidence to suggest that the UEA 

found ways to support the culture at CRU of resisting disclosure of information to 

climate change sceptics. The failure of UEA to grasp fully the potential damage to 

CRU and UEA by the non-disclosure of FOIA requests was regrettable. UEA needs 

to review its policy towards FOIA and re-assess how it can support academics whose 

expertise in this area is limited. (Paragraph 104) 

The Independent Climate Change Email Review 

15. We accept the assurances that Sir Muir Russell has given about the independence of 

the Independent Climate Change Email Review and we expect him to be scrupulous 

in preserving its impartiality. We see no reason why the Review’s conclusions and 

UEA’s response have to be published together. Indeed, it could give the impression 

that UEA was being given an advantage when it comes to responding. We consider 

that the Review’s conclusions and recommendations should not be conveyed to UEA 

in advance of publication. (Paragraph 113) 

16. With regards to the terms of reference of the Review, we consider that as well as 

measuring CRU against current acceptable scientific practice, the Review should also 

make recommendations on best practice to be followed by CRU in the future. We 

invite Sir Muir Russell to respond formally to our Report to the extent that he sets 

out whether, on the basis of its contents, he finds the Terms of Reference of his 

inquiry need to be changed. (Paragraph 114) 

17. It is unfortunate that the Independent Review got off to a bad start with the necessary 

resignation of Dr Campbell. The question of the operation of peer review is going to 

be a critical issue in the inquiry and the Review Team needs to take steps to ensure 

the insight and experience he would have brought are replaced. (Paragraph 119) 

18. We conclude that, when the Independent Review holds oral hearings or interviews, 

they should be carried out in public wherever possible and that it should publish all 

the written evidence it receives on its website as soon as possible. (Paragraph 122) 
50 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

The Scientific Appraisal Panel 

19. In our view, reputation has to be built on the solid foundation of excellent, peerreviewed science. The review of the science to be carried out by the Scientific 

Appraisal Panel, which UEA announced on 22 March, should determine whether the 

work of CRU has been soundly built and it would be premature for us to pre-judge 

that review. (Paragraph 131) 

20. Reputation does not, however, rest solely on the quality of work as it should. It also 

depends on perception. It is self-evident that the disclosure of the CRU e-mails has 

damaged the reputation of UK climate science and, as views on global warming have 

become polarised, any deviation from the highest scientific standards will be 

pounced on. As we explained in chapter 2, the practices and methods of climate 

science are a key issue. If the practices of CRU are found to be in line with the rest of 

climate science, the question would arise whether climate science methods of 

operation need to change. In this event we would recommend that the scientific 

community should consider changing those practices to ensure greater transparency. 

(Paragraph 132) 

The two inquiries 

21. The two reviews or inquiries need to map their activities to ensure that there are no 

unmanaged overlaps or gaps. If there are, the whole process could be undermined. 

(Paragraph 134) 

Conclusions 

22. The focus on Professor Jones and CRU has been largely misplaced. On the 

accusations relating to Professor Jones’s refusal to share raw data and computer 

codes, we consider that his actions were in line with common practice in the climate 

science community. We have suggested that the community consider becoming 

more transparent by publishing raw data and detailed methodologies. On 

accusations relating to Freedom of Information, we consider that much of the 

responsibility should lie with UEA, not CRU. (Paragraph 136) 

23. In addition, insofar as we have been able to consider accusations of dishonesty—for 

example, Professor Jones’s alleged attempt to “hide the decline”—we consider that 

there is no case to answer. Within our limited inquiry and the evidence we took, the 

scientific reputation of Professor Jones and CRU remains intact. We have found no 

reason in this unfortunate episode to challenge the scientific consensus as expressed 

by Professor Beddington, that “global warming is happening [and] that it is induced 

by human activity”. It was not our purpose to examine, nor did we seek evidence on, 

the science produced by CRU. It will be for the Scientific Appraisal Panel to look in 

detail into all the evidence to determine whether or not the consensus view remains 

valid. (Paragraph 137) 

24. A great responsibility rests on the shoulders of climate science: to provide the 

planet’s decision makers with the knowledge they need to secure our future. The 

challenge that this poses is extensive and some of these decisions risk our standard of 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 51

living. When the prices to pay are so large, the knowledge on which these kinds of 

decisions are taken had better be right. The science must be irreproachable. 

(Paragraph 138) 
52 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

Formal Minutes 

Wednesday 24 March 2010 

Members present: 

Mr Phil Willis, in the Chair 

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon

Graham Stringer

The Committee considered this matter. 

Draft Report (The disclosure of climate data from the Climatic Research Unit at the University of East 

Anglia), proposed by the Chair, brought up and read. 

Ordered, That the draft Report be read a second time, paragraph by paragraph. 

Paragraphs 1 to 46 read and agreed to. 

Paragraph 47 read. 

Question put, That the paragraph stand part of the Report. 

The Committee divided. 

Ayes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon 

Noes, 1

Graham Stringer 

Paragraphs 48 to 50 read and agreed to. 

Paragraph 51 read. 

Question put, That the paragraph stand part of the Report. 

The Committee divided. 

Ayes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon 

Noes, 1

Graham Stringer 

Paragraphs 52 to 65 read and agreed to. 

Paragraph 66 read. 

Amendment proposed, to leave out from the beginning to “We” in line 6 and insert “We have not taken 

enough evidence on this matter to come to a final conclusion”.—(Graham Stringer.) 

Question put, That the Amendment be made. 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 53

The Committee divided. 

Ayes, 1

Graham Stringer 

Noes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon

Paragraph 66 agreed to. 

Paragraphs 67 to 131 read and agreed to. 

Paragraph 132 read. 

Amendment proposed, to leave out from “science” in line 6 to the end and add “it would be necessary for the 

whole of climate science to increase its transparency and improve its scientific methodology”.—(Graham 

Stringer.) 

Question put, That the Amendment be made. 

The Committee divided. 

Ayes, 1

Graham Stringer 

Noes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon

Paragraph 132 agreed to. 

Paragraph 133 read and agreed to. 

Paragraph 134 read. 

Amendment proposed, at the end of line 5 to insert “Given the increasingly hostile attitudes of both sides on 

this issue, it is vital that these two inquiries have at least one member each who is a reputable scientist, and is 

sceptical of anthropogenic climate change”.—(Graham Stringer.) 

Question put, That the Amendment be made. 

The Committee divided. 

Ayes, 1

Graham Stringer 

Noes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon

Paragraphs 135 and 136 read and agreed to. 

Paragraph 137 read. 

Amendment proposed, after “answer” in line 3 add “Within our limited inquiry and the evidence we took, 

the scientific reputation of Professor Jones and CRU remains intact.”—(Dr Evan Harris.) 

Question put, That the Amendment be made. 
54 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

The Committee divided. 

Ayes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon 

Noes, 1

Graham Stringer 

Question put, That the paragraph, as amended, stand part of the Report. 

The Committee divided. 

Ayes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon 

Noes, 1

Graham Stringer 

Paragraph 138 read and agreed to. 

Summary brought up and read. 

Question put, That the summary be added to the Report. 

The Committee divided. 

Ayes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon 

Noes, 1

Graham Stringer 

Motion made, and Question put, That the Report be the Eighth Report of the Committee to the House. 

The Committee divided. 

Ayes, 3

Mr Tim Boswell 

Dr Evan Harris 

Dr Brian Iddon 

Noes, 1

Graham Stringer 

Resolved, That the Report be the Eighth Report of the Committee to the House. 

Ordered, That the Chair make the Report to the House. 

Ordered, That embargoed copies of the Report be made available, in accordance with the provisions of 

Standing Order No. 134. 

Written evidence was ordered to be reported to the House for printing with the Report, together with written 

evidence reported and ordered to be published on 24 February and 1 March 2010. 

Written evidence was ordered to be reported to the House for placing in the Library and Parliamentary 

Archives. 

[The Committee adjourned 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 55

Witnesses 

Wednesday 1 March 2010 

The Rt Hon Lord Lawson of Blaby, Chairman, and Dr Benny Peiser, Director, 

Global Warming Policy Foundation Ev 2

Richard Thomas CBE, former Information Commissioner Ev 11

Professor Edward Acton, Vice-Chancellor, University of East Anglia and

Professor Phil Jones, Director of the Climatic Research Unit Ev 27

Sir Muir Russell, Head of the Independent Climate Change E-Mails Review Ev 41

Professor John Beddington, Government Chief Scientific Adviser, Professor 

Julia Slingo OBE, Chief Scientist, Met Office, and Professor Bob Watson, Chief 

Scientist, Defra Ev 58

List of written evidence 

1 Andrew Montford Ev 159 

2 Anne Stallybrass Ev 169 

3 Aporia Ev 98 

4 Climate Change E-Mails Review Team Ev 39 

5 Clive Menzies Ev 93 

6 David Andrew Cockroft Ev 168 

7 David Holland Ev 115 

8 David Shaw Ev 99 

9 Douglas J. Keenan Ev 181 

10 Dr. Benny Peiser Ev 164 

11 Dr. D. R. Keiller Ev 103 

12 Dr. Michael Simons Ev 97 

13 Dr. Sonja Boehmer-Christiansen Ev 124, Ev 127 

14 Dr. Timothy J. Osborn Ev 129 

15 Edward Dilley Ev 76 

16 Eric Rasmusen Ev 89 

17 G R Ryan Ev 78 

18 Geoffrey Sherrington Ev 78 

19 Global Warming Policy Foundation Ev 1 

20 Godfrey Bloom MEP Ev 92 

21 Ian Goddard Ev 82 

22 Institute of Physics Ev 167 

23 J Ronan Ev 197 
56 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

24 John F Kelly Ev 191 

25 John Graham-Cumming Ev 195 

26 John Wadsworth Ev 81 

27 Lalu Hanuman Ev 81 

28 Martin Brumby Ev 82 

29 Met Office Ev 46, Ev 64 

30 Mike Haseler Ev 133 

31 Nicholas Barnes and David Jones Ev 197 

32 Peabody Energy Company Ev 191 

33 Peter Sinclair Ev 82 

34 Peter Taylor Ev 186 

35 Phillip Bratby Ev 90 

36 Professor Darrel Ince Ev 152 

37 Professor Hans von Storch and Dr. Myles R. Allen Ev 172 

38 Professor John Beddington, Government Chief Scientific Adviser Ev 45, Ev 64 

39 Professor Peter Cox Ev 132 

40 Professor Ross McKitrick Ev 140 

41 Public Interest Research Centre Ev 176 

42 Research Councils UK Ev 175 

43 Richard S Courtney Ev 68 

44 Richard Thomas CBE Ev 7 

45 Richard Tyrwhitt-Drake Ev 162 

46 Roger Helmer MEP Ev 85 

47 Ronald K Bolton Ev 119, Ev 123 

48 Royal Society of Chemistry Ev 170 

49 Royal Statistical Society Ev 185 

50 Stephen McIntyre Ev 82, Ev 144 

51 Stephen Prower Ev 86 

52 Steven Mosher Ev 151 

53 Stuart Huggett Ev 77 

54 Susan Ewens Ev 83 

55 University of East Anglia Ev 16, Ev 17, Ev 25, Ev 34, Ev 37, Ev 38 

56 Walter Radtke Ev 77 

57 Warwick Hughes Ev 153 
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 57

List of unprinted evidence 

The following written evidence has been reported to the House, but has not been printed 

and copies have been placed in the House of Commons Library, where they may be 

inspected by Members. Other copies are in the Parliamentary Archives 

(www.parliament.uk/archives), and are available to the public for inspection. Requests for 

inspection should be addressed to The Parliamentary Archives, Houses of Parliament, 

London SW1A 0PW (tel. 020 7219 3074; e-mail archives@parliament.uk). Opening hours are 

from 9.30 am to 5.00 pm on Mondays to Fridays. 

CRU 27 The Global Warming Policy Foundation annexes 

CRU 58/58a Dr Nigel Dudley memoranda 
58 The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 

List of Reports from the Committee during 

the current Parliament 

The reference number of the Government’s response to each Report is printed in brackets 

after the HC printing number. 

Session 2009–10 

First Report The work of the Committee in 2008–09 HC 103

Second Report Evidence Check 1: Early Literacy Interventions HC 44 (HC 385)

Third Report The Government’s review of the principles applying to the 

treatment of independent scientific advice provided to government 

HC 158-I (HC 384)

Fourth Report Evidence Check 2: Homeopathy HC 45

Fifth Report The Regulation of Geoengineering HC 221

Sixth Report The impact of spending cuts on science and scientific research HC 335-I

Seventh Report Bioengineering HC 220

Eighth Report The disclosure of climate data from the Climatic Research Unit at 

the University of East Anglia 

HC 387-I

Session 2008–09 

First Report Re-skilling for recovery: After Leitch, implementing skills and 

training policies 

HC 48–I (HC 365)

Second Report The Work of the Committee 2007–08 HC 49

Third Report DIUS’s Departmental Report 2008 HC 51–I (HC 383)

Fourth Report Engineering: turning ideas into reality HC 50–I (HC 759)

Fifth Report Pre-appointment hearing with the Chair-elect of the Economic and 

Social Research Council, Dr Alan Gillespie CBE 

HC 505

Sixth Report Pre-appointment hearing with the Chair-elect of the Biotechnology 

and Biological Sciences Research Council, Professor Sir Tom Blundell 

HC 506

Seventh Report Spend, spend, spend? – The mismanagement of the Learning and 

Skills Council’s capital programme in further education colleges 

HC 530 (HC 989)

Eighth Report Putting Science and Engineering at the Heart of Government Policy HC 168–I (HC 1036)

Ninth Report Pre-appointment hearing with the Chair-elect of the Science and 

Technology Facilities Council, Professor Michael Sterling 

HC 887

Tenth Report Sites of Special Scientific Interest HC 717 (HC 990)

Eleventh Report Students and Universities HC 170–I (HC 991)

Session 2007–08 

First Report UK Centre for Medical Research and Innovation HC 185 (HC 459)

Second Report The work and operation of the Copyright Tribunal HC 245 (HC 637)

Third Report Withdrawal of funding for equivalent or lower level qualifications 

(ELQs) 

HC 187–I (HC 638)

Fourth Report Science Budget Allocations HC 215 (HC 639)

Fifth Report Renewable electricity-generation technologies HC 216–I (HC 1063) 

Sixth Report Biosecurity in UK research laboratories HC 360–I (HC 1111)
The disclosure of climate data from the Climatic Research Unit at the University of East Anglia 59

Seventh Report Pre-legislative Scrutiny of the Draft Apprenticeships Bill HC 1062–I 

(HC (2008–09)262)

First Special 

Report 

The Funding of Science and Discovery Centres: Government 

Response to the Eleventh Report from the Science and Technology 

Committee, Session 2006–07 

HC 214

Session 2007–08 (Continued) 

Second Special 

Report 

The Last Report: Government Response to the Thirteenth Report 

from the Science and Technology Committee, Session 2006–07 

HC 244

Fourth Special 

Report 

Investigating the Oceans: Government Response to the Science and 

Technology Committee’s Tenth Report of Session 2006–07 

HC 506 

[incorporating 

HC 469–i]
 

1

Report of the International Panel set up by the University of East Anglia to 

examine the research of the Climatic Research Unit. 

Introduction 

1. The Panel was set up by the University in consultation with the Royal Society 

to assess the integrity of the research published by the Climatic Research Unit 

in the light of various external assertions. The Unit is a very small academic 

entity within the School of Environmental Sciences. It has three full time and 

one part time academic staff members and about a dozen research associates, 

PhD students and support staff. The essence of the criticism that the Panel was 

asked to address was that climatic data had been dishonestly selected, 

manipulated and/or presented to arrive at pre-determined conclusions that 

were not compatible with a fair interpretation of the original data. The 

members of the Panel are listed in Appendix A at the end of this report. 

2. The Panel was not concerned with the question of whether the conclusions of 

the published research were correct. Rather it was asked to come to a view on 

the integrity of the Unit’s research and whether as far as could be determined 

the conclusions represented an honest and scientifically justified interpretation 

of the data. The Panel worked by examining representative publications by 

members of the Unit and subsequently by making two visits to the University 

and interviewing and questioning members of the Unit. Not all the panel were 

present on both occasions but two members were present on both occasions to 

maintain continuity. About fifteen person/days were spent at the University 

discussing the Unit’s work. 

3. The eleven representative publications that the Panel considered in detail are 

listed in Appendix B. The papers cover a period of more than twenty years and 

were selected on the advice of the Royal Society. All had been published in 

international scientific journals and had been through a process of peer review. 

CRU agreed that they were a fair sample of the work of the Unit. The Panel 

was also free to ask for any other material that it wished and did so. 

Individuals on the panel asked for and reviewed other CRU research materials. 

4. The Panel’s work began with a detailed reading of the published work. Every 

paper was read by a minimum of three Panel members at least one of whom 

was familiar with the general area to which the paper related. At least one of 

the other two was a generalist with no special climate science expertise but 

with experience of some of the general techniques and methods employed in 

the work. Most of the members of the Panel read all the publications. The 

publications provided a platform from which to gain a deeper understanding of 

the Unit’s research and enabled the Panel to probe particular questions in more 

detail. 
 

2

5. Broadly the work of the Unit falls into two parts: 

• Construction and interpretation of tree ring chronologies extending 

over some thousands of years with a view to gaining information about 

past climates: 

• Studies of temperatures over the last few hundred years from direct 

observations. 

Dendroclimatology 

1. Tree growth is sensitive to very many factors including climate. By piecing 

together growth records from different trees, living or dead, it is possible to 

determine the temporal variation of growth patterns going back many 

hundreds of years. The dendroclimatological work at CRU seeks to go beyond 

this and to extract from the dated growth patterns the local and regional history 

of temperature variations. The Unit does virtually no primary data acquisition 

but has used data from published archives and has collaborated with people 

who have collected data. 

2. The main effort of the dendroclimalogists at CRU is in developing ways to 

extract climate information from networks of tree ring data. The data sets are 

large and are influenced by many factors of which temperature is only one. 

This means that the effects of long term temperature variations are masked by 

other more dominant short term influences and have to be extracted by 

statistical techniques. The Unit approaches this task with an independent 

mindset and awareness of the interplay of biological and physical processes 

underlying the signals that they are trying to detect. 

3. Although inappropriate statistical tools with the potential for producing 

misleading results have been used by some other groups, presumably by 

accident rather than design, in the CRU papers that we examined we did not 

come across any inappropriate usage although the methods they used may not 

have been the best for the purpose. It is not clear, however, that better methods 

would have produced significantly different results. The published work also 

contains many cautions about the limitations of the data and their 

interpretation. 

4. Chronologies (transposed composites of raw tree data) are always work in 

progress. They are subject to change when additional trees are added; new 

ways of data cleaning may arise (e.g. homogeneity adjustments), new 

measurement methods are used (e.g. of measuring ring density), new statistical 

methods for treating the data may be developed (e.g. new ways of allowing for 

biological growth trends). 

5. This is illustrated by the way CRU check chronologies against each other; this 

has led to corrections in chronologies produced by others. CRU is to be 

commended for continuously updating and reinterpreting their earlier 

chronologies. 
 

3

6. With very noisy data sets a great deal of judgement has to be used. Decisions 

have to be made on whether to omit pieces of data that appear to be aberrant. 

These are all matters of experience and judgement. The potential for 

misleading results arising from selection bias is very great in this area. It is 

regrettable that so few professional statisticians have been involved in this 

work because it is fundamentally statistical. Under such circumstances there 

must be an obligation on researchers to document the judgemental decisions 

they have made so that the work can in principle be replicated by others. 

7. CRU accepts with hindsight that they should have devoted more attention in 

the past to archiving data and algorithms and recording exactly what they did. 

At the time the work was done, they had no idea that these data would assume 

the importance they have today and that the Unit would have to answer 

detailed inquiries on earlier work. CRU and, we are told, the tree ring 

community generally, are now adopting a much more rigorous approach to the 

archiving of chronologies and computer code. The difficulty in releasing 

program code is that to be understood by anyone else it needs time-consuming 

work on documentation, and this has not been a top priority. 

8. After reading publications and interviewing the senior staff of CRU in depth, 

we are satisfied that the CRU tree-ring work has been carried out with 

integrity, and that allegations of deliberate misrepresentation and unjustified 

selection of data are not valid. In the event CRU scientists were able to give 

convincing answers to our detailed questions about data choice, data handling 

and statistical methodology. The Unit freely admits that many data analyses 

they made in the past are superseded and they would not do things that way 

today. 

9. We have not exhaustively reviewed the external criticism of the 

dendroclimatological work, but it seems that some of these criticisms show a 

rather selective and uncharitable approach to information made available by 

CRU. They seem also to reflect a lack of awareness of the ongoing and 

dynamic nature of chronologies, and of the difficult circumstances under 

which university research is sometimes conducted. Funding and labour 

pressures and the need to publish have meant that pressing ahead with new 

work has been at the expense of what was regarded as non-essential record 

keeping. From our perspective it seems that the CRU sins were of omission 

rather than commission. Although we deplore the tone of much of the criticism 

that has been directed at CRU, we believe that this questioning of the methods 

and data used in dendroclimatology will ultimately have a beneficial effect and 

improve working practices 

Temperatures from Historical Instrumental Records 

1. The second main strand of work at CRU has been the collection and collation 

of instrumental land temperature records from all over the world and the 

construction of regional, hemispherical and global scale temperature records. 

These records are irregularly distributed in space and time. Modern records 

come largely from land-based meteorological stations but their geographical 

distribution is uneven and strongly biased in favour of the northern hemisphere 
 

4

where most of the Earth’s land masses are located. Oceans cover two thirds of 

the Earth’s surface and away from the main shipping routes coverage is thin. 

For earlier centuries the record is much sparser. Deriving estimates of past 

temperatures on a global, hemispheric and regional scale from incomplete data 

sets is one of the problems faced by the Unit and in consequence an important 

current interest is the discovery of useable old temperature records from a 

variety of sources. 

2. In the latter part of the 20th century CRU pioneered the methods for taking into 

account a wide range of local influences that can make instrumental records 

from different locations hard to compare. These methods were very labour 

intensive and were somewhat subjective. Much of this work was supported by 

the US Department of Energy and was published with the details of station 

corrections several times a year. Since the 1980s the Unit has done no more of 

this work and have concentrated on the merging and interpretation of data 

series corrected by others. There have been various analyses of similar 

publicly available data sets by different international groups. Although there 

are some differences in fine detail that reflect the differences in the analytical 

methods used, the results are very similar. 

3. The Unit has devoted a great deal of effort to understanding how instrumental 

observations are best combined to derive the surface temperature on a variety 

of time and space scales. It has become apparent from a number of studies that 

there is elevation of the surface temperature in and around large cities and 

work is continuing to understand this fully. 

4. Like the work on tree rings this work is strongly dependent on statistical 

analysis and our comments are essentially the same. Although there are 

certainly different ways of handling the data, some of which might be 

superior, as far as we can judge the methods which CRU has employed are fair 

and satisfactory. Particular attention was given to records that seemed 

anomalous and to establishing whether the anomaly was an artefact or the 

result of some natural process. There was also the challenge of dealing with 

gaps in otherwise high quality data series. In detailed discussion with the 

researchers we found them to be objective and dispassionate in their view of 

the data and their results, and there was no hint of tailoring results to a 

particular agenda. Their sole aim was to establish as robust a record of 

temperatures in recent centuries as possible. All of the published work was 

accompanied by detailed descriptions of uncertainties and accompanied by 

appropriate caveats. The same was true in face to face discussions. 

5. We believe that CRU did a public service of great value by carrying out much 

time-consuming meticulous work on temperature records at a time when it was 

unfashionable and attracted the interest of a rather small section of the 

scientific community. CRU has been among the leaders in international efforts 

to determining the overall uncertainty in the derived temperature records and 

where work is best focussed to improve them. 
 

5

6. The Unit has demonstrated that at a global and hemispheric scale temperature 

results are surprisingly insensitive to adjustments made to the data and the 

number of series included. 

7. Recent public discussion of climate change and summaries and 

popularizations of the work of CRU and others often contain oversimplifications that omit serious discussion of uncertainties emphasized by the 

original authors. For example, CRU publications repeatedly emphasize the 

discrepancy between instrumental and tree-based proxy reconstructions of 

temperature during the late 20th century, but presentations of this work by the 

IPCC and others have sometimes neglected to highlight this issue. While we 

find this regrettable, we could find no such fault with the peer-reviewed papers 

we examined 

Conclusions 

1. We saw no evidence of any deliberate scientific malpractice in any of the work 

of the Climatic Research Unit and had it been there we believe that it is likely 

that we would have detected it. Rather we found a small group of dedicated if 

slightly disorganised researchers who were ill-prepared for being the focus of 

public attention. As with many small research groups their internal procedures 

were rather informal. 

2. We cannot help remarking that it is very surprising that research in an area that 

depends so heavily on statistical methods has not been carried out in close 

collaboration with professional statisticians. Indeed there would be mutual 

benefit if there were closer collaboration and interaction between CRU and a 

much wider scientific group outside the relatively small international circle of 

temperature specialists. 

3. It was not the immediate concern of the Panel, but we observed that there were 

important and unresolved questions that related to the availability of 

environmental data sets. It was pointed out that since UK government adopted 

a policy that resulted in charging for access to data sets collected by 

government agencies, other countries have followed suit impeding the flow of 

processed and raw data to and between researchers. This is unfortunate and 

seems inconsistent with policies of open access to data promoted elsewhere in 

government. 

4. A host of important unresolved questions also arises from the application of 

Freedom of Information legislation in an academic context. We agree with the 

CRU view that the authority for releasing unpublished raw data to third parties 

should stay with those who collected it. 

Submitted to the University 12 April 2010 
 

6

Addendum to report, 19 April 2010 

For the avoidance of misunderstanding in the light of various press stories, it is 

important to be clear that the neither the panel report nor the press briefing intended to 

imply that any research group in the field of climate change had been deliberately 

misleading in any of their analyses or intentionally exaggerated their findings. 

Rather, the aim was to draw attention to the complexity of statistics in this field, and 

the need to use the best possible methods. 
 

7

APPENDIX A 

PANEL MEMBERSHIP 

Chair: Prof Ron Oxburgh FRS (Lord Oxburgh of Liverpool) 

Prof Huw Davies, ETH Zürich 

Prof Kerry Emanuel, Massachusetts Institute of Technology 

Prof Lisa Graumlich, University of Arizona. 

Prof David Hand FBA, Imperial College, London. 

Prof Herbert Huppert FRS, University of Cambridge 

Prof Michael Kelly FRS, University of Cambridge
 

8

APPENDIX B 

Peer-reviewed publications for assessment. 

1. Brohan, P., Kennedy, J., Harris, I., Tett, S.F.B. and Jones, P.D., 2006: Uncertainty 

estimates in regional and global observed temperature changes: a new dataset from 

1850. J. Geophys. Res. 111, D12106. 

2. Briffa, K. R., F. H. Schweingruber, P. D. Jones, T. J. Osborn, S. G. Shiyatov, and E. 

A. Vaganov. 1998a. Reduced sensitivity of recent tree-growth to temperature at high 

northern latitudes. Nature 391:678-682. 

3. Briffa, K. R., F. H. Schweingruber, P. D. Jones, T. J. Osborn, I. C. Harris, S. G. 

Shiyatov, E. A. Vaganov, and H. Grudd, 1998b. Trees tell of past climates: but are 

they speaking less clearly today? Philosophical Transactions of the Royal Society of 

London Series B – Biological Sciences 353, 65-73. 

4. Briffa, K. R. 2000. Annual climate variability in the Holocene: interpreting the 

message of ancient trees. Quaternary Science Reviews 19, 87-105. 

5. Briffa, K.R., Osborn, T.J., Schweingruber, F.H., Harris, I.C., Jones, P.D., Shiyatov, 

S.G. and Vaganov, E.A., 2001: Low-frequency temperature variations from a 

northern tree-ring density network. J. Geophys. Res. 106, 2929-2941. 

6. Briffa, K. R., V. V. Shishov, T. M. Melvin, E. A. Vaganov, H. Grudd, R. M. 

Hantemirov, M. Eronen, and M. M. Naurzbaev. 2008. Trends in recent temperature 

and radial tree growth spanning 2000 years across northwest Eurasia. Philosophical 

Transactions of the Royal Society B-Biological Sciences 363, 2271-2284. 

7. Jones, P.D. and Moberg, A., 2003: Hemispheric and large-scale surface air 

temperature variations: An extensive revision and an update to 2001. J. Climate 16, 

206-223. 

8. Jones, P.D., Raper, S.C.B., Bradley, R.S., Diaz, H.F., Kelly, P.M. and Wigley, 

T.M.L., 1986a: Northern Hemisphere surface air temperature variations: 1851-1984. 

Journal of Climate and Applied Meteorology 25, 161-179. 

9. Jones, P.D., Raper, S.C.B. and Wigley, T.M.L., 1986b: Southern Hemisphere surface 

air temperature variations: 1851-1984. Journal of Climate and Applied Meteorology

25, 1213-1230. 

10. Jones, P.D., Groisman, P.Ya., Coughlan, M., Plummer, N., Wang, W-C. and Karl, 

T.R., 1990: Assessment of urbanization effects in time series of surface air 

temperature over land. Nature 347, 169-172. 

11. Jones, P.D., Lister, D.H. and Li, Q., 2008: Urbanization effects in large-scale 

temperature records, with an emphasis on China. Journal of Geophysical Research, 

113, D16122. 

 

 Supporting documentation 

 Briffa and Melvin (2009) which is online at 

 http://www.cru.uea.ac.uk/cru/people/briffa/yamal2009/
 

9

 TR017 – Bradley, R.S., Kelly, P.M., Jones, P.D., Goodess, C.M. and Diaz, H.F., 1985: 

A Climatic Data Bank for Northern Hemisphere Land Areas, 1851-1980, U.S. Dept. 

of Energy, Carbon Dioxide Research Division, Technical Report TRO17, 335 pp. 

 TR022 – Jones, P.D., Raper, S.C.B., Santer, B.D., Cherry, B.S.G., Goodess, C.M., 

Kelly, P.M., Wigley, T.M.L., Bradley, R.S. and Diaz, H.F., 1985: A Grid Point 

Surface Air Temperature Data Set for the Northern Hemisphere, U.S. Dept. of 

Energy, Carbon Dioxide Research Division, Technical Report TRO22, 251 pp. 

 

 TR027 – Jones, P.D., Raper, S.C.B., Cherry, B.S.G., Goodess, C.M. and Wigley, 

T.M.L., 1986: A Grid Point Surface Air Temperature Data Set for the Southern 

Hemisphere, 1851-1984, U.S. Dept. of Energy, Carbon Dioxide Research Division, 

Technical Report TR027, 73 pp. 
RA-1O Final Investigation Report Involving Dr. Michael E, Mann 

The Pennsylvania State University 

June 4, 2010 

Composition of the Investigatory Committee: 

Sarah M. Assmann, Waller Professor 

Department of Biology 

Welford Castleman, Evan Pugh Professor and Eberly Distinguished Chair in Science 

Department of Chemistry and Depmtment of Physics 

Mary Jane Irwin, Evan Pugh Professor 

Department of Computer Science and Electrical Engineering 

Nina G. Jablonski, Department Head and Professor 

Department of Anthropology 

Fred W. Vondracek, Professor 

Department of Human Development and Family Studies 

Research Integrity Officer: 

Candice Yekel, Director of the Office for Research Protections 

Background of the alleged misconduct as described in the RAlO Inquiry Report: 

On and about November 22, 2009, The Pennsylvania State University began to receive 

numerous communications (emails, phone calls and letters) accusing Dr. Michael E, 

Mann of having engaged in acts, beginning in approximately 1998, that included 

manipulating data, destroying records and colluding to hamper the progress of scientific 

discourse around the issue of anthropogenic global warming, These accusations were 

based on perceptions of the content of the emails stolen from a server at the Climatic 

Research Unit of the University of East Anglia in Great Britain as widely reported, 

Given the sheer volume of the communications to Penn State, the similarity of their 

content and the variety of sources, which included University alumni, federal and state 

politicians, and others, many of whom had had no relationship with Pel1l1 State, Dr. Eva J. 

Pell, then Senior Vice President for Research and Dean of the Graduate School, was 

asked to examine the matter. The reason for having Dr. Pell examine the matter was that 

the accusations, when placed in an academic context, could be construed as allegations of 

research misconduct, which would constitute a violation of Penn State policy, 

Under The Pennsylvania State University's policy, Research Administration Policy No, 

10, (hereafter referred to as RA-I 0), Research Misconduct is defined as: 

Page 1 of 19 
(1) fabrication, falsification, plagiarism or other practices that seriously deviate from 

accepted practices within the academic community for proposing, conducting, or 

reporting research or other scholarly activities; 

(2) callous disregard for requirements that ensure the protection of researchers, human 

participants, or the public; or for ensuring the welfare oflaboratory animals; 

(3) failure to disclose significant financial and business interest as defined by Penn State 

Policy RA20, Individual Conflict of Interest; 

(4) failure to comply with other applicable legal requirements governing research or other 

scholarly activities. 

RA-IO further provides that "research misconduct does not include disputes regarding 

honest error or honest differences in interpretations or judgments of data, and is not 

intended to resolve bona fide scientific disagreement or debate." 

On November 24, 2009, two days after receipt of the allegations, Dr. Pell initiated the 

process articulated in RA-1O by scheduling a meeting with the Dean of the College of 

Earth and Mineral Sciences (Dr. William Easterling), the Associate Dean for Graduate 

Education and Research of the College of Em1h and Mineral Sciences (Dr. Alan Scaroni), 

the Director of the Office for Research Protections (Ms. Candice Yekel), and the Head of 

the Department of Meteorology (Dr. William Brune). 

At this meeting, all were informed of the situation and of the decision to initiate an 

inquiry under RA-I O. Dr. Pell then discussed the responsibilities that each individual 

would have according to the policy. Dean Easterling recused himself from the inquiry 

due to a conflict of interest. As the next administrator in the line of management for the 

college, Dr. Scm'oni was asked to take on Dean Easterling's function in the ensuing 

inquiry. 

The Inquiry Committee assigned to conduct the inquiry into the matter consisted of Dr. 

Eva J. Pell, Senior Vice President for Research, Ms. Candice Yekel, Director of the 

Office for Research Protections, and Dr. Alan Scaroni, Associate Dean for Graduate 

Education and Research of the College of Earth and Mineral Sciences. Dr. William 

Brune, Head of the Department of Meteorology, was to serve in a consulting capacity for 

the Inquiry Committee. Dr. Henry C. Foley, then Dean of the College ofInformation 

Sciences and Technology, was added to the Inquiry Committee in an ex-officio role for 

the duration of2009, since he had been named to succeed Dr. Pell as the next Vice 

President for Research, beginning January 1,2010. 

At the time of initiation of the inquiry, no formal allegations accusing Dr. Mann of 

research misconduct had been submitted to any University official. Therefore, the emails 

and other communications were reviewed by Dr. Pell, and from these she synthesized the 

following four formal allegations. To be clear, these were not allegations that Dr. Pell put 

forth but rather her best effo11 to reduce to reviewable allegations the many different 

accusations that were received from pm1ies outside of the University. The four 

synthesized allegations were as follows: 

Page 2 of 19 
I. Did you engage in, or participate in, directly or indirectly, any actions with the intent to 

suppress or falsify data? 

2. Did you engage in, or participate in, directly or indirectly, any actions with the intent to 

delete, conceal or otherwise destroy emails, information and/or data, related to AR4,as 

suggested by Phil Jones? 

3. Did you engage in, or participate in, directly or indirectly, any misuse of privileged or 

confidential information available to you in your capacity as an academic scholar? 

4. Did you engage in, or participate in, directly or indirectly, any actions that seriously 

deviated from accepted practices within the academic community for proposing, 

conducting, or reporting research, or other scholarly activities? 

On November 29, 2009, Dr. Pell and Dr. Foley met with Dr. Mann to inform him 

personally that he had been accused of research misconduct and that an inquiry under 

RA-1O would take place. On November 30, 2009, a letter was delivered by Dr. Pell to Dr. 

Mann to notify him of these allegations and Dr. PeWs decision to conduct an inquiry 

under RA-IO. The inquiry phase ofRA-IO was thereby formally initiated on November 

30,2009. 

From November 30 to December 14,2009, staff in the Office for Research Protections 

culled through the 1073 files that contained emails or email strings that were purloined 

from a server at the University of East Anglia. A subset of the files containing emails or 

email strings was reviewed. This subset of files included emails that were sent by Dr. 

Mann, were sent to Dr. Mann, were copied to Dr. Mann, or discussed Dr. Mann (but were 

neither addressed nor copied to him). In summary, the following were found: 

206 files that contained emails or email strings that contained messageltext from 

Dr. Mann somewhere in the chain; 

91 files that contained emails or emails strings that were received by Dr. Mann, 

but in which he did not participate; and 

79 files that contained emails or email strings that dealt with Dr. Mann, his work 

or publications but that he neither authored nor was listed as copied. 

From among these 376 files, the Inquiry Committee focused on 47 files that contained 

emails or email strings that were deemed relevant. On December 17, 2009, the Inquiry 

Committee (Pell, Scaroni, Yekel), Dr. Brune and Dr. Foley met to review the emails, the 

RA-1O inquiry process, and their respective activities. It was agreed that these individuals 

would meet again in early January and that they would use the time until that meeting to 

review the relevant information, including the above mentioned e-mails, journal articles, 

OP-ED columns, newspaper and magazine articles, the National Academy of Sciences 

report entitled "Surface Temperature Reconstructions for the Last 2,000 Years," ISBN: 0-

309-66144-7 and various blogs on the internet. 

Page 3 of 19 
On January 4, 2010, Dr. Foley, in his capacity as the new Vice President for Research 

and Dean of the Graduate School, became the convener of the Inquiry Committee as Dr. 

Pell had left the University to become the Under-Secretary of Science for the 

Smithsonian Institution. On January 8, 2010, Dr. Foley convened the Inquiry Committee 

to discuss their thoughts on the evidence presented in the emails and other publically 

available materials. At this meeting, it was decided that each Inquiry Committee member 

would send to Dr. Foley specific questions to be used by the Inquiry Committee during 

the interview of Dr. Mann. During the interview, Dr. Foley would moderate the 

interview and ask each of the initial questions with follow-up questions coming from the 

other Inquiry Committee members. 

On January 12,2010, the Inquiry Committee (Foley, Yekel, Scaroni) and Dr. Brune met 

with Dr. Mann. Dr. Mann was asked to address the four allegations leveled against him 

and to provide answers to the fifteen additional questions that the Inquiry Committee had 

compiled. In an interview lasting nearly two hours, Dr. Mann addressed each of the 

questions and follow-up questions. A recording was made of the interview and was later 

transcribed. The Inquiry Committee members asked occasional follow-up questions. Dr. 

Mann answered each question carefully: 

• He explained the content and meaning of the emails about which the Inquiry 

Committee inquired; 

• He stated that he had never falsified any data, nor had he had ever manipulated 

data to serve a given predetermined outcome; 

• He stated that he never used inappropriate influence in reviewing papers by other 

scientists who disagreed with the conclusions of his science; 

• He stated that he never deleted emails at the behest of any other scientist, 

specifically including Dr. Phil Jones, and that he never withheld data with the 

intention of obstructing science; and 

• He stated that he never engaged in activities or behaviors that were inconsistent 

with accepted academic practices. 

On January 15,2010, Dr. Foley conveyed via email on behalf of the Inquiry Committee 

an additional request to Dr. Mann. Dr. Mann was asked to produce all emails related to 

the fourth IPCC report ("AR4"), the same emails that Dr. Phil Jones had suggested that 

he delete. On January 18,2010, Dr. Mann provided a zip-archive of these emails and an 

explanation of their content. In addition, Dr. Mann provided a ten page supplemental 

written response to the matters discussed during his interview. 

On January 22, 2010, the Inquiry Committee and Dr. Brune met again to review the 

evidence, including but not limited to Dr. Mann's answers to the Inquiry Committee's 

questions, both in the interview and in his subsequent submissions. Dr. Foley reviewed 

the relevant points of his conversation with Dr. Gerald North, a professor at Texas A&M 

University and the first author of the NAS 2006 report that included Dr. Mann's research 

on paleoclimatology. Dr. Foley also relayed the sentiment and view of Dr. Donald 

Kennedy of Stanford University and the former editor of Science Magazine about the 

controversy currently swirling around Dr. Mann and some of his colleagues. Both were 

Page 4 of 19 
very supportive of Dr. Mann and of the credibility of his science. Dr. Brune gave his 

opinions and suggestions for next steps of the process, and then was dismissed from 

fmther discussion pursuant to RA-1O policy role which was consult to the rest of the 

Inquiry Committee members. 

On January 26, 2010, Dr. Foley convened the Inquiry Committee, along with University 

counsel, Mr. Wendell Courtney, Esq., in case issues of procedure arose. 

After a careful review of all written material, and information obtained from the 

purloined emails, the interview of Dr. Mann, the supplemental materials provided by Dr. 

Mann and all the information from other sources, the Inquiry Committee found as follows 

with respect to each allegation: 

Allegation 1: "Did you engage in, or participate in, directly or indirectly, any actions 

with the intent to suppress or falsify data? " 

Decision 1: The Inquiry Committee determined there was no substance to this 

allegation and further investigation of this allegation was not warranted. 

Allegation 2: "Did you engage in, or participate in, directly or indirectly, any actions 

with the intent to delete, conceal or otherwise destroy emails, information and/or data, 

related to AR4, as suggested by Phil Jones?" 

Decision 2: The Inquiry Committee determined there was no substance to this 

allegation and further investigation of this allegation was not warranted. 

Allegation 3: "Did you engage in, or participate in, directly or indirectly, any misuse of 

privileged or confidential information available to you in your capacity as an academic 

scholar?" 

Decision 3: The Inquiry Committee determined there was no substance to this 

allegation and further investigation of this allegation was not warranted. 

Allegation 4: "Did you engage in, or participate in, directly or indirectly, any actions 

that seriously deviated from accepted practices within the academic community for 

proposing, conducting, or reporting research or other scholarly activities?" 

Decision 4: The Inquiry Committee determined that "given that information 

emerged in the form of the emails purloined from CRU in November 2009, which 

have raised questions in the public's mind about Dr. Mann's conduct of his 

research activity, given that this may be undermining confidence in his findings as 

a scientist, and given that it may be undermining public trust in science in general 

and climate science specifically, an Investigatory Committee of faculty peers 

from diverse fields should be constituted under RA-I 0 to further consider this 

allegation." 

Page 5 of 19 
An Investigatory Committee of faculty members with impeccable credentials was 

appointed and asked to present its findings and recommendations to Dr. Henry C. Foley 

within 120 days of being charged. 

The charge to the RA-lO Investigatory Committee: 

The Investigatory Committee was charged by Dr. Henry C. Foley, Vice President for 

Research, on March 4, 2010, as follows: 

The Investigatory Committee's charge is to determine whether or not Dr. Michael Mann 

engaged in, or participated in, directly or indirectly, any actions that seriously deviated 

from accepted practices within the academic community for proposing, conducting, or 

reporting research or other scholarly activities. 

Sources of support for the related research or publications: 

Dr. Mann's research has been sponsored by many different agencies including the 

National Science Foundation, the Department of Energy and the National Oceanic and 

Atmosphere Administration. 

Documents available to the Investigatory Committee: 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

376 files containing emails stolen from the Climate Research Unit (CRU) of the 

University of East Anglia and originally reviewed by the Inquiry Committee 

Documents collected by the Inquiry Committee 

Documents provided by Dr. Mann at both the Inquiry and Investigation phases 

Penn State University's RA-IO Inquiry Report 

House of Commons Report HC387-I, March 31,2010 

National Academy of Science letter titled, "Climate Change and the Integrity of 

Science" that was published in Science magazine on May 7, 2010 

Information on the peer review process for the National Science Foundation (NSF) 

Department of Energy's Guide to Financial Assistance 

Information on National Oceanic and Atmospheric Administration's peer review 

process 

Information regarding the percentage of NSF proposals funded 

Dr. Michael Mann's curriculum vitae 

Interview process: 

The interviews were audio-taped and verbatim transcripts were prepared. All interviewed 

individuals were provided an opportunity to review the transcripts of their interviews for 

accuracy. The transcripts will be maintained in the Office for Research Protections as 

part of the official record. Statements or information relevant to the Investigatory 

Committee's findings are noted in the paragraphs below. The Investigatory Committee 

interviewed the following individuals: 

Page 6 of 19 
April 12, 2010: Dr. William Easterling, Dean, College of Earth and Mineral Sciences, 

The PelIDsylvania State University 

April 14, 2010: Dr. Michael Mann, Professor, Department of Meteorology, The 

Pennsylvania State University 

April 20, 2010: Dr. William Curry, Senior Scientist, Geology and Geophysics 

Department, Woods Hole Oceanographic Institution 

April 20, 2010: Dr. Jerry McManus, Professor, Department of Earth and Environmental 

Sciences, Columbia University 

May 5, 2010: Dr. Richard Lindzen, Alfred P. Sloan Professor, Department of Earth, 

Atmospheric and Planetary Sciences, Massachusetts Institute of 

Technology 

Summary ofluvestigatory Committee's Interview with Dr. Michael E. Mann, 

Professor, Department of Meteorology, Penn State University - April 14, 2010 

On April 14, 2010, the RA-1O Investigatory Committee (Assmann, Castleman, Irwin, 

Jablonski, and Vondracek) and Candice Yekel interviewed Dr. Michael Mann. In 

advance of the interview, the Investigatory Committee prepared several questions 

focusing on whether Dr. Mann "engaged in, or participated in, directly or indirectly, any 

actions that seriously deviated from accepted practices within the academic community 

for preparing, conducting, or reporting research or other scholarly activities." In addition 

to the prepared questions, Investigatory Committee members asked a number of followup questions. Dr. Mann answered the questions in a detailed manner. 

The first question was "Would you please tell us what you consider in your field to be 

accepted, standard practice with regard to sharing data?" A follow-up question asked how 

Dr. Mann had dealt with requests for data that were addressed to him during the period 

covered by the stolen emails. Dr. Mann offered a brief historical perspective on the issue 

of sharing data in his field, concluding with the observation that data are made generally 

available (e.g., in the NOAA public database) after those scientists who obtained the data 

have had a chance to be the first to publish findings based on the data. He noted that 

sometimes data are made available on a collegial basis to specific scientists before those 

who collected the data have published their initial findings. Typically, this involves a 

request to not release the data to others until the data are made publically available by the 

scientists who obtained the data. Dr. Mann concluded his answer by stating that he has 

always worked with data obtained by other scientists, and that when such data were not 

already in the public domain, he made them available as soon as he was permitted to do 

so by those who initially obtained the data. 

Dr. Mann drew a distinction between actual data and intermediate data that are produced 

as part of the analytic procedures employed. He indicated that while such intermediate 

data may occasionally be shared with colleagues, it is not standard practice to publish or 

make generally available this intermediate data (to which he and others refer to as "dirty 

laundry" in one of the purloined emails). Finally, he indicated that someone who wanted 

to reproduce his work would be able to independently reproduce this intermediate data 

and that, in fact, other researchers had done this. 

Page 7 of 19 
The Investigatory Committee next inquired how he constructed his source codes and 

what he considered to be accepted practice in his field for publishing source codes. Dr. 

Mann indicated that in his field of study, in contrast with some other fields such as 

economics, publishing the source code was never standard practice until his work and 

that of his colleagues came under public scrutiny, resulting in public pressure to do so. 

He indicated that he initially was reluctant to publish his source codes because the 

National Science Foundation had determined that source codes were the intellectual 

property of the investigator. Also, he developed his source codes using a programming 

language (FORTRAN 77) that was not likely to produce identical results when run on a 

computer system different from the one on which it was developed (e.g., different 

processor makes/models, different operating systems, different compilers, different 

compiler optimizations). Dr. Mann reported that since around 2000, he has been using a 

more accessible programming style (MATLAB), and since then he has made all source 

codes available to the research community. 

The next question was "Do you believe that the perceived hostility and perceived ulterior 

motives of some critics of global climate science influenced your actions with regard to 

the peer review process, particularly in relation to the papers discussed in the stolen 

emails?" Dr. Mann responded by affirming his belief in the importance of the peer 

review process as a means of ensuring that scientifically sound papers are published, and 

not as a means of preventing the publication of papers that are contrary to one's views. 

I-Ie elaborated by stating that some of the emails regarding this issue dealt with his 

concern (shared by other scientists, the publisher, and some members of the editorial 

board of the journal in question) that the legitimacy of the peer review process had been 

subverted. 

Next, Dr. Mann was asked "Did you ever, without first getting express permission from 

the original authors, forward to a third party an in-press or submitted manuscript on 

which you were not a co-author?" In response to this question, Dr. Mann first responded 

by saying that to the best of his knowledge he had not done so. He then clarified that he 

may have forwarded such a manuscript to a specific, close colleague, in the belief that 

permission to do so had been implicit, based on his close collegial relationships with the 

paper's authors. An illustrative case of such a circumstance would have been the 

manuscript by Wahl and Ammann, which Dr. Mann forwarded to Dr. Briffa. In response 

to a follow-up question, Dr. Mann asselied that such judgments about implied consent are 

quite typical in his field, but they are made only as long as it is understood that such 

sharing would take place only among trusted colleagues who would maintain the 

confidentiality of the manuscript. 

The next question for Dr. Mann was posed as follows: "What is your reply to the email 

statements of Dr. McIntyre (a) that he had been referred to an incorrect version of your 

data at your FTP site (b) that this incorrect version was posted prior to his request and 

was not formulated expressly for him and (c) that to date, no source code or other 

evidence has been provided to fully demonstrate that the incorrect version, now deleted, 

did not infect some of Mann's and Rutherford's other work?" Dr. Mann responded by 

Page 8 of 19 
stating that neither he, nor many of his colleagues, put much reliability in the various 

accusations that Dr. McIntyre has made, and that, moreover, there is "no merit 

whatsoever to Mr. McIntyre's claims here." Specifically, Dr. Mann repeated that all data, 

as well as the source codes requested by Dr. McIntyre, were in fact made available to 

him. All data were listed on Dr. Mann's FTP site in 2000, and the source codes were 

made available to Dr. McIntyre about a year after his request was made, in spite of the 

fact that the National Science Foundation had ruled that scientists were not required to do 

so. The issue of an "incorrect version" of the data came about because Dr. McIntyre had 

requested the data (which were already available on the FTP site) in spreadsheet format, 

and Dr. Rutherford, early on, had unintentionally sent an incorrectly formatted 

spreadsheet. 

In response to a couple of follow-up questions, Dr. Mann stressed that the stolen emails 

represent part of a larger context of active communication among scientists, and that he 

remains on friendly terms with scientists with whom he has had ongoing, and sometimes 

heated, disagreements about scientific matters. He also commented that he and other 

scientists fear that the stolen emails will have a chilling effect on the way scientists 

communicate with each other, partly because members of the public may not appreciate 

the lingo or jargon (e.g., "dirty laundry" or "trick") that scientists often use when 

communicating with each other about their science. 

At the conclusion of the interview, Dr. Mann indicated that he would be very happy to 

provide additional information if the Investigatory Committee felt that this would be 

helpful. 

Summary of Investigatory Committee Interview with Dr. William Easterling, Dean, 

College of Earth and Mineral Sciences, Penn State University - April 12, 2010 

On April 12, 2010, the RA-lO Investigatory Committee (Assmann, Castleman, Irwin, 

Jablonski, and Vondracek) and Candice Yekel interviewed Dr. William Easterling, Dean 

of the College of Earth and Mineral Sciences, Penn State University. The Investigatory 

Committee had a number of prepared questions, starting with a request to learn how Dr. 

Easterling knew Dr. Mann. Dr. Easterling reported that he had known Dr. Mann for 

about six or seven years prior to his appointment at Penn State in 2008. In response to a 

question about when and how he had become aware of the allegations against Dr. Mann, 

Dr. Easterling reported that it was the week before Thanksgiving 2009, when he started 

receiving emails suggesting a connection between the stolen East Anglia emails and Dr. 

Mann's work. 

The next question for Dr. Easterling was posed as follows: "In your judgment, are 

accepted and ethical research practices in scientific fields related to global climate change 

significantly different from such practices in other fields of scientific inquiry?" Dr. 

Easterling's response to that question was "Absolutely not!" In a follow-up question, Dr. 

Easterling was asked whether he saw any difference between certain kinds of 

experimental scientific fields and observational ones like paleoclimatology. He responded 

by stating that much of what we know about climate change is the result of a combination 

Page 9 of 19 
of observation and numerical modeling, making the classic idea of falsification of a 

hypothesis, which may be applicable to a laboratory science, of limited applicability in 

the study of climate change. Thus, even though there are a number of highly 

sophisticated, physically sound models that are used to analyze and predict various 

features of the earth's climate system, humanjudgments are invariably involved, and a 

certain amount of subjectivity is introduced. 

Another follow-up question inquired about the likely number of different statistical 

models that might be applicable to Dr. Mann's work. Dr. Easterling indicated that Dr. 

Mann and his colleagues were primarily interested in looking at historical data (which 

tend to be "noisy"), using a relatively small number of statistical models, such as 

principal components analysis, which has a long tradition in various sciences. 

The next question addressed to Dr. Easterling was whether, in his judgment, Dr. Mann's 

work was very aggressive, very conservative, or somewhere in the middle in how it 

pOltrayed global warming. Dr. Easterling responded by stating that Dr. Mann's early 

work showed a more dramatic upturn in warming, but that his more recent work has led 

to the conclusion that the change has been slightly less dramatic. Moreover, Dr. 

Easterling added that Dr. Mann's findings have been replicated by independent teams of 

researchers. 

Dr. Easterling was asked whether he knew of any other investigations related to the 

stolen emails other than the University of East Anglia and Penn State University, and he 

responded that he was unaware of any others. 

The Investigatory Committee then questioned Dr. Easterling about various scientists in 

the field of climate science who might be interviewed by the Investigatory Committee 

regarding their views of what constitutes accepted and ethical practice with regard to the 

conduct of research in the field. The Investigatory Committee wanted a choice of 

scientists who had disagreed with Dr. Mann's findings as well as others who had agreed 

but who had not collaborated with Dr. Mann or his collaborators. 

At the conclusion of the interview, Dr. Easterling offered to be available to the 

Investigatory Committee if the Investigatory Committee members thought that this would 

be helpful. 

Summary ofInvestigatory Committee Interview with Dr. William Curry, Senior 

Scientist, Geology and Geophysics Department, Woods Hole Oceanographic 

Institution - April 20, 2010 

On April 20, 2010, the RA-l 0 Investigatory Committee (Assmann, Castleman, Irwin, 

Jablonski, and Vondracek) and Candice Yekel interviewed Dr. William Curry, Senior 

Scientist, Geology and Geophysics Department, Woods Hole Oceanographic Institution. 

The Investigatory Committee had four prepared questions, but Investigatory Committee 

members were free to ask additional questions as well as follow-up questions as they saw 

fit. 

Page 10 of 19 
The first question addressed to Dr. Curry was: "Would you please tell us what you 

consider in your field to be accepted standard practice with regard to sharing data and 

unpublished manuscripts?" With regard to sharing data, Dr. Curry indicated that standard 

practice is that once a publication occurs, the pertinent data are shared via some 

electronic repository. He stated that not all researchers actually comply with this practice, 

and that there may be special arrangements with the funding agency, or the journal that 

publishes the research, that specify when data need to be made available to other 

researchers. In Dr. Curry's case, for example, the National Science Foundation allows a 

two-year window during which he has exclusive rights to his data. After that period he 

must make it available to others. 

On the issue of sharing unpublished manuscripts, Dr. Curry stated that if the manuscript 

was accompanied by a request to keep it confidential, he would not share it with anyone; 

if it was not accompanied by an explicit request for confidentiality, he might talk about it 

with colleagues but would not usually forward it. 

Next, Dr. Curry was asked: "Would you please briefly explain how codes are developed 

in the process of evaluating data in your field, e.g., are these codes significantly different 

from published software packages? Then please tell us what you consider in your field to 

be accepted standard practice with regard to sharing codes." Dr. Curry reported that in 

his area, most codes are fairly basic and researchers use software packages to construct 

them. I-Ie also reported that he was not aware of any public archive for such codes, but 

that he was fairly certain that if he asked another researcher to share such codes, he 

would most likely get them. He added that overall compliance with requests to share 

codes would probably be equal to the rate of compliance with requests for sharing data. 

Next, Dr. Curry was asked to respond to the following: "How do the processes of data 

acquisition, analysis and interpretation in paleoclimatology affect practices of data 

sharing in the field? Are any of these processes unique to paleoclimatology?" Dr. Curry 

asked for clarification and was told that the question referred to whether the laborious and 

expensive way in which most data are collected in paleoclimatology had an effect on data 

sharing. He then responded that requests for raw data would be the exception rather than 

the rule, because transforming the raw data into usable information is labor intensive and 

difficult. Nevertheless, because of NSF requirements, he would release all data after two 

years. He added that some scientists, however, do seek to maintain proprietary access to 

their data even after two years. 

Finally, Dr. Curry was asked whether he wanted to share anything else with the 

Investigatory Committee. In his concluding comments to the Investigatory Committee, 

Dr. Curry noted that in the last ten years things have changed rather rapidly with regard 

to sharing data and information. He reported that he has become more aware of how he 

would be affected if people started asking him step-by-step details of his work, and that 

while he has always been diligent about documenting his work, ten years ago he would 

not have been able to document every single step in his analytical work. Thus, "accepted 

practices" are not fixed and are always evolving. 

Page II of 19 
Summary of Investigatory Committee Interview with Professor Jerry McManus, 

Professor, Department of Earth and Environmental Sciences, Columbia University 

- April 20, 2010 

On April 20, 2010, the RA-1O Investigatory Committee (Assmann, Castleman, Irwin, 

Jablonski, and Vondracek) and Candice Yekel interviewed Dr. Jerry McManus, 

Professor, Department of Earth and Environmental Sciences, Columbia University. The 

Investigatory Committee had four prepared questions, but Investigatory Committee 

members were free to ask additional questions as well as follow-up questions as they saw 

fit. 

To start the interview, Dr. McManus was asked to respond to the following question: 

"Would you please tell us what you consider in your field to be accepted standard 

practice with regard to sharing data [and] ... with regard to sharing unpublished 

manuscripts?" Dr. McManus responded by first drawing a distinction between published 

and unpublished data, noting, however, that there is a range of standard practices with 

regard to both. Nevertheless, the mode of behavior regarding unpublished data is to share 

"in a fairly limited fashion with individuals or groups who make specific requests and 

typically who are known to the researcher." Regarding published data, Dr. McManus 

indicated that standard practice is to make such data available through any of a broad 

range of means, including providing access to electronic repositories and institutional 

archives. 

Regarding the sharing of unpublished manuscripts, Dr. McManus indicated that there is a 

broad range of typical and accepted behaviors, with such manuscripts commonly shared 

with a limited number of colleagues. In a follow-up question, it was inquired whether it 

may be considered standard practice to share an unpublished manuscript with others 

without getting express permission to do so from the author. Dr. McManus responded by 

saying "no" to such sharing as standard practice, but allowing that there is not necessarily 

only one acceptable practice, as permission may be given implicitly or explicitly. Without 

specific encouragement for wider distribution, however, it is generally understood, 

according to Dr. McManus, that unpublished papers are not intended for third-party 

distribution. 

The next question was stated as follows: "Would you please briefly explain how codes 

are developed in the process of evaluating data in your field (e.g., are these codes 

significantly different from published software packages)? Then please tell us what you 

consider in your field to be accepted, standard practice with regard to sharing codes." Dr. 

McManus indicated that most, but not all, details of such methods are usually reported 

when research is published, and that some of these details may be shared in a "somewhat 

ad hoc basis." Generally, however, the tendency is to "try to provide the conditions by 

which any research can be replicated .... " Dr. McManus agreed that generally, codes are 

treated the same way as any other method. 

Page 12 of 19 
Summary of Investigatory Committee Interview with Dr. Richard Lindzen, Alfred 

P. Sloan Professor, Department of Earth, Atmospheric, and Planetary Sciences, 

Massachusetts Institute of Technology - May 5, 2010 

On May 5 20 I 0, the RA-I 0 Investigatory Committee (Assmann, Irwin, Jablonski, 

Vondracek; Dr. Castleman was not available) and Candice Yekel interviewed Dr. 

Richard Lindzen, Professor, Department of Earth, Atmospheric, and Planetary Sciences, 

Massachusetts Institute of Technology. The Investigatory Committee had four prepared 

questions, but Investigatory Committee members were free to ask additional questions as 

well as follow-up questions as they saw fit. 

Before the Investigatory Committee's questioning began, Dr. Lindzen was given some 

general background information regarding the process of inquiry and investigation into 

allegations concerning Dr. Mann, with a focus on the particular allegation that is the 

subject of the current review by the Investigatory Committee. Dr. Lindzen then 

requested, and was provided with, a brief summary of the three allegations previously 

reviewed. When told that the first three allegations against Dr. Mann were dismissed at 

the inquiry stage of the RA-lO process, Dr. Lindzen's response was: "It's thoroughly 

amazing. I mean these are issues that he explicitly stated in the emails. I'm wondering 

what's going on?" 

The Investigatory Committee members did not respond to Dr. Lindzen's statement. 

Instead, Dr. Lindzen's attention was directed to the fourth allegation, and it was 

explained to him that this is the allegation which the Investigatory Committee is charged 

to address. Dr. Lindzen was then asked the first question formulated by the Investigatory 

Committee: "Would you please tell us what you consider in your field to be accepted, 

standard practice with regard to sharing data, and the second part of the question is would 

you tell us what you consider in your field to be accepted, standard practice with regard 

to sharing unpublished manuscripts?" 

Dr. Lindzen responded by stating that "with respect to sharing data, the general practice 

is to have it available." With respect to unpublished manuscripts, he indicated that "those 

are generally not made available unless the author wishes to." In response to a number of 

follow-up questions, Dr. Lindzen indicated that if an unpublished manuscript is sent to a 

scientist by the author, it would be common practice to ask for permission before sharing 

it with others; if it was sent by someone else it would be common practice to ask if they 

had permission to share the paper. According to Dr. Lindzen, a scientist might conclude 

that there is implicit permission to disseminate an unpublished paper only when the 

author made it clear that the results may be disseminated. 

The next question inquired whether, in Dr. Lindzen's view, climatologists normally make 

their codes (used in the analysis of data) available for other people to download. Dr. 

Lindzen responded by stating that "it depends." He elaborated, saying that if the codes 

are very standard, it is unnecessary to share them, but if it's an unusual analysis it would 

be his practice to make the codes available to anyone who wishes to check them. In a 

follow-up question, Dr. Lindzen was asked whether he would have issues with people 

Page 13 of 19 
running into compatibility issues or compilation issues. He responded by saying that even 

if people "screw it up" or if you have reservations about sharing codes, "if somebody 

asks you how did you get this, you really should let them know." 

The next questions presented to Dr. Lindzen were as follows: "How do the processes of 

data acquisition, analysis, and interpretation in paleoclimatology affect practices of data 

sharing in the field? Are any of these processes unique to paleoclimatology?" Dr. 

Lindzen indicated that he did not think that these processes are unique to 

paleoclimatology, and that since most of the data are acquired using public funds, there is 

no basis for investigators being proprietary with their data. In response to a follow-up 

question, Dr. Lindzen acknowledged that prior to publication, scientists may have a 

variety of reasons to keep things confidential, but after publication "there's an obligation 

to explain exactly how you got them, especially if they're controversial." 

Standard of proof used by the Investigatory Committee: 

Preponderance a/the evidence (happen more likely than not or 51 % certainty). All 

committee votes are unanimous unless otherwise indicated. 

Level of intent considered by the Investigatory Committee: 

The Investigatory Committee considered various levels of intent in order of increasing 

severity from careless, to reckless, to knowingly, to intentional. These terms are defined 

as follows: 

• careless - a reasonable person would not have known better or honest errorthis is not considered research misconduct. 

reckless - a reasonable person should have known better. 

knowingly - a reasonable person knew better but did it anyway. 

• intentional (purposejil1) - a reasonable person knew better but did it anyway 

with the intent to deceive. 

The level of intent regarding the specific allegation will be addressed below. 

Summary of Investigation: 

The Investigatory Committee investigated the following potential acts of misconduct: 

"Did Dr. Michael Mann engage in, or participate in, directly or indirectly, any actions 

that seriously deviated from accepted practices within the academic community for 

proposing, conducting, or reporting research or other scholarly activities?" 

The Investigatory Committee was given access to 376 files that contained emails stolen 

from the Climate Research Unit (CRU) of the University of East Anglia. These emails 

were either sent by Dr. Mann, sent to Dr. Mann, copied to Dr. Mann, or discussed Dr. 

Mann (but were neither addressed nor copied to him). The Investigatory Committee also 

Page 14 of 19 
reviewed the documents collected by the Inquiry Committee, as well as the Inquiry 

Committee's findings and report. In addition, the Investigatory Committee reviewed a 

number of documents provided by Dr. Mann in response to requests from both the 

Inquiry and Investigatory Committees. A number of public documents were also made 

available to the Investigatory Committee, including a number of editorials, both pro and 

con Dr. Mann, an open letter from 255 members of the National Academy of Sciences, 

published in Science magazine, May 7, 2010, and the full text of the British House of 

Commons' Science and Technology Committee report on "The disclosure of climate data 

from the Climatic Research Unit at the University of East Anglia," which was published 

on March 31, 2010. 

In the course of the investigation, the Investigatory Committee interviewed Dr. Michael 

Mann, as well as his immediate supervisor, Dr. William Easterling, Dean of the College 

of Earth and Mineral Sciences at the Pennsylvania State University. Dean Easterling and 

Dr. David Verardo, National Science Foundation Program Director for Paleo 

Perspectives on Climate Change, agreed to suggest names of eminent scientists who 

might agree to be interviewed by the Investigatory Committee in its efforts to establish 

the range of "accepted practices within the academic community for proposing, 

conducting, or reporting research or other scholarly activities." As previously described, 

the Investigatory Committee contacted, and subsequently interviewed, three eminent 

scientists from the field of climate research: Dr. William Curry, Senior Scientist, Geology 

and Geophysics Department, Woods Hole Oceanographic Institution; Dr. Richard 

Lindzen, Alfred P. Sloan Professor, Department of Earth, Atmospheric, and Planetary 

Sciences, Massachusetts Institute of Technology; and Dr. Jerry McManus, Professor, 

Department of Earth and Environmental Sciences, Columbia University. 

Based on the documentary evidence and on information obtained from the various 

interviews, the Investigatory Committee first considered the question of whether Dr. 

Mann had seriously deviated from accepted practice in proposing his research activities. 

First, the Investigatory Committee reviewed Dr. Mann's activities that involved proposals 

to obtain funding for the conduct of his research. Since 1998, Dr. Mann received funding 

for his research mainly from two sources: The National Science Foundation (NSF) and 

the National Oceanic and Atmospheric Administration (NOAA). Both of these agencies 

have an exceedingly rigorous and highly competitive merit review process that represents 

an almost insurmountable barrier to anyone who proposes research that does not meet the 

highest prevailing standards, both in terms of scientific/technical quality and ethical 

considerations. 

NOAA and NSF research grant proposals are both evaluated through similarly rigorous 

and transparent merit review (peer review) processes. To illustrate, we describe the NSF 

review process, which has two stages. In Stage 1, proposals are sent out to several 

external experts for merit review (mail review) based on the two NSF review criteria 

established by the National Science Foundation Board -- Intellectual Merit and Broader 

Impacts. In Stage 2, the proposal and its external expert reviews (mail reviews) are taken 

to a 8-15 person external expert panel and evaluated over a several day period (panel 

review). Panel review members are not the same persons as the mail review members. In 

Page 15 of 19 
Stage I, the external reviewers only see individual proposals and rate them on a 5-point 

scale in descending order from Excellent, Very Good, Good, Fair, and Poor. In Stage 2, 

the entire panel (except those members who have a conflict of interest with the proposal) 

see all the proposals in the competition (usually about 140 proposals in the NSF program 

to which Dr. Mann has typically submitted his proposals) and rate them based on the 

same two NSF criteria on the same rating scale, but at this stage they evaluate the 

proposals in comparison with all the other proposals that were submitted. All reviews are 

then taken under advisement by the director of the particular NSF program to which the 

proposal was submitted, who then recommends whether a project should be funded. The 

program director is guided by the expert reviews, but may also take programmatic 

balance and other NSF criteria into account before making a final recommendation. The 

rate of funding varies by program, but rarely exceeds 25 percent. 

The results achieved by Dr. Mann in the period 1999-2010, despite these stringent 

requirements, speak for themselves: He served as principal investigator or co-principal 

investigator on five NOAA-funded and four NSF-funded research projects. During the 

same period, Dr. Mann also served as co-investigator of five additional NSF-and 

NOAA-funded research projects, as well as on projects funded by the Department of 

Energy (DOE), the United States Agency for International Development (USAID), and 

the Office of Naval Research (ONR). This level of success in proposing research, and 

obtaining funding to conduct it, clearly places Dr. Mann among the most respected 

scientists in his field. Such success would not have been possible had he not met or 

exceeded the highest standards of his profession for proposing research. 

The second part of the Investigatory Committee's charge was to investigate whether Dr. 

Mann had engaged in any actions that seriously deviated from accepted practices within 

the academic community for conducting research or other scholarly activities. One focus 

of the committee's investigation centered on whether Dr. Mann had deviated from 

accepted practice with regard to sharing data and source codes with other investigators. 

First, the Investigatory Committee established that Dr. Mann has generally used data 

collected by others, a common practice in paleoclimatology research. Raw data used in 

Dr. Mann's field of paleoclimatology are laboriously collected by researchers who obtain 

core drillings from the ocean floor, from coral formations, from polar ice or from 

glaciers, or who collect tree rings that provide climate information from the past 

millennium and beyond. Other raw data are retrieved from thousands of weather stations 

around the globe. Almost all of the raw data used in paleoclimatology are made publicly 

available, typically after the originators of the data have had an initial opportunity to 

evaluate the data and publish their findings. In some cases, small sub-sets of data may be 

protected by commercial agreements; in other cases some data may have been released to 

close colleagues before the originators had time to consummate their prerogative to have 

a limited period (usually about two years) of exclusivity; in still other cases there may be 

legal constraints (imposed by some countries) that prohibit the public sharing of some 

climate data. The Investigatory Committee established that Dr. Mann, in all of his 

published studies, precisely identified the source(s) of his raw data and, whenever 

possible, made the data and or links to the data available to other researchers. These 

Page 16 of 19 
actions were entirely in line with accepted practices for sharing data in his field of 

research. 

With regard to sharing source codes used to analyze these raw climate data and the 

intermediate calculations produced by these codes (referred to as "dirty laundry" by Dr. 

Mann in one of the stolen emails) with other researchers, there appears to be a range of 

accepted practices. Moreover, there is evidence that these practices have evolved during 

the last decade toward increased sharing of source codes and intermediate data via 

authors' web sites or web links associated with published scientific journal articles. 

Thus, while it was not considered standard practice ten years ago to make such 

information publicly available, most researchers in paleoclimatology are today prepared 

to share such information, in part to avoid unwarranted suspicion of improprieties in their 

treatment of the raw data. Dr. Mann's actual practices with regard to making source 

codes and intermediate data readily available reflect, in all respects, evolving practices 

within his field. Dr. Mann acknowledged that early in his career he was reluctant to 

publish his source codes because the National Science Foundation had determined that 

source codes were the intellectual propelty of the investigator. Moreover, because he 

developed his source codes using a specific programming language (FORTRAN 77), 

these codes were not likely to compile and run on computer systems different from the 

ones on which they were developed (e.g., different processor makes/models, different 

operating systems, different compilers, different compiler optimizations). Since then, 

however, he has used a more accessible method for developing his source codes 

(MA TLAB) and he has made all source codes, as well as intermediate data, available to 

the research community, thereby meeting and exceeding standard practices in his field. 

Moreover, most of his research methodology involves the use of Principal Components 

Analysis, a well-established mathematical procedure that is widely used in climate 

research and in many other fields of science. Thus, the Investigatory Committee 

concluded that the manner in which Dr. Mann used and shared source codes has been 

well within the range of accepted practices in his field. 

The issue of whether Dr. Mann had engaged in any actions that seriously deviated from 

accepted practices within the academic community for conducting research or other 

scholarly activities was examined by the Investigatory Committee via a number of 

additional means. When a scientist's research findings are well outside the range of 

findings published by other scientists examining the same or similar phenomena, 

legitimate questions may be raised about whether the science is based on accepted 

practices or whether questionable methods might have been used. Most questions about 

Dr. Mann's findings have been focused on his early published work that showed the 

"hockey stick" pattern of climate change. In fact, research published since then by Dr. 

Mann and by independent researchers has shown patterns similar to those first described 

by Dr. Mann, although Dr. Mann's more recent work has shown slightly less dramatic 

changes than those reported originally. In some cases, other researchers (e.g., Wahl & 

Ammann, 2007) have been able to replicate Dr. Mann's findings, using the publicly 

available data and algorithms. The convergence of findings by different teams of 

researchers, using different data sets, lends further credence to the fact that Dr. Mann's 

conduct of his research has followed acceptable practice within his field. Further support 

Page 17 of 19 
for this conclusion may be found in the observation that almost all of Dr. Mann's work 

was accomplished jointly with other scientists. The checks and balances inherent in such 

a scientific team approach further diminishes chances that anything unethical or 

inappropriate occurred in the conduct of the research. 

A particularly telling indicator of a scientist's standing within the research community is 

the recognition that is bestowed by other scientists. Judged by that indicator, Dr. Mann's 

work, from the beginning of his career, has been recognized as outstanding. For example, 

he received the Phillip M. Orville Prize for outstanding dissertation in the earth sciences 

at Yale University in 1997. In 2002, he received an award from the Institute for 

Scientific Information for a scientific paper (published with co-authors) that appeared in 

the prestigious journal Nature; also in 2002, he co-authored a paper that won the 

Outstanding Scientific Paper Award from the NOAA Office of Oceanic and Atmospheric 

Research, and Scientific American named him as one of 50 leading visionaries in science 

and technology. In 2005, Dr. Mann co-authored a paper in the Journal of Climate that 

won the John Russell Mather Paper award from the Association of American 

Geographers, and in the same year, the website "RealClimate.org" (co-founded by Dr. 

Mann) was chosen as one of the top 25 "Science and Technology" websites by Scientific 

American. In 2006, Dr. Mann was recognized with the American Geophysical Union 

Editors' Citation for Excellence in Refereeing (i.e., reviewing manuscripts for 

Geophysical Research Letters). All of these awards and recognitions, as well as others 

not specifically cited here, serve as evidence that his scientific work, especially the 

conduct of his research, has from the beginning of his career been judged to be 

outstanding by a broad spectrum of scientists. Had Dr. Mann's conduct of his research 

been outside the range of accepted practices, it would have been impossible for him to 

receive so many awards and recognitions, which typically involve intense scrutiny from 

scientists who mayor may not agree with his scientific conclusions. 

The third area of investigation was to address whether Dr. Mann had engaged in any 

actions that seriously deviated from accepted practices within the academic community 

for reporting research or other scholarly activities. Dr. Mann's record of publication in 

peer reviewed scientific journals offers compelling evidence that his scientific work is 

highly regarded by his peers, thus offering de facto evidence of his adherence to 

established standards and practices regarding the reporting of research. To date, Dr. 

Mann is the lead author of39 scientific publications and he is listed as co-author on an 

additional 55 pUblications. The majority of these publications appeared in the most highly 

respected scientific journals, i.e., journals that have the most rigorous editorial and peer 

reviews in the field. In practical terms, this means that literally dozens of the most highly 

qualified scientists in the world scrutinized and examined every detail of the scientific 

work done by Dr. Mann and his colleagues and judged it to meet the high standards 

necessary for publication. Moreover, Dr. Mann's work on the Third Assessment Report 

(2001) of the Intergovernmental Panel on Climate Change received recognition (along 

with several hundred other scientists) by being awarded the 2007 Nobel Peace Prize. 

Clearly, Dr. Mann's reporting of his research has been successful and judged to be 

outstanding by his peers. This would have been impossible had his activities in reporting 

his work been outside of accepted practices in his field. 

Page 18 of 19 
One issue raised by some who read the stolen emails was whether Dr. Mann distributed 

privileged information to others to gain some advantage for his interpretation of climate 

change. The privileged information in question consisted of unpublished manuscripts that 

were sent to him by colleagues in his field. The Investigatory Committee determined that 

none of the manuscripts were accompanied by an explicit request to not share them with 

others. Dr. Mann believed that, on the basis of his collegial relationship with the 

manuscripts' authors, he implicitly had permission to share them with close colleagues. 

Moreover, in each case, Dr. Mann explicitly urged the recipients of the unpublished 

manuscripts to first check with the authors if they intended to use the manuscripts in any 

way. Although the Investigatory Committee determined that Dr. Mann had acted in good 

faith with respect to sharing the unpublished manuscripts in question, the Investigatory 

Committee also found that among the expelis interviewed by the Investigatory 

Committee there was a range of opinion regarding the appropriateness of Dr. Mann's 

actions. Opinions ranged from one expert who contended that it is never acceptable to 

share an unpublished manuscript without first obtaining explicit permission from the 

author(s) to do so, to another expert who felt that, when working with close colleagues, it 

is sometimes acceptable to do so by assuming that implicit permission had been granted. 

The Investigatory Committee considers Dr. Mann's actions in sharing unpublished 

manuscripts with third parties, without first having received express consent from the 

authors of such manuscripts, to be careless and inappropriate. While sharing an 

unpublished manuscript on the basis of the author's implied consent may be an acceptable 

practice in the judgment of some individuals, the Investigatory Committee believes the 

best practice in this regard is to obtain express consent from the author before sharing an 

unpublished manuscript with third parties. 

The Investigatory Committee would like to note that Dr. Mann, after being questioned by 

the Investigatory Committee about this issue, requested and received confirmation that 

his assumption of implied consent was correct from the author of one of the papers in 

question. This "after the fact" communication was not considered by the Investigatory 

Committee in reaching its decision. 

Conclusion of the Investigatory Committee as to whether research misconduct 

occurred: 

The Investigatory Committee, after careful review of all available evidence, determined 

that there is no substance to the allegation against Dr. Michael E. Mann, Professor, 

Department of Meteorology, The Pennsylvania State University. 

More specifically, the Investigatory Committee determined that Dr. Michael E. Mann did 

not engage in, nor did he participate in, directly or indirectly, any actions that seriously 

deviated from accepted practices within the academic community for proposing, 

conducting, or reporting research, or other scholarly activities. 

The decision of the Investigatory Committee was unanimous. 

Page 19 of 19 
snijlijn 

Hier komt bij voorkeur een flaptekst te staan, in plaats 

van deze algemene tekst

Het Planbureau voor de Leefomgeving is hét nationale 

instituut voor strategische beleidsanalyses op het gebied 

van milieu, natuur en ruimte. 

Het PBL draagt bij aan de kwaliteit van het strategische overheidsbeleid door een brug te vormen tussen 

wetenschap en beleid en door gevraagd en ongevraagd, 

onafhankelijk en wetenschappelijk gefundeerd, verkenningen, analyses en evaluaties te verrichten waarbij een 

integrale benadering voorop staat. 

Planbureau voor de Leefomgeving

Locatie Bilthoven

Postbus 303

3720 AH Bilthoven

T: 030 274 274 5

F: 030 274 4479

E: info@pbl.nl

www.pbl.nl

Planbureau voor de Leefomgeving, <maand> 200X

Assessing an IPCC assessment

The Netherlands Environmental Assessment Agency (PBL) has investigated the scientific foundations for the IPCC summary conclusions 

of the Fourth Assessment Report of 2007 on projected regional 

climate-change impacts, at the request of the Dutch Minister for the 

Environment. Overall the summary conclusions are considered well 

founded and none were found to contain any significant errors. 

The Working Group II contribution to the Fourth Assessment Report 

shows ample observational evidence of regional climate-change 

impacts, which have been projected to pose substantial risks to most 

parts of the world, under increasing temperatures. However, in some 

instances the foundations for the summary statements should have 

been made more transparent. While acknowledging the essential role 

of expert judgment in scientific assessments, the PBL recommends to 

improve the transparency of these judgments in future IPCC reports. 

In addition, the investigated summary conclusions tend to single out 

the most important negative impacts of climate change. Although 

this approach was agreed to by the IPCC governments for the Fourth 

Assessment Report, the PBL recommends that the full spectrum of 

regional impacts is summarised for the Fifth Assessment Report, 

including the uncertainties. The PBL believes that the IPCC should 

invest more in quality control in order to prevent mistakes and shortcomings, to the extent possible.

Netherlands Environmental Assessment Agency, July 2010

Assessing an IPCC assessment 

An analysis of statements on 

projected regional impacts 

in the 2007 report
Assessing an IPCC assessment. An analysis of statements on projected regional impacts in 

the 2007 report

©Netherlands Environmental Assessment Agency (PBL)

The Hague/Bilthoven, 2010

PBL publication number: 500216002

Cover photo: Hollandse Hoogte

Corresponding author: leo.meyer@pbl.nl

This publication can be downloaded from: www.pbl.nl/en. A hard copy may be ordered from: 

reports@pbl.nl, citing the PBL publication number or ISBN.

Parts of this publication may be reproduced, providing the source is stated, in the form: 

Netherlands Environmental Assessment Agency, the publication title.

The Netherlands Environmental Assessment Agency (PBL) is the national institute for 

strategic policy analysis in the field of environment, nature and spatial planning. We contribute 

to improving the quality of political and administrative decision-making by conducting outlook 

studies, analyses and evaluations in which an integrated approach is considered paramount. 

Policy relevance is the prime concern in all our studies. We conduct solicited and unsolicited 

research that is both independent and always scientifically sound.

Netherlands Environmental Assessment Agency

Office The Hague Office Bilthoven

PO Box 30314 PO Box 303

2500 GH The Hague 3720 AH Bilthoven

The Netherlands The Netherlands

Telephone: +31 (0)70 328 8700 Telephone: +31 (0)30 274 2745

Fax: +31 (0)70 328 8799 Fax: +31 (0)30 274 4479

E-mail: info@pbl.nl

Website: www.pbl.nl/en
Foreword 5

Foreword

In January 2010, the media reported two errors in specific parts of the Fourth 

Assessment Report of 2007 by the Intergovernmental Panel on Climate Change 

(IPCC). The media reports gave rise to questions about the credibility of the overall 

IPCC assessment: could politics and society still rely on the IPCC for an assessment 

of the scientific knowledge on climate change?

On 28 January 2010, the Dutch Parliament asked Jacqueline Cramer, the then Dutch 

Minister for the Environment, for an investigation into the implications of these 

errors in the IPCC report. Minister Cramer subsequently requested the Netherlands 

Environmental Assessment Agency (PBL)1) to make an assessment of the reliability 

of the regional chapters (9 to 16) of the IPCC Working Group II contribution to the 

Fourth Assessment Report (the part on Impacts, Adaptation and Vulnerability), and 

to assess the effects of any errors on the summary conclusions drawn by the IPCC.

From the outset it was clear that the PBL would not be able to repeat the original 

IPCC Working Group II assessment, given the sheer volume of the scientific work 

reported in the IPCC exercise.2) What we were able to do, however, was investigate 

the extent to which the IPCC in their summaries had presented existing scientific 

knowledge to the world of policymakers, in a way that was supported by the 

underlying texts and scientific references. What is more, we also investigated what 

could be learned from our findings so that informed choices can be made in future 

assessments. This seemed the more reasonable approach, as behind the request 

from the Dutch Parliament was a more general concern: could policymakers and 

the public at large still trust the IPCC's key messages?

A scientific assessment – an analysis and weighing of the actual scientific 

knowledge – of a complex problem such as climate change is a tremendously 

difficult task. The IPCC is a bridge between science and policy: a so-called ‘science–

policy interface’. The IPCC is not conducting science, nor is it making policy, but 

should provide the guarantee that policymakers can come to their decisions on 

the basis of the best available knowledge regarding climate change. We have 

1) The Netherlands Environmental Assessment Agency (PBL) is an independent governmental body that by statute provides 

the Dutch Government and Parliament – and the European Commission, European Parliament and UN organisations – with scientific advice on problems regarding the environment, sustainability and spatial planning. PBL authors have contributed to several 

assessments of the IPCC. During the Third and Fourth Assessments, the PBL hosted the Co-Chair and Technical Support Unit of 

Working Group III (Mitigation on Climate Change).

2) Elsewhere we have reported on new insights into climate science since the Fourth Assessment Report (cf. News in Climate 

Science and Exploring Boundaries, 2009). 
6 Assessing an IPCC assessment 

investigated how this task was taken on by the IPCC Working Group II in their 

assessment of the regional impacts of climate change. Based on this investigation 

we have formulated some recommendations to further improve the quality of 

these assessments. 

The investigation was conducted realising that climate science and policymaking 

are currently taking place in a new era, characterised by a high degree of 

politicisation, a much more dynamic interaction between science and public 

discourse, and vocal citizens who either want to know whether policy measures 

under discussion are all really necessary, or who are of the opinion that suggested 

measures do not go far enough. This new context comes with new requirements 

for the IPCC assessments. Our suggestions are thus not always meant as criticism 

of the ‘architecture’ of the climate report of Working Group II as this was being 

drawn up in the years leading up to its publication in 2007. Yet we believe that, by 

being critical and self-critical of work contained in the Fourth Assessment Report, 

we can contribute to further improving the reliability of future IPCC reports. It is an 

exercise in future-oriented learning. 

Accomplishing our task would have been impossible without the cooperation of 

the IPCC authors of Working Group II, who were willing to devote a considerable 

amount of their time to answering our questions, and did so within the strict 

time tables we had set them. It is clear that the responsibility for the research, 

as well as for the presentation of the findings, the conclusions drawn, and 

the recommendations made, lies solely with the Netherlands Environmental 

Assessment Agency (PBL). 

Our investigation is focused in character and size. However, a number of 

conclusions and recommendations concern the set up of the procedures and 

processes of the IPCC. Hopefully, those suggestions can still play a role in the 

review by the InterAcademy Council, that will become available by late August 2010, 

focusing on procedural and process issues of the IPCC.3)

Professor Maarten Hajer

Director, Netherlands Environmental Assessment Agency (PBL)

3) InterAcademy Council 2010, Review of the IPCC; an evaluation of the procedures and process of the 

Intergovernmental Panel on Climate Change, http://reviewipcc.interacademycouncil.net/index.html
Foreword 7

Contents

 Executive summary 9

Part I – Main findings

 1 Introduction 19

1.1 Scientific assessment in aid of policy 19

1.2 Some background on IPCC Working Group II 22

1.3 Scope, objectives and limitations of this report 24

1.4 Structure of the report 26

 2 Methodology 27

2.1 Investigative approach 27

2.2 Typology of errors and qualifications 29

 3 Results and discussion 33

3.1 Status of IPCC summary statements on regional climate-change impacts 33

3.2 Errors 35

3.3 Risk-oriented approach 37

3.4 Comments 40

3.5 Conclusion 43

 4 Recommendations 45

4.1 Minimising the risk of errors 45

4.2 Investing in the improvement and transparency of foundations of summary conclusions 46

4.3 Strengthening the quality control by the chapter teams 47

4.4 Strengthening the review process 47

4.5 Timing of the assessments 48

4.6 Balancing the assessment of climate-change impacts 48

4.7 Investing in climate-change impact science 50

Part II – Detailed analysis of regional chapters and summaries

 5 Africa 53

5.1 Analysis of statements in Table SPM.2 of the Synthesis Report 53

5.2 Additional findings 55

5.3 Findings from the PBL registration website 56
8 Assessing an IPCC assessment 

 6 Asia 59

6.1 Analysis of statements in Table SPM.2 of the Synthesis Report 59

6.2 Additional findings 60

 7 Australia and New Zealand 63

7.1 Analysis of statements in Table SPM.2 of the Synthesis Report 63

7.2 Additional findings 64

 8 Europe 67

8.1 Analysis of statements in Table SPM.2 of the Synthesis Report 67

8.2 Additional findings 68

8.3 Findings from the PBL registration website 70

 9 Latin America 73

9.1 Analysis of statements in Table SPM.2 of the Synthesis report 73

9.2 Additional findings 74

 10 North America 77

10.1 Analysis of statements in Table SPM.2 of the Synthesis Report 77

10.2 Additional findings 78

 11 Polar regions (Arctic and Antarctic) 79

11.1 Analysis of statements in Table SPM.2 of the Synthesis Report 79

 12 Small islands 81

12.1 Analysis of statements in Table SPM.2 of the Synthesis Report 81

12.2 Additional findings 81

 Annex A Table SPM.2 of the Synthesis Report: examples of some projected 

regional impacts 83

 Annex B The error on the melting of the Himalayan glaciers 86

 Annex C The error on the Dutch land area below sea level 89

 Annex D Sea level rise: consequences for the Netherlands 91

 Annex E Abbreviations 96

 Colophon 99
Executive summary 9

Executive summary

The Netherlands Environmental Assessment Agency (PBL) has investigated the scientific 

foundations for the IPCC summary conclusions of the Fourth Assessment Report of 2007 

on projected regional climate-change impacts, at the request of the Dutch Minister for 

the Environment. Overall the summary conclusions are considered well founded and 

none were found to contain any significant errors. The Working Group II contribution to 

the Fourth Assessment Report shows ample observational evidence of regional climatechange impacts, which have been projected to pose substantial risks to most parts of the 

world, under increasing temperatures. However, in some instances the foundations for the 

summary statements should have been made more transparent. While acknowledging 

the essential role of expert judgment in scientific assessments, the PBL recommends to 

improve the transparency of these judgments in future IPCC reports. In addition, the 

investigated summary conclusions tend to single out the most important negative impacts 

of climate change. Although this approach was agreed to by the IPCC governments for 

the Fourth Assessment Report, the PBL recommends that the full spectrum of regional 

impacts is summarised for the Fifth Assessment Report, including the uncertainties. The 

PBL believes that the IPCC should invest more in quality control in order to prevent 

mistakes and shortcomings, to the extent possible. 

No significant errors found in summary conclusions

The foundations for thirty-two IPCC Fourth Assessment summary conclusions 

on the regional impacts of climate change have been investigated. These 

conclusions show examples of projections of climate-change impacts on food, 

water, ecosystems, coastal regions and health, for all the earth’s continents. These 

conclusions have not been undermined by errors, although one of the conclusions 

contains a minor inaccuracy: in hindsight, not 75 to 250 million people, but 90 to 

220 million people are projected to be exposed to increased water stress due to 

climate change in Africa, by 2020. Given the large uncertainties surrounding such 

projections, this difference is not significant.

Provenance of summary statements needs to become more transparent in future 

reports

Seven of the investigated 32 conclusions on the regional impacts of climate change 

contain information that we were unable to sufficiently trace to the underlying 

chapters in the IPCC Working Group II Report or to the references therein. For 

two of these conclusions, our critical comments pertain to insufficiently founded 

generalisations from existing scientific research, in both cases from local to regional 

scales, and for one of them also from one type of livestock to livestock in general. 

The PBL recommends to invest in the improvement and transparency of the 

foundations of summary conclusions in future IPCC reports.
10 Assessing an IPCC assessment 

The regional chapters: one significant error and some comments

In the underlying regional chapters, in addition to the two already known errors 

about the melting of the Himalayan glaciers and about the Dutch land area below 

sea level, another significant error was found: a 50 to 60% decrease in productivity 

in anchovy fisheries on the African west coast was projected on the basis of an 

erroneous interpretation of the literature references. It appeared to be about a 50 

to 60% decrease in extreme wind and seawater turbulence, with some effects on 

the anchovy population that were not quantified. We found certain inaccuracies, 

ranging from (very) small errors in numbers to imprecise literature references. 

In addition, the PBL has some critical comments to make. One of these relates 

to the fact that the report does not specify how many of the additional heatrelated deaths projected for Australian cities are actually attributable to climate 

change – a sizeable fraction is due to demographic changes alone. However, these 

shortcomings do not affect the investigated 32 summary conclusions or other parts 

of the IPCC summaries.

Examples of negative impacts dominate at summary level

The IPCC Working Group II Report focuses on climate impacts, adaptation and 

vulnerability. It was found that, in the IPCC’s highest level summary, the conclusions 

that were derived from the regional chapters of the Working Group II contribution 

to the Fourth Assessment Report single out examples of projections of negative

climate-change impacts. The IPCC authors considered these to be the most relevant 

to policymakers. The PBL has labelled this as a ‘risk-oriented’ approach, which had 

been implicitly endorsed by the governments that constitute the IPCC (including 

that of the Netherlands). The PBL subscribes to the importance of an approach 

that highlights what may go wrong under unmitigated climate change, but the 

Working Group II Report lacked a clear explanation of the choice of approach and 

its consequences. Alternatively, it could be argued that policymakers should be 

presented with a complete picture in the Summaries for Policymakers, not just with 

negative examples (without suggesting that potential positive effects cancel out 

potential negative effects). We recommend that, for the Fifth Assessment Report, 

the choice of the approach taken is made explicit. Moreover, we suggest that, at 

Summary for Policymakers level, two separate sections are included dealing with 

projected regional impacts on water, food, ecosystems, coastal regions and health: 

 One section that describes the projected full range of climate-change impacts, 

including uncertainties, positive impacts, and the relative contributions from 

other important areas, such as industrialisation, population growth, and land 

use;

 One section that describes the most important negative impacts, including 

a worst-case risk approach, based on a clearly explicated risk-assessment 

methodology.

No consequences for overarching conclusions

Our findings do not contradict the main conclusions of the IPCC on impacts, 

adaptation and vulnerability related to climate change. There is ample 

observational evidence of natural systems being influenced by climate change 

on regional levels. The negative impacts under unmitigated climate change in the 

future pose substantial risks to most parts of the world, with risks increasing at 

higher global average temperatures. 
Executive summary 11

Other recommendations 

Additional recommendations for future IPCC publications include:

 Create a public website for the submission of possible errors found in the 

published reports;

 Provide stronger underpinning of generalisations of case studies to entire 

regions or sectors, also making use of regional modelling studies;

 Ensure that statements that attribute impacts to climate change are well 

founded in scientific research, including systematic observations, modelling 

and statistics. The climate change component of impacts should be carefully 

characterised.

 Be careful with phrasing of statements that could be perceived by readers as 

heightening the projected impacts of climate change;

 IPCC governments should provide financial support for hiring chapter assistants 

to assist with quality control;

 Assure that the reviews of all draft texts are fully covered by several expert 

reviewers;

 Strengthen the expert and government reviews of the foundation for and 

provenance of statements in the summaries;

 IPCC governments should increase their investments in climate-change 

observations and modelling in developing countries. 

The Dutch Parliament resolution and the subsequent assignment given to the PBL

In January 2010, worldwide media attention was given to two errors that were 

discovered in a part of the Fourth Assessment Report of the IPCC of 2007: an 

erroneously high rate of melting of the Himalayan glaciers, and an erroneously high 

percentage of land area in the Netherlands lying below sea level. The commotion 

in the Dutch media and the subsequent discussion at the political level in the 

Netherlands led to a resolution by the Dutch Parliament on 28 January 2010, which 

declared in the preamble that the reliability of the IPCC should be undisputed but 

was now at issue. In the resolution, the parliament required that the government 

would instruct the Netherlands Environmental Assessment Agency (PBL) to provide 

a new update on climate science, including the implications of the said errors. 

Based on this resolution and the debate in parliament, the Minister for the 

Environment decided to limit her question to PBL to an investigation in the 

implications of possible errors in the regional chapters of the IPCC report of 2007 

on climate-change impacts, adaptation and vulnerability, paying specific attention 

to the Himalayan glaciers.

Specification of the assignment 

The PBL, subsequently, broadened the assignment somewhat by including an 

investigation of the conclusions on regional impacts, as described in the highest 

summary level of the combined IPCC 2007 reports (the ‘Synthesis Report’). In 

addition, the investigation report would also include annexes on the Himalayan 

glaciers, on the percentage of land area in the Netherlands lying below sea level, 

and on the protection of the Netherlands against sea level rise. The last annex 

was added, since the errors about the Himalayan glacier melting and about the 

percentage of Dutch land area below sea level had triggered a political discussion 

on whether Dutch policies dealing with sea level rise would need to be revised. 
12 Assessing an IPCC assessment 

For the ‘update on climate science’, as requested by the Dutch Parliament, we refer 

to a PBL study ‘News in Climate Science and Exploring the Boundaries’ (December, 

2009). That report, drawn up together with the Royal Netherlands Meteorological 

Institute (KNMI) and Wageningen University and Research Centre (WUR), revisited 

the IPCC conclusion that global warming since the middle of the 20th century is very 

likely to have been due to human influence on the global climate. This statement 

was found to be robust, even when taking into account peer-reviewed scientific 

literature expressing doubts on this relation. We took this and other findings of 

the Working Group I as a point of departure for our investigation into the IPCC 

statements on the projected regional impacts of climate change.1)

Given the constraints regarding time and capacity, it was not possible for the PBL to 

check a hundred per cent of all texts and references in the eight regional chapters 

of the Working Group II Report for errors, considering that it had taken hundreds of 

authors and reviewers to produce the report over the course of five years. Instead, 

we limited ourselves to the IPCC summary statements, and framed the central 

questions of this report as follows: 

Are the summary conclusions on regional impacts well founded on the underlying 

chapters and literature references? Are there errors in statements that have travelled 

from the scientific literature references and/or the main texts through to the summary 

conclusions? If errors are found, do they affect the validity of these conclusions? What 

recommendations can we derive from our investigation in order to further improve the 

quality of the assessment process for the Fifth Assessment Report (due in 2014)? 

The role of IPCC assessments and the role of the PBL

Policymakers need to be well informed about the state of the art in climate science. 

Obtaining a reliable overview of the scientific literature, with its many different and 

sometimes contradicting messages, is something policymakers can obviously not 

achieve on their own. What is more, a knowledge base that is shared and accepted 

as credible and legitimate by all parties involved, can be of great value to the policy 

process. Therefore, in 1989, the UN member governments that constitute the 

IPCC arranged a process in which worldwide groups of carefully selected expert 

authors, periodically, would assess the ever growing amounts of scientific literature 

on climate. Working within the confines of the IPCC, authors arrive at collective 

judgments that are supposed to present the state of the art in climate science, 

in a comprehensive, unbiased, non-policy-prescriptive way. The strict procedures 

established by the IPCC member governments are to guarantee precisely this. The 

credibility of an IPCC report depends both on the knowledge and integrity of the 

author teams and on the thoroughness and expertise of expert and government 

reviewers.

The PBL generally has an intermediate role in interpreting IPCC reports, so that 

these can be applied in policy-making by the Dutch Government and Parliament. 

In this case, the PBL has had to assess the reliability of some key messages derived 

from a particular IPCC report; the Working Group II Report. To this end, we needed 

1) There is a forthcoming publication by the Royal Netherlands Academy of Sciences (KNAW) that 

will examine this fundamental part of climate science in more detail.
Executive summary 13

to rely on the expert judgments of IPCC authors, to some extent, as it would have 

been impossible to track down and assess all theory, measurements, observations, 

model calculations, and all arguments that the IPCC authors may have used during 

the five-year assessment period in order to arrive at certain conclusions. However, 

we could investigate whether the key messages in the summary were being backed 

up by material discussed in the main chapters of the IPCC assessments, whether 

relevant information in the main chapters, in our opinion, had been founded 

on sound scientific sources, and whether there would be any obvious errors or 

mistakes in the (presentation of) the material investigated.

IPCC assessments are always snapshots in time – representative of the knowledge 

of science as it is at that moment. As science proceeds, insights may change. 

However, in our investigation, we have not included new scientific information that 

has become available since the Fourth Assessment Report was produced – our task 

was not to redo the IPCC assessment, but to investigate its underpinning at the 

time of finalisation, in early 2007.

Focus of the investigation 

The 2007 Fourth Assessment Report of the IPCC contains almost 3,000 pages. The 

focus of our investigation was on only a small part of this report. The IPCC report 

contains four volumes: the Working Group I Report on the physical science basis of 

climate change; the Working Group II Report on climate impacts, adaptation and 

vulnerability; the Working Group III Report on mitigation of climate change; and 

a Synthesis Report, integrating the main outcomes of these three Working Group 

reports.

The Working Group II Report contains 20 chapters, eight of which deal with the 

regions Africa, Asia, Australia and New Zealand, Europe, Latin America, North 

America, polar regions (Arctic and Antarctic), and small islands (that is, small-island 

states). 

In the Synthesis Report a table is presented, in its Summary for Policymakers, 

describing ‘examples of some projected regional impacts’, derived from the eight 

regional chapters in the Working Group II Report, each containing four statements, 

which together form 32 statements (see Annex A for a copy of this table). We 

have considered this table in the Synthesis Report as the highest summary level 

presenting the most important regional impacts, with the highest visibility to the 

policymakers.2)

The primary focus of this investigation, therefore, was on the foundation of these 

32 statements, which are a subset of largely identical statements in the Summary 

for Policymakers of the Working Group II Report.

2) The Synthesis Report was released a few weeks before the important world climate Summit of 

the UNFCCC (the UN climate convention) in Bali in December 2007 (also indicated as CoP13). At this 

conference, the UN parties to the convention took the first steps towards a long-term climate policy 

strategy following the first commitment period of the Kyoto protocol. 
14 Assessing an IPCC assessment 

Analysis of the main texts and the summaries

Each IPCC report consists of several layers. Each chapter in a Working Group report 

contains a main text and its literature references, with an ‘Executive Summary’ up 

front. Each Working Group report has both an extensive ‘Technical Summary’ which 

synthesises and summarises the information of all chapters, and a brief ‘Summary 

for Policymakers’. 

We analysed the Executive Summaries of each of the eight regional chapters, and 

checked their provenance from the relevant parts of the underlying texts in the 

main chapters and their key references. We did the same with regional information 

in the Technical Summary and the Summary for Policymakers of the Working Group 

II Report. In addition, we checked how these conclusions were entered into the 

main text of the Synthesis Report, and how they ultimately landed in its Summary 

for Policymakers (see Figure 2.1).

Registration website

The PBL launched a public website that would be available for the course of one 

month, in order to give all experts in the Netherlands the opportunity to contribute 

to our investigation. We asked for submissions of possible errors found in regional 

chapters of the Working Group II Report. By the end of that month, the PBL had 

registered 40 submissions; however, most of them were about issues related to the 

Working Group I Report. Two submissions qualified to be addressed in our report. 

All submissions and PBL’s responses have been published on the PBL website.

Process

We invited the IPCC authors of the regional chapters of the Working Group II 

Report to comment on our draft findings, and in our communication we went 

through several iterations. In some cases, we also contacted the authors of cited 

references. The draft of our report was also reviewed by selected internal and 

external, national and international experts. Moreover, this project has been 

conducted under the independent supervision of the Royal Netherlands Academy 

of Arts and Sciences (KNAW).

Methodology

This investigation has resulted in a report on:

1. findings referring to the foundation for the 32 summary conclusions on regional 

impacts in the Synthesis Report (main objective);

2. additional findings at lower levels, such as those referring to the foundations 

for statements on regional impacts in the Technical Summary and Executive 

Summaries of the regional chapters, and also some findings on parts of the main 

text that we came across while working on other findings. As our focus was on 

tracing possible errors that would affect the key messages to policymakers, we 

were unable to systematically check all statements in the main texts.

For our investigation, we used several criteria for assessing the quality of IPCC 

statements. First of all, we employed a distinction between obvious factual ‘errors’ 

– which we believe require an erratum on the IPCC Fourth Assessment web pages 

– and ‘comments’, the latter being critical remarks, made from our specific position 

of having to assess these statements on behalf of Dutch policymakers. These 
Executive summary 15

criteria were developed ‘on the job’, while categorising our findings. In total, there 

are nine criteria, and associated with these are two types of ‘errors’ (E1-E2) and 

seven types of ‘comments’ (C1-C7):

 E1: Inaccuracy of a statement; 

 E2: Inaccuracy of literature referencing;

 C1: Insufficiently substantiated attribution, when a certain impact had been 

attributed to climate change without convincing substantiation;

 C2: Insufficiently founded generalisation, when case studies had been 

extrapolated to whole regions/sectors without convincing foundation;

 C3: Insufficiently transparent expert judgment, when we could not trace the 

reasoning behind a statement from underlying texts or literature references 

given;

 C4: Inconsistency of messages, when found between the different layers in the 

IPCC report;

 C5: Untraceable reference, when a literature reference could not be found;

 C6: Unnecessary reliance on grey referencing, when reference had been made 

only to grey literature (literature other than from peer-reviewed journals), while 

adequate peer-reviewed scientific journal references would have been available;

 C7: Statement unavailable for review, when new information had been 

introduced after the last review, but was not clearly derived from a content issue 

raised in this last review.

When a certain statement in the IPCC Fourth Assessment Report would meet all the 

criteria, the statement was considered to be well founded and reliable.

Melting of Himalayan glaciers

In response to the request from the Dutch Minister for the Environment to provide 

an update on the melting of Himalayan glaciers, we performed an assessment of 

the new scientific information that had become available since the publication of 

the IPCC’s Fourth Assessment Report (see Annex B). The average, region-wide, 

glacier area retreat rate is probably between 0.1 and 0.5% per year. Although the 

glacier area will shrink substantially this century, especially in the most vulnerable 

eastern zone of the Himalayas, glaciers (such as Khumbu and Imja), will not 

disappear entirely, or even mostly, by 2035, as stated in the Working Group II 

Report.

The annual contribution from Himalayan glaciers to sea level rise is 0.06±0.04 mm 

or ~2% of the current annual sea level rise of 3.1±0.7 mm. The rate of Himalayan 

glacier melting appears to have increased over time, but so has sea level rise, and 

therefore the 2% contribution may still apply.

Dutch land area below sea level

The error about the percentage of Dutch land area below sea level stems from a 

text submitted by the PBL (see Annex C). The Working Group II Report states that 

55% of Dutch land area is below sea level. This should have read that 55% of the 

Netherlands is prone to flooding: 26% of the country is at risk because it lies below 

sea level and another 29% is susceptible to river flooding.
16 Assessing an IPCC assessment 

Sea level rise and its consequences for the Netherlands

Errors in the IPCC report regarding the Himalayan glaciers and the Dutch land 

area below sea level have triggered a discussion on whether Dutch policies that 

deal with sea level rise should be revised (see Annex D). The answer is clearly ‘no’: 

the contribution from melting Himalayan glaciers to sea level rise is very limited 

(around 2%) and the policy concerning safety against flooding in the Netherlands, 

as formulated in the National Water Policy Plan 2009-2015 (Ministry of Transport, 

Public Works and Water Management, 2009) and elaborated in the Delta 

Programme, is well founded on the available knowledge on climate change, sea 

level rise, river discharges and uncertainties. The current discussions on the IPCC 

have no effect on the assumptions under the National Water Policy Plan and the 

Delta Programme.
Part I – Main findings 17

Part I – Main findings
18 Assessing an IPCC assessment 
Introduction 19

Introduction

1.1 Scientific assessment in aid of policy

The characteristics of climate change qualify it as the most complex type of 

policy problem. It resists direct perception, hence, making policymakers and the 

public depend on experts to describe its potentially problematic features. It is a 

phenomenon, the problematic features of which may manifest themselves within 

a matter of decades, while preventative measures assume more direct action. It 

is also a global phenomenon which can only be handled effectively through joint 

participation of all countries (climate change is a classic ‘free rider’ problem). 

Addressing climate change is not a matter of controlling one clearly identified 

source, as was the case with the depletion of the ozone layer, but rather of 

controlling a multiplicity of sources. This characteristic of ‘wickedness’ (cf. Rittel 

and Webber, 1973) makes that climate science plays a very central role in the 

policy debate. To make things worse, a full understanding of the subject depends 

on the input from a large variety of scientific fields, with differing degrees of 

standing knowledge relevant to measuring and understanding processes of climate 

change, its effects and possible remedial strategies. Since both the stakes and the 

uncertainties are high, the problem of climate change needs to be addressed by 

using ‘post-normal science’; a problem-solving strategy that involves paying specific 

attention to qualitative judgments and value commitments permeating science 

(Funtowicz and Ravetz, 1993).

The role of assessments

Historically, the IPCC was established to present international policymakers with 

undisputed, policy-relevant knowledge, while acknowledging uncertainties (see 

e.g. Petersen, 2006). Since its inception in 1989, the IPCC has been effective at 

this. However, the science on climate-change impacts is still beset with large 

uncertainties, some of which are irreducible. Next to the nature of climate science, 

it is relevant to remember that perceptions of climate-change risk vary widely, both 

across the globe and within societies (cf. e.g. Hulme, 2009). 

Hence, the issue of climate change is complex, both in causes and effects and in 

terms of policy implications. The uncertainties, different interests and different risk 

perceptions pertaining to all aspects of causes, consequences and response options 

are inextricably linked to the issue of climate change. 

In such situations, scientific assessments are the institutional vehicle for presenting 

policymakers with information that would help them define and decide on policy 

strategies. Scientific assessments present the state of knowledge relevant to a 

1
20 Assessing an IPCC assessment 

particular policy problem. It is a subtle tool. For complex problems such as climate 

change, recent history has shown that a traditional, linear science-for-policy model 

– one that is based on a clearly formulated policy question followed by a sequence 

of scientific research from hypotheses to experiments and, finally, agreed on 

answers – will not work. The nature of the issue is too complex and uncertainties 

will always remain. However, the fact that uncertainties about climate change 

are large does not mean that scientific information should not or could not play a 

role in decision-making. On the contrary, well-formulated scientific information is 

considered a crucial component in addressing complex policy problems. 

The tool of scientific assessment, such as that implemented through the 

Intergovernmental Panel on Climate Change (IPCC), has been developed as a way 

of organising the science–policymaker dialogue (see, e.g., Social Learning Group, 

2001). Scientific assessment is the analysis and review of information derived from 

existing research, in the context of specific decision-making processes, in order to 

better understand the wide implications of a policy problem and to identify and 

evaluate possible response actions. Such assessment usually does not entail new 

research. The design of the process of scientific assessment needs to ensure that 

uncertainties are properly addressed and that different societal perspectives are 

included in the process, so that politicians may obtain a better sense of the possible 

futures facing the world, and of the choices that could be made. One of the 

functions of independent assessments is to assess and communicate uncertainty, 

and to make sure that the problem is analysed from different value perspectives. 

The proper course of action then is for policymakers to decide what – if anything – 

should be done.

The IPCC assessment process

This report investigates the quality of IPCC’s key messages on regional impacts 

of climate change, which were the outcome of a long production process. We 

have not reiterated the elaborate IPCC production process with its procedures for 

review. This has been described extensively in the Principles Governing IPCC Work

(http://www.ipcc.ch/pdf/ipcc-principles/ipcc-principles.pdf; for a summary, see 

http://www.ipcc.ch/pdf/ipcc-principles/IPCC%20Procedures.pdf). These procedures 

are meant to guarantee a responsible assessment process. 

It is important to note, however, that an IPCC assessment is not the product 

of an extensive bureaucratic operation. Instead, its organisational structure is 

aptly characterised as being a network organisation that spans the whole globe. 

Scientific experts from varying continents collaborate and assemble overviews 

of the knowledge in their respective fields. The quality and integrity of the IPCC 

process depends, to a large extent, on the quality and integrity of its procedures, 

and, hence, on the commitment of all contributing authors to adhere to those 

rules and procedures. In response to experiences and criticism, these procedures 

have been regularly upgraded over the course of more than 20 years to guarantee 

increasing assurance of high levels of quality and integrity.

We have chosen not to elaborate on the institutional set up of the IPCC, but have 

considered it a given. For the present report, we had set ourselves the task of 

assessing the transparency with which scientific facts on climate-change impacts 
Introduction 21

have ‘travelled’ from their source (e.g., scientific peer-reviewed literature) to their 

destinations: the Summaries for Policymakers. This final destination of scientific 

facts within the IPCC process has driven the investigation presented in the present 

report. The IPCC process should deliver useful and reliable knowledge (cf. Morgan, 

2010) to policymakers; one route to check this – followed for this report – is by 

tracing summary conclusion back to their origins.

The role of expert judgment

This report deals with the questions that have been raised on the reliability of the 

assessment process run by the IPCC. In particular, questions have been raised 

concerning the Working Group II contribution to the Fourth Assessment Report 

with its focus on Impacts, Adaptation and Vulnerability and including the regional 

chapters that were home to the two errors that caused the upset in the media 

and in society, in the winter of 2009–2010. This report has been based on the 

assumption that the quality of the IPCC scientific assessment can be measured by 

tracing statements on the effects of climate change, for example, as presented 

at the highest level of summaries for policymakers (in the Synthesis Report and 

the Working Group II Report), back to their original sources. There is obviously 

a limit to the traceability of such statements. Given the mismatch between the 

state of standing knowledge on complex environmental phenomena, on the one 

hand, and the questions raised by policymakers, on the other, those conducting 

an assessment often have to arrive at so-called expert judgments: a weighing of 

understandings and analyses to come up with the best possible assessment of 

observations, or of what one might expect to happen. Such expert judgments are a 

crucial component of any assessment. Given the fact that these judgments cannot 

be traced back to any one source in the (peer-reviewed) literature, the quality of 

these assessments grows as the expert judgments are made more transparent, that 

is, by explaining how particular conclusions have been arrived at. This procedure 

allows others to follow a certain line of reasoning, similar to how this generally 

is done in articles published in scientific journals. Another way of enhancing the 

quality of expert judgments is to present them in the form of a deliberation, stating 

the way in which key conclusions have been arrived at. This then allows other 

scientists to contradict inferences, or come up with alternative explanations.

Expert judgments are inextricably linked to assessments and are used in all phases 

of the IPCC assessment process. The lead authors of the regional chapters of the 

Working Group II Report were supposed to have read all the literature relevant 

to that particular region, and to be familiar with the environmental policy issues 

of that region. Based on their individual expertise, the lead authors collectively 

arrived at judgments on the impacts of climate change. The review process of the 

IPCC relies on two rounds of expert review and one round of government review, 

as well as on Review Editors, who umpire the review process. The objective of this 

process is to ensure a balance in the lead authors’ judgments as these are published 

in the report. The chapters themselves are very detailed. This explains the need for 

summary statements that convey the most important conclusions to policymakers. 

The Coordinating Lead Authors drafted the Summary for Policymakers of the 

Working Group II Report, and a smaller group of authors (consisting of two regional 

Coordinating Lead Authors, the two Co-Chairs and the head of the Technical 

Support Unit of Working Group II) was involved in the drafting of the Summary for 
22 Assessing an IPCC assessment 

Policymakers of the Synthesis Report. Moreover, in both of these Summaries for 

Policymakers, the role of expert judgments had been crucial for determining the 

main policy-relevant findings from the underlying chapters. 

Since it is impossible for most readers of IPCC reports to understand and check 

every reference, all data, models, calculations and measurements, they must be 

able to rely on the quality control and quality assurances given by the assessment 

process. To some extent, those readers have no choice other than to assume an 

expert judgment to reflect the best of current knowledge. However, they should 

be able to trace the plausibility of a judgment in a Summary for Policymakers back 

to the main text and references. When dealing with issues as complex as climate 

change, expert judgment is inevitable. But precisely because this is a matter of 

sound judgment, there is a virtue in maximising the possibility for others to verify 

and validate the analyses made by the IPCC authors.

1.2 Some background on IPCC Working Group II

Since the Third Assessment Report (finalised in 2001), the work of the IPCC has 

been organised into the following three distinct Working Groups: 

 Working Group I addresses the physical science of climate change; 

 Working Group II addresses the impacts of and vulnerability to climate change 

and the means for adaptation; 

 Working Group III addresses the mitigation of greenhouse gas emissions. 

Working Group II addresses impacts, vulnerability and adaptation to climate change 

with respect to human and natural systems. This particular focus calls for the 

mobilisation of authors mostly from physical, geographical and ecological sciences, 

but also from the social sciences. The particular thematic focus of Working Group 

II immediately complicates its assessment task. The interactions within biological 

systems and between biological systems and physical chemical and social systems, 

are very complex. What is more, the patterns of interaction vary, depending on 

local situations and on human choices and actions. 

Literature on climate impacts is not only very diverse, and the intensity of academic 

scholarship is divided very unevenly over the globe. This increases the challenge 

of making good assessments of what the impacts, effects and vulnerabilities of 

climate change are in those regions that have the least data to draw on. This point 

is well illustrated by a map from the IPCC report itself (see Figure 1.1).

Furthermore, impact analyses are not only spread unevenly around the world, 

it is also a field of study where often insights are found outside of the strict 

academic tradition. For instance, many impact analyses are presented in reports 

by governments and inter-governmental or private, non-academic institutions – 

given the fact that not all topics in these reports are equally suitable for, or indeed, 

targeted to scientific publication – which complicates the task of assessors. The 

IPCC guidelines specifically allow for this literature also to be assessed (in order to 

provide relevant input about local and sectoral situations and to minimise bias in 

information between developing and industrialised countries, for instance, because 
Introduction 23

impacts in developing countries are reported even less in scientific peer-reviewed 

journals). 

The scientific uncertainties that the IPCC, including Working Group II, are 

confronted with, are large and deep. Since its inception, the IPCC has given 

increasing attention to the management and reporting of uncertainties (Swart 

et al., 2009). The socio-ecological systems studied by Working Group II are not 

always suitable for numerical modelling; hence, there is a lack of information on 

quantitative uncertainty. Impact analyses or the study of vulnerabilities often 

depend on empirical observation using different scientific techniques, such as field 

research or case study analysis. Because of the uncertainties involved, it is difficult 

to establish which impacts occur, and why and to what extent observed effects 

could be attributed to climate change. Often authors need to make assessments 

of the risks that are associated with various levels of climate change, rather than 

indicating best-estimates and clearly defined uncertainty levels. In order to estimate 

Locations of significant observations of changes in physical an biological systems together 

with changes in surface air temperature, over the 1970–2004 period. Source: Climate 

Change 2007: Impacts, Adaptation and Vulnerability. Working Group II Contribution 

to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, 

Figure 1.8. Cambridge University Press.

Figure 1.1

Observed data series

Physical systems (snow, ice and frozen ground;

hydrology; coastal processes)

Biological systems (terrestrial, marine, and freshwater)

* Circles in Europe represent 1 to 7,500 data series.

1 – 30

31 – 100

101 – 800

801 – 1,200

1,201 – 7,500

Temperature change o

C

1970 – 2004

-1.0 3.52.01.00.2-0.2

Observed changes in biological and physical systems

Europe *

Figure 1.1
24 Assessing an IPCC assessment 

the possible risks of climate change, also events with very low probability but with 

large consequences have to be dealt with. 

In the context of this report, it is important to understand the mandate of 

Working Group II. Formally, Working Group II ‘assesses the scientific, technical, 

environmental, economic and social aspects of vulnerability (sensitivity and 

adaptability) to climate change and the negative and positive consequences for 

ecological systems, socio-economic sectors and human health, with an emphasis 

on regional, sectoral and cross-sectoral issues.’ (see http://www.ipcc.ch/working_

groups/working_groups.htm)

1.3 Scope, objectives and limitations of this report

In January 2010, worldwide media attention was drawn to two errors that were 

discovered in a part of the Fourth Assessment Report of the IPCC of 2007: an 

erroneously high rate of melting of the Himalayan glaciers, and an erroneously high 

percentage of land area in the Netherlands lying below sea level. The latter error 

was not the fault of the IPCC but of the PBL (see Annex C). The commotion in the 

Dutch media and subsequent discussions at the political level in the Netherlands led 

to a resolution by the Dutch Parliament on 28 January 2010, which declared in the 

preamble that the reliability of the IPCC, which should be undisputed, was now at 

issue.

In the resolution, the parliament required that the government would charge the 

Netherlands Environmental Assessment Agency (PBL) with providing a new update 

of climate science, including the implications of the said errors. Based on this 

resolution and the debate in parliament, the Minister for the Environment decided 

for the PBL to limit its investigation to the implications of possible errors in the 

regional chapters of the IPCC report of 2007 on climate-change impacts, adaptation 

and vulnerability.

The assessment presented in this report was based on an investigation of the 

original sources of the IPCC’s Fourth Assessment summary conclusions (in 

particular, the Synthesis Report, Table SPM.2, see Annex A) that pertain to a part 

of one of the Fourth Assessment Reports (that is, Chapters 9 to 16 of the Working 

Group II Report), and was aimed to derive general observations and identify 

possible improvements. 

Two additional tasks were taken on. First of all, the Minister for the Environment 

asked the Netherlands Environmental Assessment Agency (PBL) to make an 

assessment of the scientific literature on Himalayan ice, published since 2007. 

This was triggered by the need to communicate what scientists do know about 

this topic; after all, the public turmoil concerned the IPCC error that stated that 

all Himalayan ice would be melted by 2035. The results from this assessment are 

presented in Annex B. 

Second, the parliamentary debate also showed concerns over the possible 

repercussions of the state of knowledge on the issue of sea level rise for climate-
Introduction 25

change adaptation measures in the Netherlands. Therefore, the state-of-the-art 

scientific foundation on this issue is summarised in Annex D.

The PBL has taken the utmost care to prevent possible prejudice in their present 

assessment of the conclusions derived from the regional chapters of the Working 

Group II Report. A crucial mechanism to avoid any prejudice was the installation 

of a supervisory committee under the responsibility of the Royal Netherlands 

Academy of Arts and Sciences (KNAW). The KNAW was asked to supervise the 

quality of this PBL study. The KNAW appointed a scientific committee, whose role 

it was to judge whether the PBL performed its research without prejudice and 

according to established procedures of quality control.1)

Realistically speaking, a thousand-page assessment by hundreds of authors 

involving thousands of reviewers conducted within a limited timeframe could 

hardly be expected to be free of errors. Therefore, it is to be expected that some 

inaccuracies, insufficiently justified statements or other irregularities, escape even 

the most thorough drafting and review procedures. Our evaluation methodology 

was to take the statements in the various IPCC summaries as starting points and 

consider their source of origin (where did they come from?). A full evaluation of 

the reliability of statements would entail redoing the original IPCC assessments. 

Our limited evaluation could never have repeated or improved on the IPCC drafting 

and review process in a comprehensive way, nor was it our aim to redo the IPCC 

assessment. Nevertheless, we think that such a limited evaluation by scientifically 

educated policy analysts, who are not experts in climate-impacts science, is a useful 

means for the IPCC to receive feedback from an extended peer community (cf. 

Funtowicz and Ravetz, 1993). Through this evaluation, we reflect on the qualitative 

judgments and value commitments that necessarily permeate the IPCC assessment 

process.

Without recruiting experts with the range of expertise and following the careful, 

multi-step review that has characterised the IPCC reports, any review of the 

contents of IPCC chapters only has limited potential. The current PBL assessment, 

therefore, extends only to checking whether observations and interpretations had 

been captured conscientiously, how statements had been substantiated within 

IPCC reports, and to evaluate the transparency with which important expert 

judgments had been made. The analysis focused on investigating the ‘internal’ 

consistency of the report, and did not perform new, fresh reviews of the evidence 

available at the time the Fourth Assessment Report was produced. In sum, the 

analysis focused on assessing statements by looking for errors (inaccuracies in 

statements and referencing) and by applying a short list of quality criteria (see 

Chapter 2), which was developed specially for the purpose of this analysis. 

In sum, the central questions addressed in this report are the following: 

Are the summary conclusions on regional impacts well founded on the underlying 

chapters and literature references? Are there errors in statements that have travelled 

1) The terms of reference of the supervision by the Royal Netherlands Academy of Arts and Sciences 

can be found at http://www.knaw.nl/pdf/Instellingsbesluit_Toezicht_Cie_PBL.pdf (in Dutch).
26 Assessing an IPCC assessment 

from the scientific literature references and/or the main texts through to the summary 

conclusions? If errors are found, do they affect the validity of these conclusions? What 

recommendations can we derive from our investigation in order to further improve the 

quality of the assessment process for the Fifth Assessment Report (due in 2014)? 

1.4 Structure of the report

Part I of the report contains the main findings. Chapter 2 describes the 

methodology of our assessment. Chapter 3 presents the results from our analysis of 

IPCC Fourth Assessment’s statements on regional climate-change impacts. Chapter 

4 contains our recommendations for the IPCC assessment process, following from 

our analysis in Chapter 3. Part II provides our detailed analysis of the regional 

chapters and the summaries. 

References

Funtowicz, S.O. and Ravetz, J.R., 1993. ‘Science for the post-normal age’, Futures 25: 739–755.

Hulme, M., 2009. Why We Disagree about Climate Change: Understanding Controversy, Inaction and 

Opportunity. Cambridge: Cambridge University Press.

Morgan, M.S., 2010. ‘Traveling facts’, in P. Howlett and M.S. Morgan, eds., How Well Do Facts Travel?, 

Cambridge: Cambridge University Press. (http://www2.lse.ac.uk/economicHistory/pdf/Morgan/

Travelling%20facts%20_2_.pdf)

Petersen, A.C., 2006. Simulating Nature: A Philosophical Study of Computer-Simulation Uncertainties 

and Their Role in Climate Science and Policy Advice. Het Spinhuis Publishers, Apeldoorn and 

Antwerp. Download from http://hdl.handle.net/1871/11385.

Rittel, H.W.J. and M. Webber, 1973. ‘Dilemmas in a general theory of planning’, Policy Sciences, 4 (2): 

155-169.

Social Learning Group, 2001. Learning to Manage Global Environmental Risks, Volume 1: A 

Comparative History of Social Responses to Climate Change, Ozone Depletion, and Acid Rain, 

Cambridge, MA: MIT Press.

Swart, R., L. Bernstein, M. Ha-Duong, A. Petersen, 2009. ‘Agreeing to disagree: uncertainty 

management in assessing climate change, impacts and responses by the IPCC’, Climatic Change 

92: 1–29.
Methodology 27

Methodology

2.1 Investigative approach

The two primary aims of the investigation were to assess, first of all, whether errors 

would affect the content of the IPCC’s key statements on regional climate-change 

impacts, and, second, whether these key statements would meet seven quality 

criteria formulated by the PBL (see Section 2.2). These key statements were taken 

from the Summary for Policymakers of the IPCC Working Group II Report (2007) 

and, in part, presented in Table SPM.2 of the IPCC Synthesis Report (2007) (see 

Annex A). Figure 2.1 offers a graphical explanation of the area of investigation, 

Figure 2.1A places the statements on regional impacts of climate change in the 

context of the whole IPCC Fourth Assessment Report, and Figure 2.1B represents 

the approach to the chapter analysis. Table SPM.2, containing 32 bulleted 

statements on the most important projected impacts (four per region), is located at 

the top of a ‘pyramid’.

A secondary aim of our investigation was to identify errors and assess the quality 

of statements on regional climate-change impacts at levels below the Summary for 

Policymakers of the Working Group II Report.

In Figure 2.1B, the ‘pyramid’ shows the analysis levels that we used in our 

investigation. The Working Group II 2007 Report itself contains four layers. Level 

1 contains the main chapter texts, level 2 contains the Executive Summaries 

of the individual chapter texts, level 3 contains the Technical Summary (giving 

an extensive overview of the whole report), and level 4 is the Summary for 

Policymakers (a short document, approved line-by-line by the governments 

involved in the IPCC). Each chapter text refers to hundreds of literature sources on 

which the assessment was founded (‘level 0’). In some cases, we also investigated a 

few sources that were mentioned in the literature references (‘level −1’)

The Synthesis 2007 Report has only two layers, a main report, divided into six 

topics, and a Summary for Policymakers. The information in the Synthesis Report 

has a higher level of aggregation than that in the Working Group Reports (level 5). 

The information in the Summary for Policymakers of the Synthesis Report has the 

highest level of aggregation (level 6). 

The heuristic approach that was followed by PBL policy analysts involved in this 

study, was to start by reading the Executive Summaries (level 2) of the regional 

chapters with a critical eye, and from there work upwards to level 6 (Summary for 

Policymakers of the Synthesis Report) as well as downwards, to level −1. We made 

2
28 Assessing an IPCC assessment 

Figure 2.1A shows the structure of the IPCC’s Fourth Assessment Report, 2007. The Working 

Group II Report contains eight chapters (from a total of 20 chapters) describing the impacts 

in eight world regions. The main conclusions of the reports by the three Working Groups 

were summarised per report, and also included in the Fourth Assessment Synthesis Report. 

The last contains examples of some projected regional impacts; see Table SPM.2 (Annex A). 

This table was the starting point in our investigation. Figure 2.1 B shows the approach to 

the regional chapter analysis. The statements from Table SPM.2, at the top of the ‘pyramid’, 

were analysed by investigating their foundation in the five consecutive levels, and in the 

relevant underlying literature.

Stucture of the Fourth Assessment Report and regional chapter analysis Figure 2.1

Literature references

Underlying literature

Regional chapter texts Working Group II

Regional chapter including Executive summaries

• Ch. 9 Africa

• Ch. 10 Asia

• Ch. 11 Australia – 

 New Zealand

• Ch. 12 Europe

Technical Summary Working Group II 2007 report

• Section 4.2, Regional Impacts, Adaptation and 

 Vulnerability p. 48 – 58

• Box TS.6, The main projected impacts for regions, 

 p. 59 – 63

• Tabel TS.4, Example of regional impacts, p.67

Summary for policy makers Working Group II 2007 report 

(SPM WG II) p. 13 – 15.

Synthesis report 2007

Topic 3 (Climate Change and its impacts), section 3.2, 

para 3.3.2, Impacts on Regions, p.50 and 52

Summary for policy makers (SPM) Synthesis report (SYR)

Table SPM.2, Examples of some projected regional 

impacts – see Annex A

Executive summaries chapter texts

Working Group II

Technical summary Working Group II

Summary for policy makers

Working Group II

Synthesis report main text

Level of aggregation

Level of 

detail

Summary for policy

makers (SPM)

Synthesis report (SYR)

• Ch. 13 Latin America

• Ch. 14 North America

• Ch. 15 Polar regions 

 (Arctic and Antarctic)

• Ch. 16 Small Islands

B. Regional chapter analysis pyramid

-1

0

1

2

3

4

5

6

A. Structure of the Fourth Assessment Reports

Table SPM.2

Example of some

projected regional

impacts

(see Annex A)

Working Group I 

report

The Physical

Science Basis

Working Group II 

report

Impacts, Adaptation

and Vulnerability

The AR4 Synthesis

Report

Working Group III 

report

Mitigation of

Climate Change

Figure 2.1
Methodology 29

sure that we covered all the bulleted statements in Table SPM.2, looking for any 

possible weaknesses, thus, effectively working from the top layer of the Synthesis 

Report down to the more detailed regional chapter texts of the Working Group II 

Report, and in many cases to the underlying key literature references. In our search 

for the underpinning of statements, we also looked for evidence in the introductory 

and sectoral chapters of the Working Group II Report and, when this was deemed 

necessary, in the Working Group I Report.

Registration website

The PBL launched a public website that would be available for the course of one 

month, in order to give all experts in the Netherlands the opportunity to contribute 

to this investigation. We asked for submissions of possible errors found in regional 

chapters of the Working Group II Report. By the end of that month, the PBL had 

registered 40 submissions; however, most of them were about issues related to the 

Working Group I Report. Five submissions were related to the regional chapters of 

the Working Group II Report; two of those five were included in our report, three 

submissions did not qualify for inclusion. All submissions and PBL’s responses have 

been published on the PBL website.

Process

We invited the IPCC authors of the regional chapters of the Working Group II 

Report to comment on our draft findings, and in our communication we went 

through several iterations. In some cases, we also contacted the authors of cited 

references. The draft of our report was also reviewed by selected internal and 

external, national and international experts. Moreover, this project has been 

conducted under the independent supervision of the Royal Netherlands Academy 

of Arts and Sciences (KNAW).

2.2 Typology of errors and qualifications

The typology of errors and qualifications (‘comments’) presented here was arrived 

at through an inductive process – the results from which are outlined in this section. 

A distinction was made between ‘major’ and ‘minor’ errors or comments. Issues 

were labelled to be ‘major’ when we judged them to have a significant quantitative 

impact on statements pertaining to food, water, ecosystems, coastal areas, and 

health.

Errors

We used a sharp definition: ‘errors’ were considered inaccuracies in the IPCC Fourth 

Assessment Report that we believed to require repair through an erratum or a 

redoing of the assessment of the specific issue at hand. Experts – in particular, PBL 

policy analysts and IPCC authors – reached a high level of agreement on what they 

considered to be an ‘error’ in this precise sense. Both the Himalayan glacier melting 

by 2035 and the 55% of the Netherlands that was said to be below sea level, were 

considered to be such ‘errors’. 
30 Assessing an IPCC assessment 

We distinguished two types of errors:

E1. Inaccurate statement

Statements should be an accurate and correct reflection of the material found 

in literature, augmented by the expert judgment of a writing team. If a statement appeared to be factually incorrect, then we considered whether this was 

a minor imprecision or inaccuracy or if a major mistake had been made that 

could have led to misinterpretations by policymakers. We applied the following 

subcategories to this type of error: 

E1a) Inaccuracies that can be corrected by issuing an erratum – This would be 

mistakes that could be simply corrected without new scientific assessment 

considerations, such as: typographical errors, incorrect phrasing of part of a 

sentence, wrong dimensions, and wrong reference years. The error on the 

percentage of Dutch land area below sea level belongs to this subcategory (see 

Annex C).

E1b) Errors that would in fact require a redoing the assessment of the specific issue 

at hand, such as: establishing a new range of numbers by revised calculations 

from the reference sources available during the assessment period, and/ or 

rephrasing of the expert judgment including its uncertainty labelling.1) The 

Himalayan glacier error belongs to this subcategory (see Annex B).

E2. Inaccurate referencing

References used in statements should be correct. Whenever we found a reference to a wrong source, or when the source was not correctly cited, we qualified this to be ‘inaccurate referencing’. In all cases, an erratum was proposed.

Qualifications

In addition to the errors in the Fourth Assessment Report that could be corrected. 

either in an erratum or through reassessment, we distinguished between seven 

types of qualifications or comments that would signify the quality of a statement.

C1. Insufficiently substantiated attribution

In some cases, observed or projected impacts on systems are described as 

being ‘due to climate change’, ‘temperature increase’ or other climate parameters. Since, often, there are other drivers that could be involved in causing such 

an impact (e.g. population growth, industrialisation, migration, and changes 

in land use and land cover), care must be taken to provide the proper context 

in IPCC statements about the effects of climate change, by referring to other 

causes and/or stresses that may influence the impacts discussed. Statements 

that, in our judgment, did not sufficiently signal the presence of multiple 

stresses, or suggested a one-sided attribution of impacts to climate change 

while other factors also would have been expected to play a critical role, were 

qualified as ‘insufficiently substantiated attribution’.

1) This, of course, excludes new findings in literature published after completion of the assessment 

reports.
Methodology 31

C2. Insufficiently founded generalisation

In some cases, the impacts, for instance, on certain types of livestock or 

countries, were generalised or extrapolated to include other livestock or entire 

regions. In such cases, we believe authors ought to make clear why they think 

that the evidence in the references justifies such generalisation. Statements, 

for which such argumentation was lacking, and for which such evidence could 

not be found in the report or in the references, were qualified as ‘insufficiently 

founded generalisation’. 

C3. Insufficiently transparent expert judgment

Expert judgment on results that can be found in the scientific literature, is an 

essential part of the IPCC assessment process; this includes giving an indication 

of uncertainties. An assessment is thus not a simple summary of the literature. 

The expertise of an international group of authors adds value by judging the 

importance and relative weight of the information from the literature. For 

the sake of transparency, we believe that the reasoning behind these judgments, including the reasoning behind their levels of likelihood and/or confidence2), should be accessible to a non-expert reviewer who attempts to trace 

the travelling of facts from the referenced literature to the Summaries for 

Policymakers.

If, in our opinion, this was not the case, and this fact was found to be problematic, we qualified the particular judgment as an ‘insufficiently transparent expert 

judgment’. This does not imply that this IPCC expert judgment was wrong, since 

the authors as a group may have had their reasons, and may have considered additional information or knowledge that was not explicitly referred to. 

However, in such cases a convincing substantiation of the statement was 

lacking in the report. 

C4. Inconsistency of messages

When moving up from statements in the main chapter texts to statements in 

the Summary for Policymakers of a Working Group Report or of the Synthesis 

Report, the sentences were inevitably shortened and the wording may have 

needed some adjustments. However, Summary for Policymaker texts need 

to be consistent with the main text, as required by the IPCC procedures. This 

holds both for the meaning of the message as well as for the level of certainty. 

SPM approval session decisions include a list of issues that need to be checked 

for consistency with the Technical Summary and the main chapters in preparation of the final publication. Whenever we found that message content and/

or confidence level had changed when it moved from main text to Summary 

2) The IPCC authors were in fact requested to produce a ‘traceable account’ of their assessment 

of uncertainty in expert judgments in the Fourth Assessment process (see Guidance Notes for Lead 

Authors of the IPCC Fourth Assessment Report on Addressing Uncertainties, July 2005, http://www.

ipcc-wg1.unibe.ch/publications/supportingmaterial/uncertainty-guidance-note.pdf). For the Third 

Assessment, a similar guidance was given (Uncertainties in the IPCC TAR, 2000, http://stephenschneider.stanford.edu/Publications/PDF_Papers/UncertaintiesGuidanceFinal2.pdf). It is our general impression that this part of the guidance has never been fully implemented in the assessment process. Also, 

the option to have separate traceable accounts underlying the reports has not been pursued.
32 Assessing an IPCC assessment 

for Policymakers and also if there was any inconsistency between chapters, we 

qualified this as an ‘inconsistency of messages’.

C5. Untraceable reference

As stated before, references used in statements should be accurate. If we 

could not find the reference at all, we qualified this as an ‘untraceable reference’. 

C6. Unnecessary reliance on grey referencing

Grey literature was an indispensable part of the Working Group II assessment 

(this also applies to Working Group III), since not all relevant literature had 

been published in peer-reviewed scientific journals. The grey literature needed 

to meet quality standards before it could be used. There are huge differences 

in the types of literature, going from thoroughly reviewed scientific books to 

much less reviewed – or even non-reviewed – publications. When a substantial scientific statement only referred to a grey publication, although strong 

peer-reviewed journal references could have been referred to at the time, we 

qualified this to be ‘unnecessary reliance on grey referencing’.

C7. Statement unavailable for review

In the preparation of a Final Draft Report, the results from government and 

expert reviews were included. This could mean that new literature sources 

would be added in order to properly address the Second Order Draft (SOD) 

review comments. Since reviews of a SOD often asked for the addition of 

sources, such additions had to be considered as acceptable revisions. However, 

if a new statement contained new literature findings that were not clearly 

derived from a content issue raised in the Second Order Draft review, then 

this new material would have been kept out of the review process. This is not 

supposed to happen; expert reviewers or governments must have access to 

all findings that will be part of the published report. In the cases in which these 

new literature findings were added as described above, we qualified them as a 

‘statement unavailable for review’.

It is important to note that the assignments to these seven qualifications were not 

necessarily agreed on by all the experts involved in our investigation, including 

the IPCC authors. These assignments were based on the judgment of PBL policy 

analysts who had to establish the reliability of IPCC reports as part of their work at 

the science–policy interface. The aim of this exercise has been to identify potential 

areas for improvement in future IPCC assessments. The underlying assumption 

being that times have changed and that, consequently, quality and transparency 

requirements for the IPCC have been raised.
Results and discussion 33

Results and 

Discussion

This Chapter consists of four parts. First, it presents the appraisal of the statements 

on regional climate-change impacts that appear in the Summary for Policymakers of 

the Synthesis Report. Second, an overview is given of errors found in the Working 

Group II Fourth Assessment Report. Third, it addresses the approach followed by 

Working Group to arrive at its summary conclusions on the regional impacts of 

climate change. And fourth, a thematic overview is given of our comments on the 

IPCC report, aimed at further improving the IPCC process. 

Our detailed analysis of the regional chapters and summaries can be found in Part II 

of the present report.

3.1 Status of IPCC summary statements on regional climate-change impacts

In this investigation we have not encountered errors that undermine the summary 

conclusions of the Fourth Assessment Report of the IPCC on the regional impacts 

of climate change. The Working Group II Report presents ample evidence of 

observations of climate-change responses by physical and biological systems. 

Furthermore, the consequences of climate change generally impose a serious 

risk given the projections on future climate change of Working Group I. However, 

there is considerable room for scientific debate on statements that refer to 

climate-change impacts on a regional scale. We have found that, the more specific 

statements become in time and space, the larger the uncertainties. In addition, the 

body of scientific publications is smaller, in particular where it concerns local and 

regional changes occurring in developing countries.

Of all the summary statements on the regional impacts in the Working Group II 

Report – a subset of which is included in the 32 statements in Table SPM.2 of the 

Synthesis Report – only one was found to be affected by a minor error. This minor 

error, in our view, requires a response and reassessment by the IPCC. 

Besides checking the summary statements for possible errors, we also tested 

the quality of these statements with respect to a range of issues: attribution, 

generalisation, transparency of expert judgment, consistency of messages, 

traceability of references, credibility of references, and availability of statements 

for review (see Section 2.2). We have one or more comments to make on seven 

3
34 Assessing an IPCC assessment 

Results from PBL analysis for 32 statements contained in Table SPM.2 of the 

Synthesis Report

Region Topic (and system/sector) Error in statement? Comments?

Africa Water stress 

(freshwater resources)

Yes (minor) No

Yields from rain-fed agriculture 

(food, fibre and forest products)

No Yes (major)

Cost of adaptation to sea level rise 

(coastal systems and low-lying areas)

No Yes (minor)

Arid and semi-arid land 

(ecosystems)

No No

Asia Freshwater availability 

(freshwater resources)

No Yes (major)

Flooding from sea and rivers 

(coastal systems and low-lying areas)

No No

Natural resources and the environment 

(ecosystems)

No No

Diarrhoeal disease 

(health)

No No

Australia and 

New Zealand

Biodiversity loss 

(ecosystems)

No Yes (minor)

Water security 

(freshwater resources)

No No

Production from agriculture and forestry 

(food, fibre and forest products)

No No

Risks from sea level rise and increases in storms and coastal flooding 

(coastal systems and low-lying areas)

No No

Europe Inland flash floods, coastal flooding and erosion 

(industry, settlement and society; coastal systems and low-lying areas)

No No

Mountainous areas 

(ecosystems)

No Yes (minor)

Southern Europe 

(most systems/sectors)

No No

Heat waves and forest fires 

(health)

No No

Latin America Disappearance of tropical forest

(ecosystems)

No No

Biodiversity loss

(ecosystems)

No No

Productivity in crops and livestock

(food, fibre and forest products)

No Yes (major)

Water availability

(freshwater resources)

No No

North America Western mountains and water resources

(freshwater resources)

No No

Rain-fed agriculture

(food, fibre and forest products)

No Yes (minor)

Heat waves

(health)

No No

Coastal communities and habitats

(coastal systems and low-lying areas)

No No

Polar regions Biophysical effects

(ecosystems)

No No

Impacts for human communities

(industry, settlement and society)

No No

Infrastructure and indigenous lifestyle

(industry, settlement and society)

No No

Table 3.1
Results and discussion 35

of the 32 statements contained in Table SPM.2 of the Synthesis Report. If the 

criteria, as used in this investigation for assessing the quality of statements, would 

be systematically applied in future IPCC assessments, it would become easier for 

the users of these assessments, such as the PBL, to establish the reliability of IPCC 

conclusions. Table 3.1 summarises the results for the statements contained in Table 

SPM.2 of the Synthesis Report.

3.2 Errors

Inaccurate statements (code: E1)

Table 3.2 presents all newly found errors in the regional chapters of the Fourth 

Assessment Working Group II Report, as well as the error regarding the Himalayan 

glacier melting and the error about the percentage of Dutch land area below sea 

level. Of these errors, only one travelled up to the Summaries for Policymakers. The 

range for the number of people projected to be exposed to increased water stress 

due to climate change in Africa, by 2020, turned out to have been erroneously 

calculated by adding up the ranges for African sub-regions that were given in 

the underlying reference. The author of the original study – who had recently 

discovered this error – performed a recalculation, which led to a slightly smaller 

uncertainty range (90 to 220 million people, instead of 75 to 250 million). We 

considered this a minor mistake, since the difference does not significantly change 

the message and the original paper did not provide the details that would have 

allowed the correct computation of this range. In our view, the IPCC should quickly 

respond to and assess this slight inaccuracy, even though it is not significant given 

the uncertainties surrounding such projections.

Only one major new error was found, in Chapter 9 on Africa. A projected decrease 

by 50 to 60% in extreme wind and turbulence over fishing grounds was mistakenly 

represented as a 50 to 60% decrease in productivity as a result of changes in wind 

and turbulence. However, this error and the remaining five other errors that were 

newly found did not travel to the level of the summaries, and therefore do not 

affect the summary conclusions. This is also the case for the two errors that were 

discovered before we began our investigation (concerning the melting rate of the 

Himalayan ice and the percentage of Dutch land area below sea level).

Region Topic (and system/sector) Error in statement? Comments?

Vulnerability of ecosystems and habitats

(ecosystems)

No No

Small islands Sea level rise

(coastal systems and low-lying areas)

No No

Coastal conditions

(coastal systems and low-lying areas)

No No

Water resources

(freshwater resources)

No No

Invasion by non-native species

(ecosystems)

No No
36 Assessing an IPCC assessment 

Inaccurate referencing (code: E2)

Table 3.3 provides an overview of all newly found errors in referencing in the 

regional chapters of the Fourth Assessment Working Group II Report. We found 

four instances of inaccurate referencing, which could and should be repaired by 

issuing errata. These inaccuracies have no effect on the summary conclusions.

Inaccurate statements identified by the PBL in the IPCC Working Group II Report 

and the Synthesis Report of 2007a 

Region Pages Error

Major/

minor 

error

Simple 

erratum/ 

response 

needed

Effect on Summary 

for Policymakers 

WG II and 

Synthesis Report

Africa 11 (SYR), 

13, 435, 444

Increased water stress by 2020

‘between 75 and 250 million people are project to 

be exposed’ should be ‘between 90 and 220 million 

people are projected to be exposed’ (see Section 5.1)

Minor Response 

needed

Yes

435, 444 Increased water stress by 2050s

‘350-600 million people’ should be ‘350-

490 million people’ (see Section 5.2)

Minor Response 

needed

No

448 Productivity decrease in fisheries under doubling of CO2

‘extreme wind and turbulence could decrease 

productivity by 50-60%, while turbulence will 

probably bring about a 10% decline in productivity in 

the spawning grounds, and an increase of 3% in the 

main feeding grounds’ should be ‘the frequency of

extreme wind and turbulence events could decrease 

by 50-60%, while mean turbulence will probably 

decline by 10% in the spawning grounds and increase 

by 3% in the main feeding ground of anchovy (IPCC 

proposed an alternative erratum; see Section 5.2)

Major Erratum No

Asia 493 Melting of Himalayan glaciers

The second paragraph of Section 10.6.2 of the 

Working Group II contribution and a repeat 

of part of the paragraph in Box TS.6 refer to 

poorly substantiated estimates on rate of 

recession and date for the disappearance of 

Himalayan glaciers (2035). (See Annex B)

Major Response 

needed

No

Australia and 

New Zealand

50 Observed warming since 1950

‘0.3 to 0.7 °C’ should be ‘0.4 to 0.7 °C’ (see Section 7.2)

Minor Erratum No

Europe 547* Percentage of Dutch land area below sea level

‘below sea level’ should be ‘at risk of 

flooding’ (see Section 8.2)

Major Erratum No

555 Decreases in crop yields by 2050

‘legumes −30 to +5%’ should be ‘legumes −14 to +1%’;

‘sunflower −12 to +3%’ should be 

‘sunflower −12 to +1%’; and

‘tuber crops −14 to +7%’ should be ‘tuber 

crops −9 to +8%’ (see Section 8.2)

Minor Erratum No

Latin America 583 Frequency of hurricanes

‘frequency and intensity of hurricanes’ 

should be ‘frequency and intensity of 

intense hurricanes’ (see Section 9.2) 

Minor Erratum No

598 People living in water-stressed watersheds

‘Net increases in the number of people living in 

water-stressed watersheds’ should be ‘Number of 

people living in water-stressed watersheds and net 

increases due to climate change’ (see Section 9.2)

Minor Erratum No

a These statements pertain to regional impacts of climate change (addressed in Working Group 

II Chapters 9–16), together with the two known inaccuracies. Unless stated otherwise, the page 

numbers refer to the Working Group II Report.

* Erratum already published by IPCC (last checked June 25th).

Table 3.2
Results and discussion 37

The errors traced in the investigation presented above do not undermine the 

conclusions of the IPCC on the regional effects of climate change. 

3.3 Risk-oriented approach

The authors of the Summary for Policymakers of the Working Group II Fourth 

Assessment Report – and especially those of the Summary for Policymakers of the 

Synthesis Report – during the Fourth Assessment were asked by the Working Group 

II Co-Chairs to identify the conclusions in the Working Group II assessment as ‘key 

findings’. These key findings would, in turn, be described in two ways: as textual 

descriptions of ‘main projected impacts’, and as a tabulated range of possible 

impacts scaled to climate change. With regard to the main projected impacts, 

examples of those that travelled to the Synthesis Report, Co-Chair Dr. Parry, 

explained the approach to us as follows:

‘In outline: i) we asked author teams to identify a) a tabulated range of impacts for which 

uncertainty ranges were known and b) a description of the ‘main projected impacts’ (as well 

as vulnerabilities and adaptation issues); ii) we made clear that by ‘main’ we meant the 

most significant challenges and opportunities that need to be faced; iii) we asked that each 

author team’s list should cover the main relevant impact fields (e.g. water, food, health, 

coasts, ecosystems, etc.) but not exhaustively if some were not relevant; iii) reviewers and 

review editors commented on the selection of main projected impacts, and the lists were 

frequently revised; v) the order of listing is not a rank order; vi) there should be a line of sight 

connection between the listings of conclusions, i.e. from executive summary of the chapter, 

through the Technical Summary, the Summary for Policymakers of the Working Group II 

Report and the Synthesis Report, but vii) the wording of the conclusions might need to vary 

according to its context in these different places. . . . 

[Thus] our conclusions list the main projected impacts. Most of these impacts involve 

potential losses in human welfare and therefore are negative. This stems from two things: 

i) because the balance of impacts is, indeed, negative in welfare terms and also ii) because 

positive impacts can often be exploited by mechanisms other than policy, i.e. autonomous 

adaptation.’ (Parry, personal communication, 20 June 2010)

Inaccurate references newly identified by the PBL in the IPCC Working Group II 

Report of 2007, Chapters 9–16

Region Page Proposed erratum

Asia 59, 471 Coral reefs

The reference to Wilkinson (2004) should be replaced by Wilkinson (2000) (see Section 6.2)

Australia and 

New Zealand

518 Great Barrier Reef

The references to Berkelmans et al. (2004), Crimp et al. (2004) and Jones et al. (2004b) 

should be replaced by a reference to Sheehan et al. (2006) (see Section 7.1)

Latin America 598 Water supply

The references to Vásquez (2004) and UNMSM (2004) should be interchanged (see Section 9.2)

Small islands 697 Freshwater lens

The reference to World Bank (2000) after ‘would reduce the freshwater 

lens by 65%’ should be replace by Falkland (1999) (see Section 12.2)

Table 3.3
38 Assessing an IPCC assessment 

The approach of Working Group II towards the selection of specific conclusions 

from the regional chapters has thus focused on which impacts would be the most 

important for policymakers to respond to. 

The Synthesis Report, on page 47 (Section 3.3 on ‘Impacts of future climate 

change’), gives a brief explanation of the selection of the particular key findings:

‘The following is a selection of key findings14 regarding the impacts of climate change on 

systems, sectors and regions, as well as some findings on vulnerability, for the range of 

climate changes projected over the 21st century.’

Footnote 14 specifies the selection criteria:

‘Criteria of choice: magnitude and timing of impact, confidence in the assessment, 

representative coverage of the system, sector and region.’

And in the Summary for Policymakers of the Synthesis Report, the note to Table 

SPM.2 (‘Examples of some projected impacts’) states:

‘The magnitude and timing of impacts that will ultimately be realised will vary with the 

amount and rate of climate change, emissions scenarios, development pathways and 

adaptation.’

As a result of the approach that was followed, mostly negative impacts were 

singled out (see Table SPM.2, reproduced in Annex A of the present report). In fact, 

the reason why hardly any example of a positive impact of climate change can be 

found in Table SPM.2 of the Synthesis Report is that, according to the IPCC authors, 

most of the positive impacts were not of sufficient policy relevance. The Working 

Group II Co-Chair further explained the following:

‘i) most main impacts are negative, because mankind’s activities are generally tuned to 

current climate, not to something different from that; most change therefore implies a cost; 

ii) in net welfare terms, as Working Group II found, aggregate impacts are indeed estimated 

to be negative (see much of Chapter 20 of the Working Group II Report, summarised on 

page 17 of the Summary for Policymakers); iii) autonomous adaptation is better at making 

more of opportunities than protecting against the adverse, and iv) policy is experienced in 

providing protection against the adverse.’ (Parry, personal communication, 20 June 2010)

We labelled this approach to summarising as a ‘risk-oriented’ approach – mainly, 

because it implicitly guides assessors to focus more on risk than on opportunities. 

We believe that this risk-oriented approach has been implicit in the process to 

arrive at conclusions based on the chapters and that by accepting its outcomes 

it has in effect been approved by all the governments. The consequence of this 

approach for which facts did or did not travel to the highest levels of summary, 

can be illustrated by several examples. For instance, the focus in Table SPM.2 of 

the Synthesis Report is on crops for which yields are likely to be reduced in Africa, 

but the table does not mention the crops for which yields are likely to increase due 

to climate change. For Australia and New Zealand, Table SPM.2 shows many risks 

but only one benefit of climate change (initial benefits for agricultural production 
Results and discussion 39

in New Zealand), while Section 11.4 on key future impacts and vulnerabilities 

concludes: ‘climate change is likely to give rise to six key risks in specific sectors. 

[...]. There are also four key benefits for particular sectors.’ Furthermore, in Africa, 

also decreases in the numbers of people suffering from water-stress due to climate 

change are projected for some areas. Of course, these numbers cannot be simply 

subtracted from increases in water stress elsewhere (other people are involved), 

but they constitute potential positive effects of climate change. Finally, an example 

of a selection against mentioning the positive effects of climate change was found 

with respect to the likely positive effect of climate change on forest production in 

North Asia (Russia), which some policymakers may find to be of relevance to their 

decision making. This potentially positive effect did not reach the Summaries for 

Policymakers, partly because the authors attached a ‘medium confidence’ to their 

statement.

A second aspect of the risk-oriented approach that was followed by the Working 

Group II Fourth Assessment Report, is that in summary statements on the impacts 

of climate change, information rightfully was provided on the climate-change

risk. However, this information was often not contextualised in the summary 

statements by also mentioning other impact factors. This was even the case when 

these other factors were much larger than the impact that was attributable to 

climate change. To give an example, Arnell (2004) showed that the number of 

people living in water-stressed watersheds (defined as having less than 1,000 m3

of water per year, per person), even without climate change, would rise strongly 

over time (a rise of 1.5 to 2 billion people by 2025, globally, compared to 1995, see 

his Table 5), mainly due to population growth in already water-stressed areas. This 

increase is much larger than the additional increase related to climate change. 

Again, some policymakers may wish to see both numbers – that is, changes with 

and without climate change – within the same context in a summary.

A third, and final, characteristic of the risk-oriented approach is that, often, the 

upper ends of uncertainty ranges (the worst outcomes that are projected) were 

highlighted. One example comes from the chapter on Australia and New Zealand, 

in which 60% of the Great Barrier Reef was projected to regularly bleach, by 2020. 

This percentage is at the upper end of the uncertainty range. Another example, 

in the Synthesis Report, in Table SPM.2, is the illustration of the risk of species 

loss in the European mountains, by the statement ‘in some areas up to 60% under 

high emissions scenarios by 2080.’ This statement highlights the upper end of the 

uncertainty range (the most sensitive situations and the scenario with the most 

impact), and does not specify the whole uncertainty range; although the words ‘up 

to’ signal that there is a range, only the upper end was specified, causing it to be 

highlighted. 

We believe it is appropriate – and even necessary – to apply a risk-oriented 

approach to climate-change impact assessment. It is a common approach in the 

fields of, for example, public health, public safety, and anti-terrorism. Nevertheless, 

we note that the nature of such an approach, especially for the Summary for 

Policymakers of the Synthesis Report, could easily go unrecognised by readers. 

The legitimacy of the findings could be increased if the reader would be explicitly 

informed about the fact that a particular approach has been employed. However, 
40 Assessing an IPCC assessment 

we feel that a risk-oriented approach is not enough. After all, some policymakers 

may wish to also receive information on the positive regional consequences and 

the full ranges of uncertainty at the highest level of summary. Therefore, a more 

explicit discussion on this issue in the IPCC Plenary may be considered.

3.4 Comments

In this investigation, besides looking for errors, we have evaluated the quality of 

statements on regional climate-change impacts in the IPCC Fourth Assessment 

Report by using seven criteria (see Section 2.2). A full account of our findings is 

given in Part II. In the section below, an overview is given of our main comments, 

grouped along our seven criteria.1)

Attribution (code: C1)

The regional chapters in the Working Group II Report are typically judicious in 

separating out the impacts that they attribute to anthropogenic climate change. In 

the text examined, only one instance was found of an ‘insufficiently substantiated 

attribution’ (major comment), in which a future attribution is questionable and 

more care should have been taken to specify what part of a projected change 

depends on climate change as its driver.

In the Technical Summary, in Table TS.4, the number of additional heat-related 

deaths per year is given for Australia, with the suggestion that all 3,000 to 5,000 

additional heat-related deaths projected for 2050 would be dependent on an 

increase in temperature. However, a large part of this increase is solely due 

to changes in population size and age distribution (see Section 7.2). In future 

assessments, even more care must be taken to prevent insufficiently substantiated 

attributions.

Generalisation (code: C2)

In most cases, the generalisations contained in the regional chapters are wellunderpinned by the chapter material. In the text examined only two instances 

were found where the generalisation is questionable (both major comments). Two 

statements in Table SPM.2 of the Synthesis Report involve ‘insufficiently founded 

generalisations’. First, it is questionable whether freshwater availability will 

decrease in all parts of Asia as indicated (Central, South, East and South-East Asia) 

(see Section 6.1). Second, it is questionable, on the basis of evidence presented in 

the chapter, whether productivity will decrease also for livestock farming other 

than cattle, and, more generally in Latin America, because the evidence provided 

relates only to Bolivia and Argentina (see Section 9.1). In future assessments, more 

care must be taken to prevent insufficiently founded generalisations.

1) Part II also presents five of our comments that do not fit our qualification categories. Among 

these, there is one major comment on the ambiguity of the meaning of phrases such as ‘in the Sahelian region’ (see Section 5.3).
Results and discussion 41

Transparency of expert judgment (code: C3)

One of the main findings of our investigation is that, for six out of the 32 key 

statements on climate-change impacts in different regions (Table SPM.2), the 

regional chapters insufficiently convey the IPCC authors’ reasoning behind their 

weighing of the evidence that was available from the literature (‘insufficiently 

transparent expert judgment’). For other important statements in the regional 

chapters, also, information on the reasoning behind them was lacking. Expert 

judgment will always have to play a crucial role in IPCC assessments, or in fact in any 

assessment. But in those cases where readers may wish to ascertain the reliability 

of a statement, having the authors’ reasoning explained in the text would be of 

great help in improving the credibility of the claims made.

Below, by way of example, is a list of six questions pertaining to statements in Table 

SPM.2, which we were unable to find the answer to, on the basis of the regional 

chapters and the references contained therein:

1. Why is a projected decline in cereal yields in dry years in Morocco, by 2020, 

representative of an overall decline in yields in ‘rain-fed agriculture’ in ‘some 

countries’ in Africa? (see Section 5.1) (major comment)

2. With respect to the cost of adaptation to the projected sea level rise in Africa 

towards the end of the 21st century, we asked: how was the range of 0 to 156% 

of GDP for different countries in the referenced study translated to a 5 to 10% 

range for the whole continent? (see Section 5.1) (minor comment)

3. In which way is the information about changes in ‘freshwater availability’ in Asia 

(which in the chapter is given in both absolute flows and per-capita availability) 

combined to arrive at the assessment that freshwater availability is projected to 

decrease in many parts of Asia by the 2050s? (see Section 6.1) (major comment)

4. The report provides information on the range of temperature change for which 

a regular bleaching of 60% of the Great Barrier Reef is expected; how was this 

temperature range converted into the statement that this level of bleaching 

would be reached by the year 2020? (see Section 7.1) (minor comment)

5. What is the underpinning for the projected loss of up to 60% in plant and animal 

species, in some European mountainous areas, under high emissions scenarios 

by 2080 – given that the references named in the chapter only state this 

percentage in relation to plant species? (see Section 8.1) (minor comment)

6. How was the large range (from negative to positive) of projected changes in 

yields of rain-fed crops (varying over crops and locations) for North America 

translated to a 5 to 20% increase in ‘aggregate yields’? (see Section 10.1) (minor 

comment)

Relying on expert judgment is a necessary practice in situations where wellresearched findings are scarce and potential risks are great. In such situations, 

the joint deliberation of experts may simply be the best available way to make an 

assessment. Yet, the transparency of the reasoning in such a deliberation would 

enhance the verifiability and hence the potential credibility of the assessments that 

experts arrived at. This investigation ex post reconstructed some of the reasoning. 

Through communication with the IPCC authors, a large part of the answers was 

received (these answers have been included in Part II), which led to an increase 

in our confidence about the statements. Although, for a significant part of the 

cases, the IPCC authors, when asked, were able to reconstruct their reasonings 
42 Assessing an IPCC assessment 

and convey them to us, we are convinced that, especially for statements that are 

included in the Summaries for Policymakers, it is necessary in future assessments to 

have such reasoning available in the underlying material (chapter texts). 

Consistency of messages (code: C4)

Our investigation revealed only two instances of inconsistencies (minor comments) 

within the report. This concerned inconsistencies in the likelihood level of increases 

in fires in Australia and New Zealand (‘likely’ according to the main text of Chapter 

11 and ‘virtually certain’ according to the Executive Summary of the Chapter) and 

inconsistencies in the confidence level about the relevance of past experience 

with extreme events to policy-making on adaptation in Europe (‘high confidence’ 

in the Executive Summary of Chapter 12 and ‘very high confidence’ in the Technical 

Summary and Summary for Policymakers). Such minor flaws could have been 

prevented by even more careful cross-checking and editing than was done for the 

Working Group II Fourth Assessment Report.

Another issue related to consistency of messages is the general lack of specification 

in the summaries about baseline years or periods related to the statements. Part 

II does not address this issue, since it deals with individual statements; here it is 

flagged as a more general issue. Variable baselines (20th century; 1961-1990; 1990; 

1995; ‘today’) are used in the regional chapters of the Working Group II Report (as 

well as in the underlying references), but many statements in the summaries are 

vague about which particular baseline is being assumed for the projected ‘changes’. 

For qualitative accounts this is less important, but baselines need to be crystal clear 

when numbers are involved, since these are sensitive to the choice of baseline. 

Ideally, baselines should be the same throughout a report, but this is not always 

possible.

Traceability of references (code: C5)

Of the references that we checked, only very few turned out to be untraceable. 

For instance, the projected ‘up to 50%’ reduction in yields in rain-fed agriculture 

for some countries in Africa, was ultimately based on an untraceable reference 

(MATUHE, 2001). Given the high importance attached to the statement, this is 

a major comment. On several other examples we had only minor comment to 

make. A hyperlink turned out to be outdated; it probably has been functioning in 

2006/2007, but it was not in 2010 (FAO, 2002, see Section 9.2). Yet another example 

concerns a paper on a projected increase in coffee-leaf miners in Brazil, which was 

listed in the references of Chapter 13 as ‘submitted’ to Climatic Change, but we 

found that the paper had never been accepted by this particular journal, which 

makes this reference untraceable. Although we found a paper with the same title 

that was published in 2008, in Pesquisa Agropecuária Brasileira, Brasília, the paper 

had not been accepted by any journal at the time the IPCC report was published. It 

would be our advice that papers may only be used as literature reference in future 

IPCC reports if they have been accepted for publication prior to the publication date 

of the IPCC assessment report.

Credibility of references (code: C6)

Only two instances were found of statements relying on references to grey 

literature, when literature published in peer-reviewed journals was also available to 
Results and discussion 43

which referencing would have been more appropriate. For example, the statement 

that up to 40% of the Amazon Rainforest could react drastically to even a slight 

reduction in precipitation was underpinned by a reference to a peer-reviewed 

WWF/IUCN report of 2000 on forest fires, while – in this case – also more relevant 

high profile peer-reviewed journal articles had been published by Cox et al. (2000; 

2004). In our opinion, this issue deserves more attention from IPCC authors in the 

future. We consider it a minor comment.

Availability of statements for review (code: C7)

In one instance, we found that a new, quantitative conclusion was added after the 

Second Order Draft had been completed, without there being sufficient grounds 

from review comments for such an addition. This statement, on the ‘up to 50%’ 

reduction in yields from rain-fed agriculture for some countries in Africa, thus, had 

been unavailable for review, while it was allowed to travel from an untraceable 

source up to the highest level of summary (Table SPM.2 of the Synthesis Report). 

This is a major comment. It is our conviction that such high-impact information 

should have been made available for review, and that this is applied in future IPCC 

reports.

3.5 Conclusion

No significant errors found in summary conclusions

The foundations for thirty-two IPCC Fourth Assessment summary conclusions 

on the regional impacts of climate change have been investigated. These 

conclusions show examples of projections of climate-change impacts on food, 

water, ecosystems, coastal regions and health, for all the earth’s continents. These 

conclusions have not been undermined by errors, although one of the conclusions 

contains a minor inaccuracy: in hindsight, not 75 to 250 million people, but 90 to 220 

million people are projected to be exposed to increased water stress due to climate 

change in Africa, by 2020.

The provenance of summary statements needs to become more transparent in 

future reports

Seven of the investigated 32 conclusions on the regional impacts of climate change 

contain information that we were unable to sufficiently trace to the underlying 

chapters in the IPCC Working Group II Report or to the references therein. For 

two of these conclusions, our critical comments pertain to insufficiently founded 

generalisations from existing scientific research, in both cases from local to 

regional scales, and for one of them also from one type of livestock to livestock in 

general. While acknowledging the essential role of expert judgment in scientific 

assessments, the PBL recommends to improve the transparency of these 

judgments in future IPCC reports.

The regional chapters: one error and some comments

In the underlying regional chapters, in addition to the two already known errors 

about the melting of the Himalayan glaciers and about the Dutch land area 

below sea level, another significant error was found: a 50 to 60% decrease in 

productivity in anchovy fisheries on the African west coast was projected on the 
44 Assessing an IPCC assessment 

basis of an erroneous interpretation of the literature references. We found certain 

inaccuracies, ranging from (very) small errors in numbers to imprecise literature 

references. In addition, we have some critical comments to make. One of them 

relates to the fact that the report does not specify how many of the additional heatrelated deaths projected for Australian cities are actually attributable to climate 

change – a sizeable fraction is due to demographic changes alone. However, these 

shortcomings do not affect the investigated 32 summary conclusions or other parts 

of the IPCC summaries.

Examples of negative impacts dominate at summary level

The IPCC Working Group II Report focuses on climate impacts, adaptation and 

vulnerability. It was found that, in the IPCC’s highest level summary, the conclusions 

that were derived from the regional chapters of the Working Group II contribution 

to the Fourth Assessment Report single out projections of negative climate-change 

impacts. Our investigation revealed that IPCC authors considered these to be 

the most relevant to policymakers. The PBL has labelled this as a ‘risk-oriented’ 

approach, which had been implicitly endorsed by the governments that constitute 

the IPCC (including that of the Netherlands). The PBL subscribes to the importance 

of an approach that highlights what may go wrong under unmitigated climate 

change, but the Fourth Assessment Report lacked a clear explanation of the 

choice of approach and its consequences. Alternatively, it could be argued that 

policymakers should be presented with a complete picture in the Summaries for 

Policymakers, not just with negative examples (without suggesting that potential 

positive effects cancel out potential negatives effects).

No consequences for overarching conclusions

Our findings do not contradict the main conclusions of the IPCC on impacts, 

adaptation and vulnerability related to climate change. There is ample 

observational evidence of natural systems being influenced by climate change on 

regional levels. And the negative impacts under unmitigated climate change in the 

future pose substantial risks to most parts of the world, with risks increasing at 

higher global average temperatures. 
Recommendations 45

Recommendations

Based on our findings and conclusions in the previous chapters, we present 

recommendations for the Fifth Assessment Report process. Again, there will be 

four assessment reports (by Working Groups I, II, and III and a Synthesis Report), 

due in 2013 and 2014. The report by Working Group II will consist of two separate 

volumes; a sectoral part A and a regional part B (IPCC, 2009). Our recommendations 

primarily apply to part B of the Fifth Assessment Report by Working Group 

II, although most of our recommendations may also apply to the entire Fifth 

Assessment Report. 

4.1 Minimising the risk of errors

In comprehensive scientific assessments of thousands of pages that refer to a 

large corpus of scientific work, errors are inevitable, especially when dealing 

with quantitative information. The current IPCC review process is meant to be 

instrumental in removing errors that creep into the texts during the writing of the 

several drafts. If errors do occur in the main texts, they rarely propagate to the level 

of the Summary for Policymakers, since this relatively short text of approximately 15 

pages is reviewed and approved line by line, word by word by the IPCC Plenary. In 

our investigation, we found only one minor error in the investigated 32 statements. 

However, the full range of IPCC reports is supposed to be authoritative, also in the 

underlying texts. Therefore, as much as possible, errors need to be detected and 

removed. With regard to finding and correcting errors, we believe that an open, 

pro-active and transparent policy, involving wide circles of the assessment report 

users, would increase the reliability of IPCC, given its recent increased visibility and 

public exposure. It is a complex and highly dynamic field. Small mistakes belong to 

this process and should not be seen to jeopardise the overall credibility of science. 

However, concerns about statements that are made in such elaborate assessments 

are inevitable. By enabling constant updating of documents and by publishing 

‘errata’ as an integral part of the process, the IPCC may strengthen the societal 

legitimacy of its work.

In light of the above, we recommend:

 Create a public registration website for the submission of possible errors found in the 

published report. Errors are inevitable, therefore, the IPCC should make provisions 

to deal with these, immediately following publication of an IPCC report. Since 

a few months, erratum pages are available on the IPCC website for the Fourth 

Assessment Reports of 2007. For the Fifth Assessment Reports to be published

4
46 Assessing an IPCC assessment 

in 2013 and 2014, such erratum pages should be made available directly after 

publication. In addition, we suggest the launch of a public registration website 

for the reporting of possible errors. Subsequently, these errors (e.g. regarding 

numbers or typographic errors), when found to be genuine, could be quickly corrected by publishing them on the erratum website. For more substantial errors, 

additional comments and explanations from the IPCC should be added and also 

posted on the website. From our own recent experience, we believe that such 

a registration website, if designed properly, would be manageable. Experience 

with this approach may be gained by setting up experimental registration websites for the two upcoming Special Reports in 2011.

 Check and double-check references. More checks should be done with regard 

to accurate referencing to the scientific literature, which should always be 

traceable and available for reviewers. With regard to the traceability, a userfriendly option would be to create hyperlinks within the text leading to the 

original publications.

 Investigate possibilities of pre-publication ‘crowd sourcing’. Finding errors, notably 

in numbers, is a time-consuming effort for authors, reviewers, Co-Chairs and 

Technical Support Units, who already are limited in terms of time and resources. 

The possibility of using ‘crowd sourcing’ technologies as part of the review 

during the last stages of completion, therefore, should be investigated. Crowd 

sourcing has the advantage of providing a large reservoir of voluntary resources 

to be tapped. A disadvantage could be the amount of resources needed to 

manage and moderate tasks that would be handed out to the public.

4.2 Investing in the improvement and transparency 

of foundations of summary conclusions

The findings in Section 3.4, and the more detailed analysis in Part II of this report, 

lead us to the following recommendations:

 Make sure that statements that attribute impacts to climate change are well 

founded in scientific research, including systematic observations, modelling 

and statistics. The climate-change component of impacts should be carefully 

characterised.

 Make sure that expert judgments that are incorporated in summaries are made 

transparent and plausible by explaining the line of reasoning behind them in the 

main text. This applies especially to: 

– case-study generalisations to whole sectors, areas or regions; such generalisations should be well supported by scientific evidence; 

– choosing certain intervals of specific numbers from a wide range of figures. 

This would be a great help to any institute that is asked to assess the plausibility 

and reliability of statements, and could enhance the credibility of the assessment 

process. Authors are faced with the dilemma that strict limitations on page 

lengths are imposed by the Working Group management, limiting the amount of 

information that can be included.

 Check and double-check consistency of messages and confidence intervals/

likelihood levels between chapters and different summaries. 

 Grey literature should only be used when peer-reviewed scientific journal 

literature on a certain subject is not available. 
Recommendations 47

4.3 Strengthening the quality control by the chapter teams

IPCC authors work on a voluntary basis, or their work is a part of their regular jobs 

as scientists and, often, it is an addition to their regular workload. The budget for 

the IPCC organisation is relatively small, and assessment time lines are kept very 

strict. The high cost-effectiveness is both a strength and a weakness in the IPCC 

process. 

The amount of scientific literature on climate change is skyrocketing, policy 

implications become more visible and, recently, the credibility of the IPCC has come 

under scrutiny by the media. This implies a need for higher quality standards, next 

to a future increase in the amount of assessment work. Moreover, Coordinating 

Lead Authors have to manage their writing teams, and deal with large flows of 

information within a limited amount of time. Therefore, we recommend that a 

procedure is set up – within the shortest possible time frame, and which will be 

implemented at the very beginning of the writing process – for appointing and 

employing chapter assistants who will help the Coordinating Lead Authors with 

quality-control issues, such as the checking of references (including basic checks 

on quantitative information), the cross-checking between findings on regional and 

sectoral levels, the cross-checking between statements concerning regional climate 

projections in the Working Group II Report and those in the report by Working 

Group I. In order to make all this possible, a serious increase in the contributions 

from the governments that are involved in the IPCC would be necessary. 

4.4 Strengthening the review process

We recommend that certain improvements are made in the review process, in 

order to further minimise the risk of errors and to improve the foundation of the 

summary conclusions: 

 Ensure that all texts are fully covered by expert reviewers. The current review 

process is generally rather passive — expert reviewers are free to comment 

on those parts of the text that they are interested in, but there is no guarantee 

that qualified expert reviewers will scrutinise all parts of all texts including the 

references. A pro-active targeted expert review is therefore recommended.

The IPCC could establish a web-based nomination procedure, in which experts 

subscribe to reviewing certain parts of a report. The Co-Chairs and Technical 

Support Units could then select the experts and see to it that all parts of the 

texts are covered by at least a few qualified reviewers.

 Do not allow new material to be introduced after the last review if this material is not 

related to the review. After the Second Order Draft review, new material should 

only be added if it is clearly in response to, or related to substantive issues raised 

in review comments. Otherwise, newly introduced material would be excluded 

from expert review. The IPCC procedures should stipulate more clearly how new 

material should be dealt with.

 Do not allow referencing to scientific articles that have been submitted but not (yet) 

accepted for publication by the time the IPCC assessment report is published. This 

has already been good practice in many of the IPCC reports, but IPCC procedures 

should provide some ruling on this issue. 
48 Assessing an IPCC assessment 

 Review by governments of the foundation and provenance of statements in Summaries 

for Policymakers. Governments have an important responsibility in the review 

process and, therefore, should seriously invest in the government/expert 

review of Second Order Drafts. Apart from the reviewing of issues of specific 

national interest, governments, assisted by independent experts, could review 

provenance and foundations for statements in the draft version of the Summary 

for Policymakers. This report provides an example of how this could be done 

methodologically. Since this would be a time-consuming effort that draws from 

many resources, the IPCC could consider extending the expert/government 

review period for the Fifth Assessment Report. Governments could work 

together in this process and share resources. In addition, this would provide 

them with a solid frame of reference for the line-by-line approval session of the 

final Summary for Policymakers.

4.5  Timing of the assessments

The quality of regional chapters could be further improved by making better use of 

information that is available in the sectoral chapters of Working Group II reports. 

The best way to achieve this would be to finalise regional chapters not until sectoral 

chapters have become final. This would help to integrate information at regional 

levels and contribute to the transparency of the expert judgments. In particular, 

this can help to provide stronger underpinning of generalisations of case studies to 

entire regions or sectors, by also making use of regional modelling studies.

4.6  Balancing the assessment of climate-change impacts

The section on ‘risk-oriented approach’ (3.3) reveals that, at summary level, the 

most important negative impact projections have been highlighted. This approach 

is understandable and justifiable, but it has not been made explicit. We believe 

that such a risk-oriented approach, although essential, is also one-sided. It could be 

argued that the Summary for Policymakers should provide policymakers with a full 

picture; it should not only present the most important negative examples (without 

suggesting that potential positive effects cancel out potential negative effects). We 

recommend that the Fifth Assessment Report explains which approach has been 

taken. 

In addition, we suggest that a Summary for Policymakers includes two separate 

sections dealing with projected regional impacts on water, food, ecosystems, 

coastal regions and health, divided as follows:

 One section that describes robust outcomes, with the full range of projections 

of climate-change impacts, including not only the uncertainties, but also any 

positive impacts, containing:

– A description of uncertainties, as has been employed by Working Group III in 

their report in the Fourth Assessment. Per statement, an indication could be 

given of the amount of evidence (number and quality of independent sources) 

and level of agreement (on a particular finding). At summary level, the first 

choice would be to highlight statements that are based on a large number of 
Recommendations 49

publications all converging to the same conclusions. ‘Likelihood’ statements 

could only be made if sufficient scientific evidence would be available, and 

‘confidence’ statements would be expert judgments, which are subjective by 

nature and, normally, are untraceable for the reader;

– An indication of the relative importance of other drivers that affect food, 

water, ecosystems, coastal areas, and health, compared to drivers of climate 

change, such as population growth, industrialisation and land use. The impacts 

that can be attributed to climate change should be separated, as clearly as 

possible, from those that are caused by other factors.

 One section that explicitly describes the most important negative impacts, as 

was done for the Fourth Assessment Report, but this time including a worstcase risk approach, based on clearly explicated risk assessment methodology. 

This section could highlight future impacts that may be unlikely to occur, 

but could have major consequences, and therefore would pose significant 

risks. Such a section could also include more speculative projections of high 

climate sensitivity, accelerated warming, and tipping points in ecosystems. An 

exploration of such an approach has been presented in the PBL study ‘News in 

Climate Science and Exploring the Boundaries’ (December, 2009).

Reasons to consider this line of action include:

 The outsiders of the scientific community that study impacts, adaptation and 

vulnerability, will not automatically understand the ‘risk-oriented approach’, 

as we have labelled the Working Group II approach that leads to a sample of 

the most important negative impacts. Working Group I followed a different 

approach, establishing the full range of possible projections including its 

uncertainties. Without proper explanation, the results at the summary level of 

Working Group II could easily be interpreted as being an alarmist view (as was 

our interpretation at the outset of our analysis, before we received further 

explanation from the Working Group II Co-Chair and authors).

 We believe that policymakers (and their analysts) need to see the complete 

picture at summary level. Positive impacts are important, because they may also 

become smaller when mitigation measures are introduced. There may be offsets 

possible between positive and negative impacts of climate change, for instance, 

in relation to agricultural yields. Although there are justifiable objections against 

making a cost-benefit analysis of positive and negative impacts, to not mention 

them would be ‘policy-pre-emptive’. 

 In order to obtain a complete picture, we also believe that policymakers – in 

both public and private sectors – would need to be informed about the worstcase risks, when they are developing risk management, disaster prevention and 

insurance policies. 

 Finally, some of the comments made and the errors found – including those 

received through our registration website – were related to the issue of 

accentuating or even heightening the severity of climate-change impacts. We 

recommend that the authors of the Fifth Assessment Report (AR5) be made 

specifically aware of this issue. Care should be taken with phrasing of statements 

that could be perceived by readers as heightening the projected impacts of 

climate change.
50 Assessing an IPCC assessment 

4.7 Investing in climate-change impact science

Climate-change impact science is relatively young, although the amount of 

literature is expanding rapidly. Figure 1.8 in the Working Group II Report and Figure 

SPM.1 in the Synthesis Report (see Figure 1.1 in this report) show that there are 

many observational data available on Europe and the United States that indicate 

physical and biological changes, over time, but there is only a small amount of 

observational data available on such changes in developing countries, while 

developing countries are more vulnerable to climate change. In addition, model 

projections become more uncertain at geographically more detailed levels, while 

policymakers ask for concrete information on their specific region or location. 

This calls for intensifying investments in regional climate impact observation 

systems, regional modelling and capacity building, particularly in developing 

countries

References

IPCC, 2009. Chapter outline of the Working Group II contribution to the IPCC Fifth Assessment report 

(AR5), http://www.ipcc.ch/meetings/session31/doc20-rev1.pdf, 

PBL, 2009. News in Climate Science and Exploring the Boundaries. http://www.rivm.nl/bibliotheek/

rapporten/500114013A.pdf
Part II – Detailed analysis of regional chapters and summaries 51

Part II – Detailed analysis of 

regional chapters and 

summaries

This Part presents:

 our findings referring to the foundation for the 32 summary conclusions on regional impacts in the Synthesis Report (see Annex A) – below, these conclusions 

have been typeset as headers, followed by our findings;

 additional findings at lower levels, such as those referring to the foundations for 

statements on regional impacts in the Technical Summary and Executive Summaries of the regional chapters, and also some findings on parts of the main text 

that we came across while working on other findings – below, statements have 

been typeset as headers, followed by our findings.

For our investigation, we used several criteria for assessing the quality of IPCC 

statements. First of all, we employed a distinction between obvious factual ‘errors’ 

– which we believe require an erratum on the IPCC Fourth Assessment web pages 

– and ‘comments’, the latter being critical remarks, made from our specific position 

of having to assess these statements on behalf of Dutch policymakers. These criteria were developed ‘on the job’, while categorising our findings. In total, there are 

nine criteria, and associated with these are two types of ‘errors’ (E1-E2) and seven 

types of ‘comments’ (C1-C7):

 E1: Inaccuracy of a statement; 

 E2: Inaccuracy of literature referencing;

 C1: Insufficiently substantiated attribution, when a certain impact had been attributed to climate change without convincing foundation;

 C2: Insufficiently founded generalisation, when case studies had been extrapolated 

to whole regions/sectors without convincing foundation;

 C3: Insufficiently transparent expert judgment, when we could not trace the reasoning behind a statement from underlying texts or literature references given;

 C4: Inconsistency of messages, when found between the different layers in the 

IPCC report;

 C5: Untraceable reference, when a literature reference could not be found;

 C6: Unnecessary reliance on grey referencing, when reference had been made 

to grey literature (literature other than from peer-reviewed journals), while 

adequate peer-reviewed scientific journal references would have been available;
52 Assessing an IPCC assessment 

 C7: Statement unavailable for review, when new information had been introduced 

after the last review but was not clearly derived from a content issue raised in 

this last review.

When a certain statement in the IPCC Fourth Assessment Report would meet all the 

criteria, the statement was considered to be well founded and reliable. 
Africa 53

Africa 

5.1 Analysis of statements in Table SPM.2 of the Synthesis 

Report

By 2020, between 75 and 250 million of people are projected to be exposed to 

increased water stress due to climate change. (E1; minor)

This statement was based on Table 11 in a study by Arnell (2004), which contains 

results on increased water stress by 2025 (instead of 2020) as computed by 

several climate models for five sub-regions in Africa (and other continents). In 

our communication with Nigel Arnell, concerning the addition of numbers in the 

table (which gives a range of between 75 and 240 million people, consistent with 

the rounded numbers in the statement), he informed us that he had recently 

noticed that, in this case, the addition of ranges was an unreliable method for 

determining the total range. For the HADCM3 model the table contains a range of 

model outcomes from three model runs of the SRES A2 scenario and a range from 

two model runs of the B2 scenario, but the low and high values did not originate 

from the same model run. Simply adding up these extreme values from the model 

outcomes for the sub-regions may thus overstate the range. It must be added here 

that, although this type of error is well known to occur in scientific practice, the 

IPCC authors, including Nigel Arnell who was a Lead Author of a sectoral chapter 

(Chapter 3) of the Working Group II Report, considered the straightforward 

addition of those extreme values appropriate at the time. With the benefit of 

hindsight, a relatively simple reanalysis of the data would have brought this error 

to light. According to Arnell (2010), the correct range that results from adding 

the right numbers for Africa would be: 88 to 217 million people. This implies that 

‘between 75 and 250 million’ should have read ‘between 90 and 220 million’ when 

rounded to the nearest 10 (E1b). However, since the difference is only small and 

uncertainties are considerable, and because the original paper did not provide the 

details that would have allowed correct computing of this range, we consider this 

to be a minor inaccuracy.

By 2020, in some countries, yields from rain-fed agriculture could be reduced by up 

to 50%. (C3,C5,C7; major)

This statement is not directly a statement on climate change, but on climate 

variability: in individual years, droughts can cause up to 50% in yield reductions. 

The implicit message here is that when droughts would become more frequent 

due to climate change, more years with up to 50% in yield reductions would occur. 

The statement could easily mislead readers into thinking that average annual 

yields could be reduced by up to 50% due to climate change. In the Summary for 

Policymakers of the Working Group II Report, the paragraph that contains this 

5
54 Assessing an IPCC assessment 

statement starts with a sentence introducing the notion of climate variability, which 

puts the statement more into context. We have several major comments to make 

about its provenance. The statement was based on a report by the International 

Institute for Sustainable Development (IISD) (Agoumi, 2003) that referred to 

studies performed within the framework of a UNEP–GEF project that we were 

unable to trace (C5) and to the Initial National Communications (INCs) of Morocco, 

Algeria and Tunisia. However, only the INC of Morocco (Kingdom of Morocco, 

2001) reported a decline in yields for 2020: ‘a 50% reduction in cereal yields in 

Morocco might occur in dry years and 10% in normal years’. The IPCC authors 

explained to us that present-day climates and projected future climate change in 

the three countries are very similar – with a reference to Figure 11.2 in the Working 

Group I Report of AR4 – and also that only cereals are grown without irrigation. 

Furthermore, using information from the European statistical information service 

(EUROSTAT) the authors made plausible that, due to current climate variability, the 

yields in Algeria, Morocco and Tunisia have been varying annually, including yield 

reductions of nearly 70% in individual years, in the period between 2000 and 2006. 

In hindsight, these additional explanations could have provided further foundations 

for the statement, had they been included in Chapter 9 (C3). 

The Initial National Communication of Morocco indicated that this statement was 

based on ‘The Study of Morocco’s Vulnerability to CC Effects’ (MATUHE, 2001). 

However, we were unable to trace this study, nor could it be provided by the IPCC 

authors (C5). Although the IPCC does not formally require that authors check 

references of references of references, it would have been very helpful in this case; 

the IPCC requires that the chapter team reviews the quality and validity of nonpeer-reviewed sources (C5). As it stands, the statement cannot be traced back to 

any original scientific research.

Finally, the statement seemed to have been added to the main text of Chapter 9 

after the Second Order Draft had been reviewed, but we were unable to find the 

particular substantive comment(s) made in this review that could have led to the 

addition of this statement (C7).

Agricultural production, including access to food, in many African countries is 

projected to be severely compromised. This would further adversely affect food 

security and exacerbate malnutrition.

This statement – which together with the previous statement comprises the second 

bulleted statement on Africa in Table SPM.2 (see Annex A) – is fully supported by 

the underlying material.

Towards the end of the 21st century, projected sea level rise will affect low-lying 

coastal areas with large populations. The cost of adaptation could amount to at least 

5 to 10% of Gross Domestic Product (GDP). (C3; minor)

The first sentence of this statement is fully supported by the underlying material. 

We have a comment to make about the second sentence of the statement, which 

was based on a table in a study by Niang-Diop (2005). The text does not explicate 

how the experts derived the 5 to 10% in adaptation costs from this study, since 

it covered a range from 0.5 to 156.2% (C3). The IPCC authors explained to us that 

they used 5 of 12 studies summarised in the table in Niang-Diop (2005). Those that 
Africa 55

scored below the range of 5 to 10% were either countries that did an assessment 

not covering the entire country (Cameroon: 0.5%; Gambia: 1%), or those with a very 

short coastline (Democratic Republic of the Congo: 4.1%), or relatively rich countries 

(Nigeria: 2.2%). Those that scored well above this range were either very small island 

states (Comoros: 49.7%) or countries for which the costs had been overestimated 

(Mauritania: 156.2%). This answer explains the 5 to 10% range. We believe it would 

have been helpful if this explanation had been provided in Chapter 9.

By 2080, an increase of 5 to 8% of arid and semi-arid land in Africa is projected 

under a range of climate scenarios.

This statement is fully supported by the underlying material.

5.2 Additional findings

The population at risk of increased water stress in Africa is projected to be between 

75–250 million and 350–600 million people by the 2020s and 2050s, respectively 

(E1; minor).

This statement is part of the Executive Summary and is also included in Table 

TS.4 on page 67 of the Technical Summary. The range of 75 to 250 million people 

is discussed above in Section 5.1 (first statement). From Nigel Arnell, the author 

of the underlying 2004 study, we learned that also the second range (350 to 600 

million people in the 2050s) was incorrectly computed from Table 11 in his study; 

the extreme values for the sub-regions cannot be added together. This remained 

unnoticed during the writing of the Working Group II Report. According to Arnell, 

in hindsight, the correct range should have been 350 to 490 million (E1b). Given the 

large uncertainties, we consider this a minor error. This inaccuracy has no impact on 

the IPCC conclusions in the various Summaries for Policymakers.

In South Africa, fisheries could be affected [...]. Recent simulations based on the 

NCAR GCM under a doubling of carbon dioxide indicate that extreme wind and 

turbulence could decrease productivity by 50-60%, while turbulence will probably 

bring about a 10% decline in productivity in the spawning grounds and an increase 

of 3% in the main feeding grounds. (E1; major) 

This statement originates from Section 9.4.4 of the main text, and was based on 

a study by Clark et al. (2003), which indicated that the changes in the frequency of 

extreme wind and turbulence events is expected to decrease quite substantially, by 

50 to 60%; it did not refer to a similar decrease in fishery productivity (E1a). Also, in 

the latter part of the sentence it should have been the changes in mean turbulence

that decline by 10% and increase by 3%, and not the productivity. The IPCC authors 

initially indicated that the reference to ‘productivity’ should have been deleted and 

‘for anchovy’ should have been added at the end. In that case, the statement would 

read as follows:

Recent simulations based on the NCAR GCM under a doubling of carbon dioxide indicate 

that the frequency of extreme wind and turbulence events could decrease by 50 to 60%, 

while mean turbulence will probably decline by 10% in the spawning grounds and increase 

by 3% in the main feeding grounds for anchovy.
56 Assessing an IPCC assessment 

From the IPCC authors we received the following proposal for an erratum: 

Recent simulations based on the NCAR GCM under a doubling of carbon dioxide indicate 

that changes in extreme wind and turbulence could decrease potential spawning and 

increase potential larval feeding of anchovy.

This major error has no impact on the IPCC conclusions in the various Summaries for 

Policymakers.

5.3 Findings from the PBL registration website

Agricultural intensification and/or expansion into marginal lands can trigger 

additional conflicts, cause crop failure, exacerbate environmental degradation (e.g., 

Olsson et al., 2005). (minor)

This statement can be found on page 441, Section 9.2.2.4, of the main text. One of 

the reactions we received on the registration website indicated that the reference 

to Olsson et al. (2005) in relation to this particular statement is not the most 

relevant one, because the article does not focus primarily on the intensification 

and/or expansion of agriculture and its consequences. Olsson et al. (2005) is about 

changes in precipitation patterns that have resulted in the greening of the Sahel 

over the past 10 years (5 years at the time of writing). We believe that other, more 

appropriate references would have been available for this statement, but we have 

not assigned this less appropriate referencing to one of our qualification categories. 

It has no impact on the IPCC conclusions in the various Summaries for Policymakers.

In the Sahelian region of Africa, warmer and drier conditions have led to a reduced 

length of growing season with detrimental effects on crops. (major)

This statement was taken from page 9 of the Summary for Policymakers of the 

Working Group II Report. One of the reactions we received indicated that this 

summary statement did not adequately convey the nuances contained in Chapter 

9.1) In response to that reaction, we wish to make a major comment on this 

statement. It was based on two sentences in Section 1.3.6.1 of the main text: ‘In 

Sahelian countries, increasing temperature in combination with rainfall reduction has led 

to a reduced length of vegetative period, no longer allowing present varieties to complete 

their cycle (Ben Mohamed et al., 2002).’ and: ‘In the case of the Sahel region of Africa, 

warmer and drier conditions have served as a catalyst for a number of other factors that 

have accelerated a decline in groundnut production (Van Duivenbooden et al., 2002).’

Even though the statement does not directly refer to the regional chapters, it is a 

statement in the Summary for Policymakers on a regional impact. 

Although, the Summary for Policymakers’ text is consistent with the formulations 

in the main text, we believe that the subject of both case studies, which deal with 

millet, groundnut and cowpeas in Niger, cannot be generalised to describe ‘the 

Sahelian region’ and to ‘effects on crops’. In reaction to this, the IPCC authors replied 

that ‘the Sahelian region’ is not the same as ‘the entire Sahelian region’ and ‘effects on 

1) This reaction and the previous one were received as one combined reaction; in our report we treat 

them separately, since they pertain to different statements.
Africa 57

crops’ is not the same as ‘effects on all crops’. In addition, the authors have pointed 

to the heading of this section (p. 9, Working Group II Summary for Policymakers) 

that says these are ‘examples’ and ‘have not yet become established trends’. The 

authors further commented that these are obviously not universal statements. In 

their opinion, it would be unnecessary for every statement on certain parts of a 

place or region to say ‘parts of’, and similarly about statements on systems to say 

‘some of’, because in their view this non-universality is implicit. 

Indeed, the disclaimer in the header does explain that these are examples and not 

established trends, but to us it is not obvious that the statement therefore refers 

to some crops in a part of, for instance, the Sahelian region, and we believe it would 

have been better if these specifications had been added. This major comment 

has no consequences for the IPCC conclusions in the various Summaries for 

Policymakers.

References

Agoumi, A., 2003. Vulnerability of North African countries to climatic changes: adaptation and 

implementation strategies for climatic change. Developing Perspectives on Climate Change: 

Issues and Analysis from Developing Countries and Countries with Economies in Transition. IISD/

Climate Change Knowledge Network, 14 pp. http://www.cckn.net//pdf/north_africa.pdf

Arnell, N.W., 2004. Climate change and global water resources: SRES emissions and socio-economic 

scenarios. Global Environ. Change, 14, 31-52. 

Arnell, N.W., 2010. Personal communication.

Ben Mohamed A., Duivenbooden, N. van, Abdoussalam, S., 2002, Impact of climate change on 

agricultural production in the Sahel- part I. methodological approach and case study for millet in 

Niger, Climatic Change, 54, pg 327-348.

Clark, B.M., S. Young and A. Richardson, 2003. Likely effects of global climate change on the purse 

seine fishery for Cape anchovy Engraulis capensis off the west coast of Southern Africa (SE 

Atlantic). http://swfsc.noaa.gov/uploadedFiles/Education/lasker_events.doc.

Van Drunen, M.A., R. Lasage and C. Dorland, Eds., 2005. Climate Change in Developing Countries: An 

Overview of Study Results from the Netherlands Climate Change Studies Assistance Programme. 

CABI Publishing,Amsterdam, 320 pp.

Duivenbooden, N. van, Abdoussalam, S., Ben Mohamed A., 2002. Impact of climate change on 

agricultural production in the Sahel- part II> Cae study for groundnut and cowpea in Niger, 

Climatic Change, 54, pg 349-368.

Kingdom of Morocco, 2001. Initial National Communication on the United Nations Framework 

Convention on Climate Change, Ministry of Land-use Planning, Housing, and the Environment, 

October. http://unfccc.int/resource/docs/natc/mornc1e.pdf

MATUHE, 2001. Etude de Vulnérabilité du Royaume du Maroc face aux impacts des changements 

climatiques. Ministère chargé de l’Aménagement du Territoire, de l’Urbanisme, del’Habitat et de 

l’Environnement, septembre 2001.

Niang-Diop, I., 2005. Impacts of climate change on the coastal zones of Africa. Coastal Zones in 

Sub-Saharan Africa: A Scientific Review of the Priority Issues Influencing Sustainability and 

Vulnerability in Coastal Communities, IOC, Ed., IOCWorkshop Report No. 186. ICAM Dossier No. 4, 

27-33.

Olsson, L., Eklundh, L., Ardö, J., 2005. A recent greening of the Sahel—trends, patterns and potential 

causes. Journal of Arid Environments, 63, pg 556–566. 

Otter, L., Olago, D.O. and Niang, I. (eds), 2007. Global Change Processes and Impacts in Africa: A 

Synthesis. START/East African Educational Publishers, Nairobi. 346 pages. September 2007.
58 Assessing an IPCC assessment 
Asia 59

Asia 

6.1 Analysis of statements in Table SPM.2 of the Synthesis Report

By the 2050s, freshwater availability in Central, South, East and South-East Asia, 

particularly in large river basins, is projected to decrease. (C2,C3; major)

This statement originates from the Executive Summary of Chapter 10 of the main 

text. Section 10.4.2 of the main text does not clearly reveal projections indicating 

an absolute decrease in freshwater availability in all indicated areas by the 2050s. 

There is no indication of how freshwater availability should be interpreted in this 

statement, that is, whether it concerns absolute flows or per-capita availability, 

which makes it hard to establish the line of reasoning that was followed, since a 

combination of evidence using different metrics has been cited in the chapter (C3). 

In the main chapter text, there are various referenced projections of ‘water stress’ 

(availability relative to demand) for South and Southeast Asia by 2050, projections 

of reduced run-off in mid-China by the end of the 21st century, and reductions in 

the per-capita availability of fresh water in India by 2050. There is a reference to a 

region in Central Asia (Hindu-Kush) and its dependence on meltwater from glaciers, 

but no long-term projections are given. However, according to model calculations, 

the runoff projections for 2100 in the sectoral Chapter 3 (‘Freshwater resources and 

their management’) show an increase in annual runoff in South, East and Southeast 

Asia (Figure 3.8, on page 201). This, of course, does not contradict possible severe 

water stress during dry seasons, but neither does it support the generalisation that 

led to the statement in the Summary for Policymakers (C2).

Coastal areas, especially heavily populated megadelta regions in South, East and 

South-East Asia, will be at greatest risk due to increased flooding from the sea and, 

in some megadeltas, flooding from the rivers.

This statement is fully supported by the underlying material.

Climate change is projected to compound the pressures on natural resources and 

the environment associated with rapid urbanisation, industrialisation and economic 

development.

This statement is fully supported by the underlying material.

Endemic morbidity and mortality due to diarrhoeal disease primarily associated with 

floods and droughts are expected to rise in East, South and South-East Asia due to 

projected changes in the hydrological cycle. (C3; minor)

This statement is fully supported by the underlying material.

6
60 Assessing an IPCC assessment 

6.2 Additional findings

The already discovered error on the melting rate of glaciers (Section 10.6.2 of the 

main text) is treated in Annex B.

The frequency and extent of forest fires in northern Asia is expected to increase in 

the future due to climate change and extreme weather events that could likely limit 

forest expansion. (C3,C6; major) 

See the Executive Summary, page 471, the Technical Summary, pages 49, and Box 

TS.6 on page 60. We have a comment to make on this statement. In the underlying 

text, nuances and uncertainties are clearly described: ‘The observations in the 

past 20 years show that the increasing intensity and spread of forest fires in North and 

South-East Asia were largely related to rises in temperature and declines in precipitation 

in combination with increasing intensity of land uses (see Section 10.2.4.4).Whether 

this trend will persist in the future or not is difficult to ascertain in view of the limited 

literature on how the frequency and severity of forest and brush fires will likely respond 

to expected increase in temperature and precipitation in North and South-East Asia (see 

Section 10.3.1).’ In the text, the expected future increase was solely underpinned 

by a reference to a single conference paper, which was based on a speech by the 

Vice-Minister of the Ministry of the Russian Federation for Civil Defence and did not 

include any references (Vorobyov, 2004). Given the response by the authors that 

the statement is an adequate reflection of other literature not listed in the chapter, 

we concluded that the statement is an insufficiently transparent expert judgment 

(C3) and that the referencing was unnecessarily relying on grey literature (C6). The 

IPCC authors explained to us the policy relevance of including this statement in the 

Executive Summary of the chapter and in the Technical Summary. Even though the 

research in this area was in an early stage and the modelling of future impacts was 

limited in 2006, the authors considered this subject to be sufficiently important 

because of its potential scale of effect. The statement was awarded only medium 

confidence (having an even chance of being correct), but given the use of the word 

‘likely’, we think it could have been worded more carefully. These major comments 

have no consequences for the IPCC conclusions in the various Summaries for 

Policymakers.

Recent risk analysis of coral reef suggests that between 24% and 30% of the reefs 

in Asia are likely to be lost during the next 10 years and 30 years, respectively. (E2; 

minor)

For this statement, see the Executive Summary, page 471, and the Technical 

Summary, Box TS.6 on page 59. We identified an inaccurate reference affecting this 

statement. The statement cannot be deduced from the figures presented in the 

included table (Table 10.6). Also, the underlying reference (Wilkinson, 2004) does 

not provide information to support the statement. The IPCC authors indicated that 

(Wilkinson, 2004) was cited incorrectly; this should have been (Wilkinson, 2000) 

(E2). This minor inaccuracy has no impact on the IPCC conclusions in the various 

Summaries for Policymakers.
Asia 61

The Third Assessment Report (TAR) predicted that the area-averaged annual mean 

warming would be about 3°C in the decade of the 2050s and about 5°C in the 

decade of the 2080s over the land regions of Asia as a result of future increases in 

atmospheric concentration of greenhouse gases (Lal et al., 2001a). (minor).

For this statement, see Section 10.1.1 of the main text, page 471. We have one 

minor comment to make on this statement (which could not be placed in one of 

the categories). In the First Order Draft, the statement used the word ‘reported’ 

instead of ‘predicted, and one of the comments in the review of the First Order 

Draft was that ‘The TAR reported that...’ should be changed to ‘The TAR predicted 

that...’ or ‘The TAR forecasted that...’. This comment was responded to, saying that 

‘IPCC Assessment Reports do not predict or forecast – they only report’, however, 

the sentence was changed nevertheless to ‘The TAR predicted that...’. It may 

be useful to note that in the IPCC reports, normally, the verb ‘to project’ is used 

instead of ‘to predict’ to denote the fact that a statement is made relative to a 

particular scenario. A comment on the Second Order Draft, made by the Republic of 

Korea, was to ‘specify the SRES scenario used for this assessment statement’. This 

comment was accepted according to the authors’ response (‘Appropriate revisions 

and editing made’), but no changes were implemented in the final texts. This minor 

comment has no consequences for the IPCC conclusions in the various Summaries 

for Policymakers.

References

Vorobyov,Y., 2004. Climate change and disasters in Russia. Proc. World Climate Change Conference, 

Moscow, Y. Izrael, G. Gruza, S. Semenov and I. Nazarov, Eds., Institute of Global Climate and 

Ecology, Moscow, 293-298.

Wilkinson, C., Ed., 2000. Status of Coral Reefs of the World: 2000.Australian Institute of Marine 

Science, Townsville, 363 pp.

Wilkinson, C., Ed., 2004. Status of Coral Reefs of the World: 2004, Volume 1. Australian Institute of 

Marine Science, Townsville, 302 pp.
62 Assessing an IPCC assessment 
Australia and New Zealand 63

Australia and 

New Zealand 

7.1 Analysis of statements in Table SPM.2 of the Synthesis Report

By 2020, significant loss of biodiversity is projected to occur in some ecologically 

rich sites, including the Great Barrier Reef and Queensland Wet Tropics. (E2,C3; 

minor)

In the text of Chapter 11 of the Working Group II Report, which underpins the 

statement, we found an inaccurate reference. Biodiversity loss on the Great Barrier 

Reef is covered by Table 11.6, in which is stated, for 2020: ‘Bleaching and damage 

to the Great Barrier Reef equivalent to that in 1998 and 2002 in up to 50% of years. 

(Berkelmans et al., 2004; Crimp et al., 2004) and 60% of the Great Barrier Reef regularly 

bleached (Jones, 2004b).’ The references for the first and second statement are 

not correct (E2). The IPCC authors indicated that the correct reference for both 

statements is Sheehan et al. (2006).

Furthermore, we were unable to derive the statements in Table 11.6 from the 

provided, correct reference (Sheehan et al, 2006). The reference relates impacts 

to temperature changes, not to specific years, which in this case would have been 

2020. Implicitly, the IPCC authors converted warming into years by using Table 11.4, 

that is, a near-coast warming in the range of 0.1 to 1.0 °C by 2020. As also confirmed 

by the IPCC authors, this conversion should have been explicitly stated in the 

caption of Table 11.6 (C3). We consider this to be a minor comment.

By 2030, water security problems are projected to intensify in southern and eastern 

Australia and, in New Zealand, in Northland and some eastern regions.

This statement is fully supported by the underlying material.

By 2030, production from agriculture and forestry is projected to decline over much 

of southern and eastern Australia, and over parts of eastern New Zealand, due to 

increased drought and fire. However, in New Zealand, initial benefits are projected 

in some other regions.

This statement is fully supported by the underlying material.

By 2050, ongoing coastal development and population growth in some areas of 

Australia and New Zealand are projected to exacerbate risks from sea level rise and 

increases in the severity and frequency of storms and coastal flooding.

This statement is fully supported by the underlying material.

7
64 Assessing an IPCC assessment 

7.2 Additional findings

3,000 to 5,000 more heat related deaths per year. (C1; major)

This example of regional impact was taken from Table TS.4 of the Technical 

Summary. We have a major comment to make on this statement.

From Table TS.4 readers may mistakenly conclude that 3,000 to 5,000 more heatrelated deaths could be directly attributed to temperature change (as is the case 

with the ‘additional people suffering from increased water stress’ also shown 

in the table). However, the largest part of the 3,000–5,000 range is dependent 

on expected changes in population sizes and age distributions within cities (C1). 

With the other drivers remaining constant, temperature change would only be 

responsible for 300 to 900 more heat-related deaths (this can be calculated 

from Table 2 in McMichael et al., 2003). This is only a small part of the 3,000 to 

5,000 more heat-related deaths mentioned in the table. The underpinning for the 

statement in Table TS.4 is given in Section 11.4.11 on health. The relevant sentence 

is: ‘Assuming no planned adaptation, the number of deaths is likely to rise from 1,115/

yr at present in Adelaide, Melbourne, Perth, Sydney and Brisbane to 2,300 to 2,500/yr 

by 2020, and 4,300 to 6,300/yr by 2050, for all SRES scenarios, including demographic 

change (McMichael et al., 2003)’. The stated numbers include both demographic and 

temperature effects. In the chapter is not mentioned that the effect of temperature 

change alone is relatively small, compared to the combined effect of temperature 

change and changes in population sizes and age distributions. 

We believe that the total number of heat-related deaths by 2050 (3,000–5,000) is 

a number that IPCC should publish, as policymakers should be made aware of all

heat-related deaths, not just those due to climate change. However, it is also our 

opinion that an IPCC report on climate-change impacts should show the different 

components including the climate-change related component, if available or 

traceable. It is no easy matter to establish a climate-change related component, 

but one way of doing this would be to make a comparison between changes in 

the demographic situation of 2050 with climate change and without. A simple 

calculation performed by the PBL resulted in 1,000 to 3,000 additional heat-related 

death per year, as a consequence of climate change, which is significantly less 

than the 3,000 to 5,000 range given in Table TS.4. This major comment has no 

consequences for the IPCC conclusions in the various Summaries for Policymakers.

Heat waves and fires are virtually certain to increase in intensity and frequency. (C4; 

minor)

This statement originates from the Executive Summary. We have a minor comment 

to make on this statement. In our opinion, there is an inconsistency between the 

level of likelihood in the Executive Summary and in the chapter text. The relevant 

chapter text in Section 11.3.1 repeatedly uses ‘likely’ for increases related to fires, 

and it not clear why ‘likely’ was subsequently translated to ‘virtually certain’ in the 

Executive Summary (C4). The IPCC authors have concluded, after consultation with 

authors of relevant references, that the likelihoods in the chapter may have been 

understated. This minor comment has no consequences for the IPCC conclusions in 

the various Summaries for Policymakers.
Australia and New Zealand 65

Since 1950 there has been a 0.3 to 0.7 °C warming in the region. (E1; minor)

This statement, taken from the Technical Summary on page 50, is different from 

the Executive Summary, which states: ‘Since 1950, there has been 0.4 to 0.7°C 

warming,...’. This is a minor inaccuracy (E1a). The authors confirmed this to be a 

typographical error in the Technical Summary; 0.4 °C is the correct number. This 

minor inaccuracy has no impact on the IPCC conclusions in the various Summaries 

for Policymakers.

References

Berkelmans, R., G. De’ath, S. Kininmonth and W.J. Skirving, 2004. A comparison of the 1998 and 2002 

coral bleaching events of the Great Barrier Reef: spatial correlation, patterns and predictions. 

Coral Reefs, 23, 74-83.

Crimp, S., J. Balston, A. Ash, L. Anderson-Berry, T. Done, R. Greiner, D. Jones, R.N., 2004b. Managing 

climate change risks. The Benefits of Climate Policies: Analytical and Framework Issues, J. Corfee 

Morlot and S. Agrawala, Eds., OECD, 251-297.

http://www.oecd.org/document/35/0,2340,en_ 2649_34361_ 34086819_1_1_1_1,00.html.

McMichael, A., R. Woodruff, P. Whetton, K. Hennessy, N. Nicholls, S. Hales, A. Woodward and 

T. Kjellstrom, 2003. Human Health and Climate Change in Oceania: A Risk Assessment 2002. 

Commonwealth Department of Health and Ageing, 128 pp. http://www.health.gov.au/internet/

main/publishing.nsf/Content/2D4037B384BC05F6CA256F1900042840/$File/env_climate.pdf

Sheehan, P., R. Jones, A. Jolley, B.L. Preston, M. Clark, P.J. Durack, S. Islam, F.Sun, and P.H. 

Whetton, 2006. Climate Change and the Global Knowledge Economy: An Immediate Challenge. 

CSES Climate Change Working Paper No. 11. Victoria University Centre for Strategic Economic 

Studies, Melbourne, 35 pp.

http://www.cfses.com/documents/climate/11_Sheehan_Jones_et_al_Climate_Change_.pdf
66 Assessing an IPCC assessment 
Europe 67

Europe

8.1 Analysis of statements in Table SPM.2 of the Synthesis Report

Climate change is expected to magnify regional differences in Europe’s natural 

resources and assets. Negative impacts will include increased risk of inland flash 

floods and more frequent coastal flooding and increased erosion (due to storminess 

and sea level rise). 

This statement is fully supported by the underlying material.

Mountainous areas will face glacier retreat, reduced snow cover and winter tourism, 

and extensive species losses (in some areas up to 60% under high emissions 

scenarios by 2080). (C3; minor)

We have one minor comment to make on this statement (which is also included, 

verbatim, in the Summary for Policymakers of the Working Group II Report). 

The Executive Summary only addresses extensive species loss in ‘mountain 

plant communities, which face a loss in species of up to 60% under high emission 

scenarios’, but does not mention mountain fauna, nor does Thuiller et al. (2005), 

the publication on which the example was based. Thus, the example given in the 

statement in the Summary for Policymakers (‘in some areas up to 60%...’) was not 

fully derived from the text in the Executive Summary or from the main text of 

Chapter 12. In addition, it is not transparent how the authors have underpinned 

the example (C3). After an additional literature search on our part, it appeared 

that there was a reference that the authors could have used for underpinning the 

example. A report by the Potsdam Institute for Climate Impact Research (PIK) 

(Schröter et al., 2004) that was not included in the references of the chapter, 

explicitly presented quantitative results which refer to species loss in mountain 

flora and fauna of up to 60% (Figure 25 in Schröter et al., 2004). We judged this 

omission of a reference and of a more traceable line of reasoning from the chapter 

to the Summary for Policymakers to be of minor effect.

In southern Europe, climate change is projected to worsen conditions (high 

temperatures and drought) in a region already vulnerable to climate variability, and 

to reduce water availability, hydropower potential, summer tourism and, in general, 

crop productivity.

This statement is fully supported by the underlying material.

Climate change is also projected to increase the health risks due to heat waves and 

the frequency of wildfires.

This statement is fully supported by the underlying material.

8
68 Assessing an IPCC assessment 

8.2 Additional findings

The already discovered error on the Dutch land area below sea level (in the main 

text of Chapter 12, on page 547) is discussed in Annex C.

Adaptation to climate change is likely to benefit from experiences gained in 

reactions to extreme climate events, by specifically implementing proactive climatechange risk management adaptation plans (very high confidence). (C4; minor)

This statement appears in the Technical Summary (page 53) and in the Summary for 

Policymakers of the Working Group II Report (page 14). However, the exact same 

statement received a lower confidence level (‘high confidence’) in the Executive 

Summary (page 543) (C4). The IPCC authors replied that the correct level of 

confidence should have been ‘very high confidence’ This was an editorial mistake; 

IPCC has indicated that it will issue an erratum. This minor inconsistency has no 

impact on the IPCC conclusions in the Summary for Policymakers of the Synthesis 

Report.

In southern Europe, general decreases in yield (e.g., legumes -30 to +5%; sunflower 

-12 to +3% and tuber crops -14 to +7% by 2050) and increases in water demand (e.g., 

for maize +2 to +4% and potato +6 to +10% by 2050) are expected for spring sown 

crops (Giannokopoulos et al., 2005; Audsley et al., 2006). (E1; minor)

The statement, which was taken from the main text of Chapter 12 (page 555), 

is slightly inaccurate (E1a). In Giannokopoulos et al. (2005), four Mediterranean 

regions have been distinguished. Only two are located in Europe (those labelled 

‘N-W’ and ‘N-E’). The numbers (ranges) presented in the IPCC Working Group II 

Report refer to all four regions, including North Africa, Jordan, Libya and Egypt, 

and should not have been reported in the chapter on Europe. The correct ranges 

for the European regions should read: Legumes (beans/soybean) -14 % to +1 %; C3 

(sunflower) -12% to + 1%; and Tubers (potato) -9% to +8 %). The IPCC authors have 

told us that this finding will lead to an IPCC erratum. These minor inaccuracies have 

no impact on the IPCC conclusions in the various Summaries for Policymakers.

Annual average runoff is projected to increase in northern Europe (north of 47°N) 

by approximately 5 to 15% up to the 2020s and 9 to 22% up to the 2070s, for the 

SRES A2 and B2 scenarios and climate scenarios from two different climate models 

(Alcamo et al., 2007) (Figure 12.1). Meanwhile, in southern Europe (south of 47°N), 

runoff decreases by 0 to 23% up to the 2020s and by 6 to 36% up to the 2070s (for the 

same set of assumptions). (C3; minor)

We have a minor comment to make on this statement. The quantitative information 

in this statement (Chapter 12, page 549), as well as in Figure 12.1, could not be 

found in the specific paper cited (Alcamo et al., 2007) (C3). In a reaction, the IPCC 

authors responded by saying that – for the accomplishment of the most up-to-date 

review of scientific knowledge – they sometimes also used underlying data that had 

formed the basis of the results in the peer-reviewed publications; they obtained 

this background data by contacting the authors of these publications and asking for 

specific figures. Our comment does not imply that such information should not be 

used, but only that the provenance of this information could have been made more 

transparent. This minor comment has no consequences for the IPCC conclusions in 

the various Summaries for Policymakers.
Europe 69

Climate-related increases in crop yields are expected mainly in northern Europe, 

e.g., wheat: +2 to +9% by 2020, +8 to +25% by 2050, +10 to +30% by 2080 

(Alexandrov et al., 2002; Ewert et al., 2005; Audsley et al., 2006; Olesen et al., 2007), 

and sugar beet +14 to +20% until the 2050s in England and Wales (Richter and 

Semenov, 2005),... The impacts on autumn sown crops are more geographically 

variable; yield is expected to strongly decrease in most southern areas, and increase 

in northern or cooler areas (e.g., wheat: +3 to +4% by 2020, -8 to +22% by 2050, -15 

to +32% by 2080) (Santos et al., 2002; Giannakopoulos et al., 2005; Audsley et al., 

2006; Olesen et al., 2007). (C3; minor)

We have a minor comment to make on this statement. It is not transparent how 

the quantitative information on wheat yield and sugar beet as presented in this 

statement (Chapter 12, page 555) was derived from the numbers given in the 

cited references (C3). The IPCC authors explained to us in more detail how these 

numbers had been established on the basis of the cited references. Although we 

have no reason to doubt the correct underpinning of these results, we do think that 

the provenance of presented quantitative information in IPCC reports – if it cannot 

be derived from the cited references – should be clearly explained for reasons of 

transparency and traceability. This minor comment has no consequences for the 

IPCC conclusions in the various Summaries for Policymakers.

However, on balance health risks are very likely to increase (C3; major)

We have a major comment to make on this statement, which was taken from the 

Executive Summary (page 543) and summarises the IPCC authors’ expert judgment 

on the balance between positive and negative effects of climate change on health. 

Supportive material is presented in the main text (see e.g. Section 12.4.11 on pages 

557–558; Figure 12.3 on page 558; Table 12.1 on page 546, and Table 12.4 on page 

565), which also provides a more regionally differentiated picture. However, a 

quantitative underpinning for the statement is lacking in the main text (C3).

The IPCC authors indicated to us that they weighed the positive and negative 

effects and this was their final assessment. Pages 557 to 558 of Chapter 12 give a 

detailed account of all impacts, and we found the negative impacts to be the norm, 

not the positive ones. However, although the list of negative effects is longer, 

no indication of their relative weights is given, which makes the expert judgment 

about the ‘balance’ in our view insufficiently transparent.

Although the statement was not included in the various Summaries for 

Policymakers, many readers may consider it to be highly policy relevant.

This major comment has no consequences for the IPCC conclusions in the various 

Summaries for Policymakers.
70 Assessing an IPCC assessment 

8.3 Findings from the PBL registration website

For the first time, wide-ranging impacts of changes in current climate have been 

documented: retreating glaciers, longer growing seasons, shift of species ranges, 

and health impacts due to a heat wave of unprecedented magnitude. (minor)

We received a reaction to this statement which criticised the presenting of the 

2003 heat wave as a consequence of climate change. In response to that reaction, 

we wish to make a minor comment on this statement (although it does not fit one 

of our qualification categories). The Working Group II Summary for Policymakers 

states, for Europe, on page 14: ‘For the first time, wide-ranging impacts of changes in 

current climate have been documented: retreating glaciers, longer growing seasons, shift 

of species ranges, and health impacts due to a heat wave of unprecedented magnitude. 

The observed changes described above are consistent with those projected for future 

climate change.’ This text, as well as its counterparts in Table TS4.2 of the Technical 

Summary (page 51) and in the Executive Summary (page 543), present the health 

impacts from the 2003 heat wave as an example of ‘wide-ranging impact of 

changes in current climate’. Thus, the text implicitly suggests that the 2003 heat 

wave is the result of recent climate change. However, one can never attribute a 

specific extreme weather event of the past – such as that particular heat wave 

–to changes in current climate. In fact, Schär and Jendritzky (2004) concluded: 

‘The European heatwave of 2003: was it merely a rare meteorological event or a first 

glimpse of climate change to come? Probably both.’ The IPCC authors have indicated 

that the statement in the Summary for Policymakers does not contain a formal 

attribution – otherwise, the second sentence ‘the observed changes described 

above are consistent with those projected for future climate change’ would not 

have been necessary; the authors’ reaction is factually correct, but the wording 

remains suggestive in our view. Chapter 1 of the Working Group II Report contains 

statements that are more careful, assessing the evidence of observed increases in 

heat waves in Europe. This minor comment has no consequences for the Summary 

for Policymakers of the Synthesis Report.

References

Alcamo, J., M. Floerke and M. Maerker, 2007. Future long-term changes in global water resources 

driven by socio-economic and climatic changes. Hydrological Sciences, 52, 247-275.

Alexandrov, V., J. Eitzinger, V. Cajic and M. Oberforster, 2002. Potential impact of climate change on 

selected agricultural crops in north-eastern Austria. Glob. Change Biol., 8, 372-389.

Audsley, E., K.R. Pearn, C. Simota, G. Cojocaru, E. Koutsidou, M.D.A. Rounsevell, M. Trnka and V. 

Alexandrov, 2006. What can scenario modelling tell us about future European scale agricultural 

land use, and what not? Environ. Sci. Pol., 9, 148-162.

Ewert, F., M.D.A. Rounsevell, I. Reginster, M.J. Metzger and R. Leemans, 2005. Future scenarios of 

European agricultural land use I. Estimating changes in crop productivity. Agr. Ecosyst. Environ., 

107, 101-116.

Giannakopoulos, C., M. Bindi, M. Moriondo, P. LeSager and T. Tin, 2005. Climate Change Impacts 

in the Mediterranean Resulting from a 2oC Global Temperature Rise. WWF report, Gland 

Switzerland. Accessed 01.10.2006 at http://assets.panda.org/downloads/medreportfinal8july05.

pdf.
Europe 71

Olesen, J.E., T.R. Carter, C.H. Díaz-Ambrona, S. Fronzek, T. Heidmann, T. Hickler, T. Holt, M.I. 

Mínguez, P. Morales, J. Palutikhof, M. Quemada, M. Ruiz-Ramos, G. Rubaek, F. Sau, B. Smith and 

M. Sykes, 2007: Uncertainties in projected impacts of climate change on European agriculture 

and terrestrial ecosystems based on scenarios from regional climate models. Climatic Change, 81, 

S123-S143.

Richter, G. and M. Semenov, 2005: Re-assessing drought risks for UK corps using UKCIP02 climate 

change scenarios. Final report of Defra project CC0368. 

Schär, C. and G. Jendritzky, 2004: Climate change: hot news from summer 2003. Nature, 432, 559-

560.

Schröter, D., et al., 2004. Advanced Terrestrial Ecosystem Analysis and Modelling (ATEAM): Final 

Report 2004. Section 5 and 6 and Annex 1 to 6. Accessed 22-4-2010 at http://www.pik-potsdam.de/

ateam/ateam_final_report_sections_5_to_6.pdf.

Santos, F.D., K. Forbes, and R. Moita, Eds., 2002: Climate change in Portugal: scenarios, impacts and 

adaptation measures. SIAM project report, Gradiva, Lisbon, 456 pp.

Thuiller, W., S. Lavorel, M.B. Araújo, M.T. Sykes and I.C. Prentice, 2005: Climate change threats plant 

diversity in Europe. Proc. Natl. Acad. Sci. U.S.A., 102, 8245-8250.
72 Assessing an IPCC assessment 
Latin America 73

Latin America 

9.1 Analysis of statements in Table SPM.2 of the Synthesis report

By mid century, increases in temperature and associated decreases in soil water are 

projected to lead to gradual replacement of tropical forest by savanna in eastern 

Amazonia. Semi-arid vegetation will tend to be replaced by arid-land vegetation.

This statement is fully supported by the underlying material.

There is a risk of significant biodiversity loss through species extinction in many 

areas of tropical Latin America.

This statement is fully supported by the underlying material.

Productivity of some important crops is projected to decrease (a) and livestock 

productivity to decline (b), with adverse consequences for food security. In 

temperate zones, soybean yields are projected to increase (c). Overall, the number 

of people at risk of hunger is projected to increase (d). (C2; major) 

We subdivided this statement by inserting (a) to (d) between issues. We have one 

comment to make on part (b).

Part (a) (Productivity of some important crops is projected to decrease) This is fully 

supported by the underlying material.

Part (b) (Livestock productivity is projected to decline). The ‘livestock’ appears to be 

limited to ‘cattle’ and this part of the statement is underpinned by references to 

studies on Bolivia and central Argentina. The generalisation to other livestock, 

which would include pigs, poultry, small ruminants and other animals (together also 

comprising a substantial fraction of livestock production in Latin America) and to 

the whole of Latin America, is not underpinned in Chapter 13. We therefore believe 

that this is an ‘insufficiently founded generalisation’ (C2). 

Statement (c) (In temperate zones, soybean yields are projected to increase). This is fully 

supported by the underlying material.

Statement (d) (The number of people at risk of hunger is projected to increase). This is 

fully supported by the underlying material.

Changes in precipitation patterns and the disappearance of glaciers are projected to 

significantly affect water availability for human consumption, agriculture and energy 

generation.

This statement is fully supported by the underlying material.

9
74 Assessing an IPCC assessment 

9.2 Additional findings

Table 13.6 Net increases in the number of people living in water-stressed 

watersheds. (E1; minor)

Table 13.6 (on page 598 of Chapter 13) combines the absolute number of people 

that live under water-stressed conditions (for the non-climate cases, columns 2, 3 

and 5) with the number of people that experience an increase in water stress due to 

climate change (Arnell, 2004). The table’s caption, however, suggests all numbers 

are increases, but this is not correct (E1a). This issue also had been recognised by 

the IPCC authors, who have indicated to us that they intend to have an erratum 

issued. Note that we consider this to be a minor inaccuracy and that we have no 

comment to make about the table’s content. This minor inaccuracy has no impact 

on the IPCC conclusions in the various Summaries for Policymakers.

...the frequency of occurrence of weather and climate extremes is likely to increase 

in the future; as is the frequency and intensity of hurricanes in the Caribbean Basin. 

(E1; minor)

This statement was taken from the Executive Summary of Chapter 13, page 583, 

right column, second paragraph. The statement contains a minor inaccuracy. The 

Technical Summary of the Working Group I 2007 Report (p. 74) states: ‘A synthesis 

of the model results to date indicates, for a warmer future climate, increased peak wind 

intensities and increased mean and peak precipitation intensities in future tropical 

cyclones, with the possibility of a decrease in the number of relatively weak hurricanes, 

and increased numbers of intense hurricanes. However, the total number of tropical 

cyclones globally is projected to decrease’. We conclude that ‘frequency and intensity 

of hurricanes’ should have read ‘frequency and intensity of strong hurricanes’ 

(E1a). This minor inaccuracy has no impact on the IPCC conclusions in the various 

Summaries for Policymakers.

Up to 40% of the Amazonian forests could react drastically to even a slight 

reduction in precipitation; this means that the tropical vegetation, hydrology and 

climate system in South America could change very rapidly to another steady state. 

(C6; minor)

We have a minor comment to make on this statement, which originates from 

Section 13.4.1 of Chapter 13 (page 596). The statement was based on Rowell 

and Moore (2000), which is a peer-reviewed report by the World Wide Fund for 

Nature and the International Union for Conservation of Nature (WWF/IUCN) on 

a global review of forest fires, and not a study on changes in vegetation due to 

climate change. That report, in turn, was mainly based on Nepstad et al. (1999) 

(in Nature). In our opinion, both documents were not the most obvious choice of 

reference in this case, as their focus is on forest fires (and logging). More adequate 

peer-reviewed, scientific journal literature would have been available to support 

this statement, such as Cox et al. (2000; 2004) (C6). This minor comment has no 

consequences for the IPCC conclusions in the various Summaries for Policymakers.

...some plants will become locally extinct because the elevation range would not 

permit natural adaptation to temperature increase (FAO, 2002). (C5; minor)

This statement originates from Section 13.4.1 of Chapter 13 (page 596, right 

column). We were unable to trace FAO (2002) and the URL in the references does 
Latin America 75

not provide a link to the FAO website (C5). This issue also has been recognised by 

the IPCC authors, who have indicated that they intend to have an erratum issued. 

This minor comment has no consequences for the IPCC conclusions in the various 

Summaries for Policymakers.

...the incidence of the coffee leafminer (Perileucoptera coffeella) and the nematode 

Meloidogyne incognita are likely to increase in future in Brazil’s production area. 

The number of coffee leaf miner cycles could increase by 4%, 32% and 61% in 2020, 

2050 and 2080, respectively, under the SRES A2 scenarios (Ghini, 2007). (C5; minor)

This statement originates from Section 13.4.2 of Chapter 13 (page 597, right 

column). We have a minor comment to make on this statement. The reference to 

the paper by Ghini et al. (2007) was listed as having been submitted to the journal 

Climatic Change, but was never published in Climatic Change (C5). It was published 

in Pesquisa Agropecuária Brasileira, Brasília 43 (2): 187-194, in 2008 (Ghini et al., 

2008). We are of the opinion that referring to submitted papers that have not been 

accepted for publication is incorrect, since they have not yet completed the peerreview process. Moreover, the reference may end up to be untraceable, as was 

the case here. In our opinion, papers should at least be accepted for publication, 

or be in press, so that traceability is guaranteed. This minor comment has no 

consequences for the IPCC conclusions in the various Summaries for Policymakers.

A highly stressed condition is projected between 2015 and 2025 in the water 

availability in Colombia, affecting water supply and ecosystem functioning in the 

páramos (IDEAM, 2004), and very probably impacting on the availability of water 

supply for 60% of the population of Peru (Vásquez, 2004). The projected glacier 

retreat would also affect hydroelectricity generation in some countries, such as 

Colombia (IDEAM, 2004) and Peru; one of the more affected rivers would be the 

Mantaro, where an hydroelectric plant generates 40% of Peru’s electricity and 

provides the energy supply for 70% of the country’s industries, concentrated in Lima 

(UNMSM, 2004). (E2; minor)

In this statement, taken from Section 13.4.3 of Chapter 13 (page 598, right column) 

we found that the references to UNMSM and Vasquez had been mixed up. The 

statement on the water supply in Peru is not supported by the Vásquez (2004) 

paper, but by UNMSM (2004). and the statement on hydro-electricity in the 

Mantaro river is supported by Vásquez (2004) (E2). This minor inaccuracy has no 

impact on the IPCC conclusions in the various Summaries for Policymakers.

References

Arnell, N.W., 2004. Climate change and global water resources: SRES emissions and socio-economic 

scenarios. Global Environ. Change, 14, 31-52. 

Cox, P.M., R.A. Betts, C.D. Jones, S. Spall and I.J. Totterdell, 2000. Acceleration of global warming 

due to carbon-cycle feedbacks in a coupled climate model. Nature, 408, 184-187.

Cox, P.M., R.A. Betts, M. Collins, P. Harris, C. Huntingford, C.D. Jones, 2004. Amazon dieback under 

climate-carbon cycle projections for the 21st century. Theor. Appl. Climatol. 78, 137–156. (doi:10. 

1007/s00704-004-0049-4)

FAO, 2002. El cambio climático y los bosques. Boletín electrónico Julio 2002, FAO, Rome. http://

www.ecosur.net/cambio_climatico_y_los_bosques.html.

Ghini, R., E. Hamada, M.J. Pedro Júnior and J.A. Marengo, 2007. Climate change and coffee pests in 

Brazil. Climatic Change, submitted. 
76 Assessing an IPCC assessment 

Ghini, R., E. Hamada, M.J. Pedro Júnior and J.A. Marengo, 2008. Risk analysis of climate change on 

coffee nematodes and leaf miner in Brazil. Pesquisa Agropecuária Brasileira, Brasília 43 (2): 187-

194.

Nepstad, D.C., A. Veríssimo, A. Alencar, C. Nobre, E. Lima, P. Lefebvre, P. Schlesinger, C. Potter, 

P. Mountinho, E. Mendoza, M. Cochrane, V. Brooks, 1999. Larg e scale impoverishment of 

Amazonian forests by logging and fire, Nature, Vol. 398, 8 April. 

Rowell, A. and P.F. Moore, 2000. Global Review of Forest Fires. WWF/IUCN, Gland, Switzerland, 66 

pp. http://data.iucn.org/dbtw-wpd/edocs/2000-047.pdf. 

UNMSM, 2004. Calor intenso y largas sequías. Especials, Perú. http://www.

unmsm.edu.pe/Destacados/contenido.php?mver=11.

Vásquez, O.C., 2004. El Fenómeno El Niño en Perú y Bolivia: Experiencias en Participación Local. 

Memoria del Encuentro Binacional Experiencias de prevención de desastres y manejo de 

emergencias ante el Fenómeno El Niño, Chiclayo, Peru. ITDG, 209 pp.
North America 77

North America 

10.1 Analysis of statements in Table SPM.2 of the Synthesis Report

Warming in western mountains is projected to cause decreased snowpack, more 

winter flooding and reduced summer flows, exacerbating competition for overallocated water resources.

This statement is fully supported by the underlying material.

In the early decades of the century, moderate climate change is projected to 

increase aggregate yields of rain-fed agriculture by 5 to 20%, but with important 

variability among regions. Major challenges are projected for crops that are near the 

warm end of their suitable range or which depend on highly utilised water resources. 

(C3; minor)

From Chapter 14 and the references therein we could not derive the reasoning that 

led to the 5 to 20% range of yield increases under moderate climate change. Of the 

six references named in the chapter, Adams et al. (2003), appears to be the main 

reference. Adams et al. report (in their Table I) changes in rain-fed yield, from –8% to 

+41% for 2060 (with adaptation) (C3). Adams et al. and other references (Thomson 

et al. 2005, Tsvetsinkaya et al., 2003) show there are regions with significant yield 

decreases for specific rain-fed crops (5 to 25%), even if adaptation measures are 

adopted. It could be argued that this observation is covered by the phrase ‘but with 

important variability among regions’, but this could not be derived from the chapter 

text. The IPCC authors provided us with the reasoning that had been followed: for 

Adams et al. (2003), the projected yield changes for the four food crops considered 

(corn, soy, wheat, sorghum) are +7, +17, +13, and +5% respectively, according to the 

Regional Climate Model with adaptation. The whole idea of that paper was that 

the Regional Climate Model gave more credible projections than the Global Climate 

Model, so those are the numbers that the IPCC authors used. Furthermore, they 

confirmed that they highlighted variability due to, for example, different varieties 

of wheat with the statement that there is ‘important variability among regions’. 

They found it hard to see how they could have been more precise without reporting 

individual values from tables. We believe that the above explanation of the 

reasoning behind the 5 to 20% range could have been made more transparent with 

just a few extra lines of text. We wish to add here that the IPCC authors’ expert 

judgment had been based on a consideration of a vast body of literature and we 

have no reason to doubt the quality and correctness of the expert judgment.

10
78 Assessing an IPCC assessment 

Cities that currently experience heat waves are expected to be further challenged by 

an increased number, intensity and duration of heat waves during the course of the 

century, with potential for adverse health impacts).

This statement is fully supported by the underlying material.

Coastal communities and habitats will be increasingly stressed by climate change 

impacts interacting with development and pollution.

This statement is fully supported by the underlying material.

10.2 Additional findings

Figures TS.15 and 14.1 on the economic damages from hurricanes. (minor)

Figures TS.15 (Technical Summary, page 55) and 14.1 (Chapter 14, page 621) in 

the Working Group II Report illustrate the statement in the Technical Summary 

that reads ‘Over the past several decades, economic damage from hurricanes in 

North America has increased over fourfold, due largely to an increase in the value of 

infrastructure at risk’. We have a minor comment to make on the figures. Because 

of a shorter averaging period at the end of the time series (6 years instead of 

10), the most recent increase was higher, compared with the increase that would 

have resulted from applying a uniform averaging period at the end of the series. 

However, a different choice of statistical method would not have affected the 

statement. Furthermore, we have found this statement to be fully supported by 

the underlying material. This minor comment has no consequences for the IPCC 

conclusions in the various Summaries for Policymakers.

References

Adams, R.M., B.A. McCarl and L.O. Mearns, 2003. The effects of spatial scale of climate scenarios on 

economic assessments: An example from U.S. agriculture. Clim. Change, 60, 131-148.

Brown, T. J., B. L. Hall and A. L.Westerling, 2004. The impact of twenty-first century climate change 

on wildland fire danger in the western United States:An applications perspective. Clim. Change, 

62, 365-388.

Flannigan, M.D., K.A. Logan, B. D.Amiro,W. R. Skinner and B. J. Stocks, 2004. Future area burned in 

Canada. Clim. Change, 72, 1-16.

Thomson, A.M., R.A. Brown, N.J. Rosenberg, R.C. Izaurralde and V. Benson, 2005. Climate change 

impacts for the conterminous USA:An integrated assessment. Part 3:Dryland production of grain 

and forage crops. Clim. Change, 69, 43-65.

Tsvetsinskaya, E.A., L.O. Mearns, T. Mavromatis, W. Gao, L. McDaniel and M.W. Downton, 2003. The 

effect of spatial scale of climatic change scenarios on simulated maize, winter wheat, and rice 

production in the southeastern United States. Clim. Change, 60, 37-72.
Polar regions (Arctic and Antarctic) 79

Polar regions 

(Arctic and Antarctic) 

11.1 Analysis of statements in Table SPM.2 of the Synthesis Report

The main projected biophysical effects are reductions in thickness and extent of 

glaciers, ice sheets and sea ice, and changes in natural ecosystems with detrimental 

effects on many organisms including migratory birds, mammals and higher 

predators.

This statement is fully supported by the underlying material.

For human communities in the Arctic, impacts, particularly those resulting from 

changing snow and ice conditions are projected to be mixed.

This statement is fully supported by the underlying material.

Detrimental impacts would include those on infrastructure and traditional 

indigenous ways of life.

This statement is fully supported by the underlying material.

In both polar regions, specific ecosystems and habitats are projected to be 

vulnerable, as climatic barriers to species invasions are lowered.

This statement is fully supported by the underlying material.

11
80 Assessing an IPCC assessment 
Small islands 81

Small islands 

12.1 Analysis of statements in Table SPM.2 of the Synthesis Report

Sea level rise is expected to exacerbate inundation, storm surge, erosion and other 

coastal hazards, thus threatening vital infrastructure, settlements and facilities that 

support the livelihood of island communities.

This statement is fully supported by the underlying material.

Deterioration in coastal conditions, for example through erosion of beaches and 

coral bleaching, is expected to affect local resources.

This statement is fully supported by the underlying material.

By mid-century, climate change is expected to reduce water resources in many 

small islands, e.g. in the Caribbean and Pacific, to the point where they become 

insufficient to meet demand during low-rainfall periods.

This statement is fully supported by the underlying material.

With higher temperatures, increased invasion by non-native species is expected to 

occur, particularly on mid- and high-latitude islands.

This statement is fully supported by the underlying material.

12.2 Additional findings

Lower rainfall coupled with accelerated sea-level rise compounds the threat on 

water resources; a 10% reduction in average rainfall by 2050 is likely to correspond 

to a 20% reduction in the size of the freshwater lens on Tarawa Atoll, Kiribati. (C3; 

minor)

We have a minor comment to make on this statement. The statement appears in 

the Technical Summary (Table TS2, page 58; Box TS.6, page 63), in the Executive 

Summary of Chapter 16 (page 689) and in the main chapter text (pages 696 and 

697), and is indicated to have been based on a World Bank report (World Bank, 

2000). However, this reference indicates a 14% reduction in the freshwater lens, 

as a consequence of a 10% reduction in precipitation. Figures change when other 

factors combine with changes in precipitation; −12% change in the freshwater lens 

is projected for 0.4 m sea level rise, and −38% for 0.4 m sea level rise in combination 

with reduced island size due to erosion. The World Bank report also notes that a 

10% increase in evapotranspiration would lead to a 6% reduction in the freshwater 

lens. The chapter text is not clear on why the effect of a projected 10% increase in 

evapotranspiration should be added to the effect of a projected 10% reduction in 

12
82 Assessing an IPCC assessment 

precipitation. The Coordinating Lead Authors explained to us that these changes 

are correlated and that, hence, the 6% effect from evapotranspiration change 

should indeed be added to the 14% effect from changes in precipitation. The 

statement is correct, but we believe that this last point should have been explained 

in the text. This minor comment has no consequences for the IPCC conclusions in 

the various Summaries for Policymakers.

Less rainfall coupled with accelerated sea-level rise would compound this threat. 

Studies conducted on Bonriki Island in Tarawa, Kiribati, showed that a 50 cm rise in 

sea level accompanied by a reduction in rainfall of 25% would reduce the freshwater 

lens by 65% (World Bank, 2000). (E2; minor)

Another instance of inaccurate referencing was found in the main text of Chapter 

16 (page 697). However, the World Bank study does not report a 50 cm sea level 

rise. The reference should have been to a study by Falkland (1999) (E2). This minor 

inaccuracy has no impact on the IPCC conclusions in the various Summaries for 

Policymakers.

References

World Bank, 2000: Cities, Seas and Storms: Managing Change in Pacific Island. Economies. Vol. IV: 

Adapting to Climate Change. World Bank, Washington, District of Columbia, 135 pp

Falkland, A., 1999: Impacts of climate change on water resources of Pacific islands. PACCLIM 19 

Workshop, Modelling the Effects of Climate Change and Sea Level Rise in Pacific Island 20 

Countries. Auckland, New Zealand, 33pp.
Annex A 83

Annex A Table SPM.2 of the 

Synthesis Report: examples of 

some projected regional impacts

This table including its footnote is a direct copy of the original table SPM.2 on page 

11 of Climate Change 2007 – Synthesis Report, A report of the IPCC.
84 Assessing an IPCC assessment 

Table SPM.2. Examples of some projected regional impacts {3.3.2}

Africa • By 2020, between 75 and 250 million of people are projected to be 

exposed to increased water stress due to climate change.

• By 2020, in some countries, yields from rain-fed agriculture could be reduced by up to 50%. Agricultural 

production, including access to food, in many African countries is projected to be severely 

compromised. This would further adversely affect food security and exacerbate malnutrition.

• Towards the end of the 21st century, projected sea level rise will affect low-lying coastal areas with large 

populations. The cost of adaptation could amount to at least 5 to 10% of Gross Domestic Product (GDP).

• By 2080, an increase of 5 to 8% of arid and semi-arid land in Africa is 

projected under a range of climate scenarios (TS).

Asia • By the 2050s, freshwater availability in Central, South, East and South-East 

Asia, particularly in large river basins, is projected to decrease.

• Coastal areas, especially heavily populated megadelta regions in South, East and South-East Asia, will be at 

greatest risk due to increased flooding from the sea and, in some megadeltas, flooding from the rivers.

• Climate change is projected to compound the pressures on natural resources and the environment 

associated with rapid urbanisation, industrialisation and economic development.

• Endemic morbidity and mortality due to diarrhoeal disease primarily associated with floods and droughts 

are expected to rise in East, South and South-East Asia due to projected changes in the hydrological cycle.

Australia and 

New Zealand

• By 2020, significant loss of biodiversity is projected to occur in some ecologically 

rich sites, including the Great Barrier Reef and Queensland Wet Tropics.

• By 2030, water security problems are projected to intensify in southern and eastern 

Australia and, in New Zealand, in Northland and some eastern regions.

• By 2030, production from agriculture and forestry is projected to decline over much of southern 

and eastern Australia, and over parts of eastern New Zealand, due to increased drought and 

fire. However, in New Zealand, initial benefits are projected in some other regions.

• By 2050, ongoing coastal development and population growth in some areas of 

Australia and New Zealand are projected to exacerbate risks from sea level rise and 

increases in the severity and frequency of storms and coastal flooding. 

Europe • Climate change is expected to magnify regional differences in Europe’s natural resources 

and assets. Negative impacts will include increased risk of inland flash floods and more 

frequent coastal flooding and increased erosion (due to storminess and sea level rise).

• Mountainous areas will face glacier retreat, reduced snow cover and winter tourism, and extensive species losses (in some areas up to 60% under high emissions scenarios by 2080).

• In southern Europe, climate change is projected to worsen conditions (high temperatures 

and drought) in a region already vulnerable to climate variability, and to reduce water 

availability, hydropower potential, summer tourism and, in general, crop productivity.

• Climate change is also projected to increase the health risks due to heat waves and the frequency of wildfires.

Latin America • By mid-century, increases in temperature and associated decreases in soil water are 

projected to lead to gradual replacement of tropical forest by savanna in eastern 

Amazonia. Semi-arid vegetation will tend to be replaced by arid-land vegetation.

• There is a risk of significant biodiversity loss through species extinction in many areas of tropical Latin America.

• Productivity of some important crops is projected to decrease and livestock productivity to decline, with 

adverse consequences for food security. In temperate zones, soybean yields are projected to increase. 

Overall, the number of people at risk of hunger is projected to increase (TS; medium confidence).

• Changes in precipitation patterns and the disappearance of glaciers are projected to significantly 

affect water availability for human consumption, agriculture and energy generation.

Table A.1
Annex A 85

North America • Warming in western mountains is projected to cause decreased snowpack, more winter flooding 

and reduced summer flows, exacerbating competition for over-allocated water resources.

• In the early decades of the century, moderate climate change is projected to increase aggregate yields of rainfed agriculture by 5 to 20%, but with important variability among regions. Major challenges are projected for 

crops that are near the warm end of their suitable range or which depend on highly utilized water resources.

• Cities that currently experience heat waves are expected to be further challenged 

by an increased number, intensity and duration of heat waves during the 

course of the century, with potential for adverse health impacts.

• Coastal communities and habitats will be increasingly stressed by climate 

change impacts interacting with development and pollution.

Polar Regions • The main projected biophysical effects are reductions in thickness and extent of glaciers, 

ice sheets and sea ice, and changes in natural ecosystems with detrimental effects on 

many organisms including migratory birds, mammals and higher predators.

• For human communities in the Arctic, impacts, particularly those resulting from 

changing snow and ice conditions, are projected to be mixed.

• Detrimental impacts would include those on infrastructure and traditional indigenous ways of life.

• In both Polar Regions, specific ecosystems and habitats are projected to be 

vulnerable, as climatic barriers to species invasions are lowered.

Small Islands • Sea level rise is expected to exacerbate inundation, storm surge, erosion and other coastal hazards, thus 

threatening vital infrastructure, settlements and facilities that support the livelihood of island communities.

• Deterioration in coastal conditions, for example through erosion of beaches 

and coral bleaching, is expected to affect local resources.

• By mid-century, climate change is expected to reduce water resources in many small islands, e.g. in the 

Caribbean and Pacific, to the point where they become insufficient to meet demand during low-rainfall periods.

• With higher temperatures, increased invasion by non-native species is 

expected to occur, particularly on mid- and high-latitude islands.

Note:

Unless stated explicitly, all entries are taken from the text of the Working Group II Summary for Policymakers. They are either ‘very high confidence’ or ‘high confidence’ statements, related to different 

sectors (agriculture, ecosystems, water, coasts, health, industry and settlements). The Working Group 

II Summary for Policymakers refers to the source of the statements, timelines and temperatures. The 

magnitude and timing of impacts that may ultimately occur will vary according to the amount and 

rate of climate change, emissions scenarios, development pathways and adaptation.
86 Assessing an IPCC assessment 

Annex B The error on the 

melting of the Himalayan 

glaciers

In the media, much attention was paid to this erroneous statement in the chapter 

on Asia, page 493 of Section 10.6.2, which suggested the disappearance of 

Himalayan glaciers by 2035:

Glaciers in the Himalaya are receding faster than in any other part of the world [...] 

the likelihood of them disappearing by the year 2035 and perhaps sooner is very 

high if the Earth keeps warming at the current rate. Its total area will likely shrink 

from the present 500,000 to 100,000 km2

 by the year 2035 (E1; major)

Here, we provide an analysis of this error and some background information, 

which also provides an update that includes literature that became available after

production and publication of the Fourth Assessment Report. 

In the Synthesis Report (IPCC, 2007) and in the Summary for Policymakers (SPM), 

and the Technical Summary (TS) of the Working Group I contribution to the IPCC 

Fourth Assessment Report (Solomon et al., 2007), no explicit statements were 

made on Himalayan glacier retreat. In the Summary for Policymakers of the 

Working Group II Report (Parry et al., 2007), there was only one reference to the 

Himalayas: 

Glacier melt in the Himalayas is projected to increase flooding, and rock avalanches from 

destabilised slopes, and to affect water resources within the next two to three decades. This will 

be followed by decreased river flows as the glaciers recede.

This statement is rather general and correct, although the impact on water 

resources due to glacier retreat is very likely to be rather limited in the next two to 

three decades. However, in the Technical Summary, a major error occurred in Box 

TS6, on page 59:

If current warming rates are maintained, Himalayan glaciers could decay at very rapid rates, 

shrinking from the present 500,000 km2 to 100,000 km2

 by the 2030s.

This text was based on page 493 of Section 10.6.2 of Working Group II Report, 

which states:
Annex B 87

Glaciers in the Himalaya are receding faster than in any other part of the world [...] the 

likelihood of them disappearing by the year 2035 and perhaps sooner is very high if the Earth 

keeps warming at the current rate. Its total area will likely shrink from the present 500,000 

to 100,000 km2

 by the year 2035 (WWF 2005).

The first mention of ‘2035’ was taken from a publication by the World Wide Fund for 

Nature (WWF, 2005), which cited a news story (Pearce, 1999) about an unpublished 

study (Hasnain, 1999) that does not estimate a date for disappearance of Himalayan 

glaciers.

The second mention of ‘2035’, again a major error, was not taken from the (WWF, 

2005), but could be traced to a rough estimate (Kotlyakov, 1996) on the shrinkage 

of all non-polar glaciers (excluding those in basins of internal drainage) between 

the present day and the year 2350. Furthermore, the Himalayan glaciers do not 

cover 500,000 km2

, but about 30,000 km2, as was correctly indicated in the first 

sentence of Section 10.6.2, which stated that: Himalayan glaciers cover about three 

million hectares.

In conflict with knowledge of glacier–climate relationships, disappearance by 2035 

would require a 25-fold acceleration of the loss rate during the 1999-2035 period, 

compared to the estimated past loss rate between 1960 and 1999 (Dyurgerov and 

Meier, 2005), a situation which is extremely unlikely to occur.

Here, we provide some additional background information, mainly based on a 

study by Oerlemans et al. (2007) and on a recent American Geophysical Union 

lecture by leading experts in this field (Kargel et al., 2010). The lecture was held 

on 14 December 2009 in San Francisco, in response to media attention concerning 

the claim in the IPCC Working Group II Report that the Himalayan glaciers might be 

gone by 2035 (Parry et al., 2007).

Glacier retreat and water supply

The region-wide average glacier area retreat rate is probably between 0.1 and 0.5% 

per year. Although the size of the glacier area will change substantially this century, 

especially in the most vulnerable eastern zone of the Himalayas, glaciers (such as 

Khumbu and Imja), will not disappear entirely, or even mostly, by 2035, as stated in 

the Working Group II contribution to the Fourth Assessment Report of the IPCC.

All glaciers combined store roughly 3,400 km3

 of fresh water. This represents 

almost three years in combined water discharges from the Indus, the Ganges and 

the Brahmaputra. The prevalent negative balances of Himalayan glaciers (meaning 

the annual net withdrawals of ice) total between 0.4 and 1.2% of current river flows, 

which is not very significant, basin-wide. So, although over half a billion people 

draw water from these melting glaciers, this only represents between one and a 

few per cent of their water supplies.

Increased melting may further increase water discharge by 1 to 2%, in the next few 

decades. Within 50 to 100 years, the discharge could decrease by several per cent, 

as the current negative balance contribution decreases. However, on the scale of 

the subcontinent, hydrological problems are likely to stem more from population 
88 Assessing an IPCC assessment 

growth, inefficiency in water resource distribution and application, and from 

changes in the Monsoon and the Westerlies, due to global warming and changes in 

the Elevated Heat Pump (EHP). Changes in the Monsoon and the Westerlies already 

appear to be substantial and are likely to be increasingly impactful.

On a local scale, the seasonal influence of glacier melt can be strong, and even 

rivers as large as the Indus fluctuate widely throughout the year because of the 

seasonality of water storage (mostly in the form of ice), and melting of both 

glaciers and high altitude snow fields. The greatest vulnerability of large numbers of 

people to glacier state and dynamics is to those in arid and semi-arid lands, such as 

north-eastern Afghanistan, Uzbekistan, parts of Pakistan, and western China. This 

concerns tens of millions of people.

Sea level rise

Since 1900, the majority of the observed glaciers around the world are retreating, 

including those in central Asia. Based on Oerlemans et al. (2007), global glacial 

contribution to sea level rise was 4.5±0.7 cm during the 1900-2000 period, 

compared to a total sea level rise of ~19.5 cm in the 1870-2004 period. This estimate 

is higher than the IPCC estimate of 3.0±1.0 cm for the 20th century, and better 

explains the observed sea level rise as caused by thermal expansion and changes 

in the cryosphere. Based on Oerlemans et al. (2007) and Kargel et al. (2010), it is 

roughly estimated that the annual contribution from Himalayan glaciers to sea level 

rise is 0.06±0.04 mm, or ~2% of the current annual sea level rise of 3.1±0.7 mm (Parry 

et al., 2007). The rate of Himalayan glacier melting appears to have increased over 

time, but so has sea level rise, and therefore the 2% contribution may still apply.

References

Dyurgerov, M.B. and M.F. Meier, 2005. Glaciers and the changing earth system, a 2004 snapshot, 

Occasional Paper No. 58. Boulder, Institute of Arctic and Alpine Research, University of Colorado: 

117

Hasnain, S.I., 1999. Report on Himalayan Glaciology. New Dehli, ICSI working group on Himalayan 

Glaciology: 22.

IPCC, 2007. Climate Change 2007. Synthesis report. Contribution of Working Groups I, II and III to the 

Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Core writing team: 

Pachauri, R.K. and Reisinger, A. (Eds.). Geneva, Switzerland.

Kargel, J.S., R. Armstrong et al., 2010. Satellite-era glacier changes in High Asia. Background support 

presentation for NASA “Black Carbon and Aerosols” press conference associated with Fall AGU, 

Dec. 14, 2009. Updated and expanded Feb. 17, 2010.

Kotlyakov, V.M., 1996. Variations of Snow and Ice in the past and at present on a Global and Regional 

Scale. I. H. PROGRAMME. Paris, UNESCO.

Oerlemans, J., M. Dyurgerov et al., 2007. "Reconstructing the glacier contribution to sea-level rise 

back to 1850." The Cryosphere 1: 59-65.

Parry, M.L., O.F. Canziani et al., 2007. Climate Change 2007: Impacts, Adaptation and Vulnerability. 

Cambridge, Cambridge University Press.

Pearce, F., 1999. Flooded Out. New Scientist. 162.

Solomon, S.D., M.M. Qin et al., 2007. Climate Change 2007: The Physical Science Basis. Contribution 

of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate 

Change., Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.

WWF, 2005. An overview of Glaciers, glacier retreat and its subsequent impacts in Nepal, India and 

China: 67.
Annex C 89

Annex C The error on 

the percentage of Dutch 

land area below sea level

In February 2010, an error was detected in a text supplied by the Netherlands 

Environmental Assessment Agency (PBL) for Chapter 12 (on Europe), page 547, 

Subsection 12.2.3, of the Working Group II contribution to the Fourth Assessment 

Report, regarding the risks of flooding in the Netherlands: 

The Netherlands is an example of a country highly susceptible to both sea-level rise 

and river flooding because 55% of its territory is below sea level where 60% of its 

population lives and 65% of its Gross National Product (GNP) is produced. (E1; 

major)

The error was discovered by a Dutch journalist of a weekly newspaper (Vrij 

Nederland) who checked information about the Netherlands, triggered by the 

publicity around the error on the Himalayan glaciers. On page 547 of the chapter 

text, it says that 55% of the Netherlands is located below sea level. This should 

have read that 55% of the Netherlands is at risk of flooding (E1); 26% of the country 

is at risk because it is located below sea level, and another 29% is susceptible to 

river flooding.1) Examples of the latter are the near floodings in the mid-1990s, 

of areas along the rivers Meuse and Waal – areas that are well above sea level. 

The Netherlands is sensitive to climate change. Sea level rise as well as peak river 

discharges have been analysed frequently, and according to government policy this 

requires precautionary measures. 

The original text that included this error was submitted by the Netherlands 

Environmental Assessment Agency (PBL) in mid 2005. A similar error also occurred 

in publications by others. A commentary that was published in Nature in 2005 

(Kabat et al., 2005) refers to 60% of the Dutch land area as being located below sea 

level. In 2007, an English version of a note from the Dutch Ministry of Transport 

also stated (Dutch Ministery of Transport, 2007) that about 60% of the country was 

1) Here, the area of the Netherlands that is prone to flooding has been divided in a binary manner, starting with determining the part of the country that lies below NAP (a particular reference sea level) – which is 26% – the rest of the country at risk 

of flooding is then categorised as being at risk of flooding by rivers; of course, more finegrained subdivisions could also have 

been made, since obviously the part of the country that lies below sea level is also at risk of flooding by rivers, and parts of the 

country that lie above NAP are also at risk of flooding by the sea.
90 Assessing an IPCC assessment 

situated below sea level. It would have been correct to state that about 60% of 

the Netherlands would be prone to flooding by sea if the sea level would rise by 1 

m above the 2000 level and a severe storm would hit the coast during spring tide 

– but this is not the same thing. In the review process of the IPCC Working Group 

II Report, there were several comments and suggested changes relating to this 

chapter, but this error remained unnoticed.

Immediately after detection, the PBL reported the error to the international 

community. It was also reported to the IPCC, and on 27 April 2010 is was included in 

a list of errata on IPCC reports. The incorrect wording in the IPCC report does not 

affect the message of the conclusion. The error has no significance for the main 

conclusions in the chapter, or for those on Europe, nor for the conclusions in the 

various Summaries for Policymakers.

We acknowledge that this error was not the fault of the IPCC (Coordinating) 

Lead Authors or Co-Chairs. The error was made by a Contributing Author from 

the PBL, and the (Coordinating) Lead Authors are not to blame for relying on 

Dutch information provided by a Dutch agency. The lesson to be learned for an 

assessment agency such as ours, is that quality control is needed at the primary 

level of a literature assessment. This should be done by checking basic data, to 

the extent feasible, even when authoritative references are available. Of course, 

there are limitations to checking data, given the vast amounts of specific regional 

information. 

References

Kabat, P., Vierssen, W. van, Veraart, J.A., Vellinga, P., Aerts, J., 2005. Climate proofing the 

Netherlands. Nature, vol. 438, p. 283-284

Ministry of Transport, Public Works and Water Management, 2007. Flood maps in the Netherlands, 

http://www.safecoast.org/editor/databank/File/folder%20engels%20def%201%20febr07.pdf
Annex D 91

Annex D Sea level rise: 

consequences for the 

Netherlands

Summary

Errors in the IPCC report regarding the Himalayan glaciers and the Dutch land area 

below sea level have triggered a discussion on whether the Dutch policies that deal 

with sea level rise should be revised. The answer is clearly ‘no’: the contribution 

from melting Himalayan glaciers to sea level rise is very limited (around 2%). Moreover, the policy concerning safety against flooding in the Netherlands, as formulated 

in the National Water Policy Plan 2009-2015 (Ministry of Transport, Public Works and 

Water Management, 2009) and elaborated in the Delta Programme, is well founded 

on the available knowledge on climate change, sea level rise, river discharges and 

uncertainties. The current discussions related to the IPCC have no effect on the 

assumptions under the National Water Policy Plan and the Delta Programme.

There is scientific agreement that the main causes of mean global sea level rise 

are thermal expansion of sea water and melting of land ice due to increasing 

temperatures. However, future projections of sea level rise are uncertain because 

of the large range of temperature projections and the limited knowledge about 

the melting behaviour of especially the arctic ice shields (PBL2007, Katsman et al., 

2008). 

Sea level rise: limited contribution from the Himalayan glaciers

Over the last century, the sea levels have risen, world wide, by 14 to 20 cm, corresponding with an average of 1.7±0.3 mm a year (Church et al., 2006). Along the 

Dutch coast, the sea level rise was of the same order of magnitude: 19 cm with an 

average of 1.8±0.2 mm a year (PBL, 2010). Satellite data indicate an accelerated 

trend in global sea level rise of 3.1±0.7 mm a year, over the past 15 years (Church 

and White, 2006; Rahmstorf et al., 2007). It is unclear whether this acceleration 

will continue. The effect of glacier melting in the Himalayas on global sea level 

rise is limited: based on Oerlemans et al. (2007) and Kargel et al. (2010), it is estimated that the annual contribution from the Himalayan glaciers to sea level rise 

is 0.06±0.04 mm, that is, around 2% of the current annual sea level rise of 3.1±0.7 

mm (Parry et al., 2007). Therefore can be concluded that the contribution from the 

melting of the Himalayan glaciers to sea level rise is very limited.
92 Assessing an IPCC assessment 

Estimates on future sea level rise for the Netherlands: large uncertainties

There is scientific agreement that the main causes of mean global sea level rise 

are thermal expansion of sea water and the melting of land ice due to increasing 

temperatures. However, future projections of sea level rise are uncertain because 

of the large range of temperature projections and the limited knowledge about 

the melting behaviour of especially the arctic ice shields (PBL, 2007; Katsman et 

al., 2008). The range for temperature rise is due to i) natural variability, such as the 

recent slowing down of the increase in average global temperature, ii) the climate 

sensitivity of the atmosphere, iii) uncertainties in atmospheric processes, and iv) 

uncertainties in future emission of greenhouse gases.

When considering the possible sea level rise for the Netherlands, the most recent 

estimates on sea level rise for the Netherlands cover a range of 35 to 85 cm for 2100 

(KNMI, 2006), or in case of high-end/worst-case estimates, the rise is between 130 

cm (Deltacommissie, 2008) and 150 cm (PBL, 2007). 

In 2007, in collaboration with the Royal Netherlands Meteorological Institute 

(KNMI), the PBL explored plausible future extreme sea level rise scenarios for the 

Netherlands and – based on paleoclimatological data – estimated a worst-case sea 

level rise of 1.5 m/century (PBL, 2007). Given the technical adaptive capacity of the 

Netherlands and the considered safety margins, the PBL concluded that, with the 

available techniques, the delta region of the Netherlands could be kept safe even 

in case of such an extreme sea level rise, but that in the long term, spatial measures 

could be required (PBL, 2007). In 2008, the Dutch Delta committee (Deltacommissie) presented a high-end estimate on sea level rise of between 65 and 130 cm for 

2100, based on a temperature-rise scenario of 6 degrees, and also concluded that 

technically the delta region of the Netherlands could protect itself against flooding (Deltacommissie 2008). The maximum ranges for sea level rise given by the 

IPCC (2007) and the KNMI (2006), were based on scenarios with a maximum of 4 

degrees temperature rise. A 6-degree temperature rise thus exceeds the 4-degree 

scenario of the IPCC, but still lies within the total range of uncertainty with respect 

to temperature rise (e.g. PBL, 2009, News in Climate Science). 

Consequences for policy on safety against flooding 

Following the advice of the Delta committee in 2008, the policy on safety against 

flooding and freshwater management will be revised, given the increases in population and economical values in the area that has been sensitive to flooding since 

the 1960s, and the challenges imposed by climate change. The recent National 

Water Policy Plan 2009-2015 (Dutch Ministry of Transport, Public Works and Water 

Management, 2009) has announced that i) the national standards for the safety 

against flooding will be reconsidered, and ii) a long-term adaptive strategy on 

climate proofing of the Netherlands will be constructed, including an analysis of 

possible consequences for spatial development. To effectuate these points, the 

Delta Programme has been launched, together with a Delta Act, Delta Fund and 

a Delta Commissioner responsible for the coordination of the programme. In this 

Delta Programme, the four available KNMI scenarios (KNMI, 2006) are taken as the 

plausible range for future uncertainties for the Netherlands, with respect to climate 

change and sea level rise. 
Annex D 93

Current investments in the protection against flooding in the Netherlands are 

justified by the fact that a substantial number of dikes (24%) did not meet presentday safety standards because of insufficient maintenance and new hydrological 

conditions. The investments that are required for the improvement of the dikes 

are not fully known yet, because for 32% of the dikes, the quality status is currently 

unknown (Dutch Ministry of Transport, Public Works and Water Management, 

2006).

Given that sea level rise is a slow process and that the uncertainties in future projections are considerable, future additional investments in the safety of the Netherlands will depend on progressing scientific insights. Sea level rise is addressed in 

the current policy on safety against flooding. For safety measures in coastal areas, 

decisions with a short lead time (5 years) are based on the present sea level rise of 

20 cm/century and – based on earlier KNMI scenarios – decisions with lead times of 

50 to 100 years are based on 60 cm sea level rise. Spatial reservations for the long 

term (100 to 200 years) are based on a high sea level rise of 85 cm. Possible impacts 

of climate change from floods due to extreme river discharges have also been 

incorporated in the management strategy ‘Room for the River’ through spatial 

reservations in certain areas for which additional measures could be required in the 

future. Finally, the process of regular evaluations of the quality of the dikes, every 

six years, will include the advancing knowledge about climate change, sea level 

rise and its consequences (Dutch Ministry of Transport, Public Works and Water 

Management, 2009). 

An additional, important factor that will determine river discharges in the Netherlands is the river and safety management upstream in the Rhine and Meuse river 

basins. Since 2009, the EU High Water Directive has been operational. It provides a 

legal basis for international appointments in flood risk management on the scale of 

transboundary river basins. 

Considering the slow process of sea level rise, the adaptive capacity of the Netherlands with respect to technical measures (e.g., strengthening of dikes, storm barriers, dunes) is thought to be sufficient – even in worst-case scenarios – provided 

that the required budgets become available (PBL, 2007; Deltacommissie, 2008). 

However, the study by the PBL (2007) and Ligtvoet et al. (2009) point out that, in 

the long run, management of peak river discharges will be the main challenge for 

the Netherlands, and that adaptive capacity concerning spatial developments is 

limited, because of their relative slowness and irreversibility. Therefore, choices 

that will be made in the coming decades, in urban development, nature development and configuration of the main water system in the Netherlands, will also 

determine the future climate resilience of the Netherlands (PBL, 2009). It has been 

shown that a combination of spatial and technical measures regarding river areas, 

IJsselmeer area and the south-western delta, may contribute most to reducing the 

vulnerability of the Netherlands, in both the short term and the long term (Ligtvoet 

et al., 2009).

Conclusion

The current policy on safety against flooding in the Netherlands, as formulated 

in the National Water Policy Plan and elaborated in the Delta Programme, is well 
94 Assessing an IPCC assessment 

founded on the available knowledge on climate change, sea level rise, river discharges and uncertainties. The current discussions about the IPCC have no effect on 

the assumptions underlying the Dutch National Water Plan. 

The required investments in the period up to 2020 (although not fully known yet) 

are to ensure that the quality of the dikes will meet present-day standards (Water 

Act). The required future maintenance and investment levels will depend on the 

new safety standards and the measures that will be adopted within the framework of a long-term adaptive strategy. The recently launched Delta Programme 

within the framework of the National Water Policy Plan 2009-2015 (Dutch Ministry 

of Transport, Public Works and Water Management, 2009) contains an extensive 

research agenda to focus future research on safety against flooding and on freshwater supply in the Netherlands. In addition, the agenda lists the drafting of a longterm adaptive strategy, addressing the still large uncertainties concerning climate 

change. For 2015, further policy decisions are expected on extra measures and 

investments for safety against flooding and for freshwater supply. On a political 

level, it has been decided that within the context of the Delta Programme, by 2020, 

financial reservations will be made for additional investments and maintenance.

References

Church, J.A., and N.J. White, 2006. A 20th century acceleration in global sea-level rise, Geophys. 

Res. Lett., 33, L01602, doi:10.1029/2005GL024826. Update data to 2007 on PSMSL website (link to 

data)

Deltacommissie, 2008. Samen werken aan water. Bevindingen van de Deltacommissie. 

Deltacommissie, Den Haag, the Netherlands. 

Kargel, J.S., R. Armstrong et al., 2010. Satellite-era glacier changes in High Asia.Background support 

presentation for NASA “Black Carbon and Aerosols” press conference associated with Fall AGU, 

Dec. 14, 2009. Updated and expanded Feb. 17, 2010.

Katsman, C.A., W.Hazeleger, S.S. Drijfhout, G.J. van Oldenborgh & G. Burgers, 2008. Climate 

scenarios of sea level rise for the northeast Atlantic Ocean: a study including the effects of ocean 

dynamics and gravity changes induced by ice melt, Climatic Change 91:351–374

KNMI, 2006. Climate change scenarios for the Netherlands. Scientific report WR 2006-01. Koninklijk 

Nederlands Meteorologisch Institute, Bilthoven, the Netherlands.

Ligtvoet, W.J. Knoop, B. Strengers & A. Bouwman, 2009. Flood protection in the Netherlands: 

framing long term challenges and options for a climate resilient delta. PBL publication number 

500078004. Netherlands Environmental Assessment Agency – PBL, Bilthoven/Den Haag, the 

Netherlands.

Ministry of Transport, Public Works, and Water management, 2006. Primaire Waterkeringen 

getoetst, Landelijke rapportage Toetsing 2006. Ministry of Public Works, Transport and 

Watermanagement, Den Haag, the Netherlands

Ministry of Transport, Public Works, and Water management, 2009. Nationaal Waterplan 2009-2015. 

Ministry of Public Works, Transport and Water management, Den Haag, the Netherlands.

Oerlemans, J., M. Dyurgerov et al., 2007. "Reconstructing the glacier contribution to sea-level rise 

back to 1850." The Cryosphere 1: 59-65.

Parry, M. L., O.F. Canziani et al., 2007. Climate Change 2007: Impacts, Adaptation and Vulnerability. 

Cambridge, Cambridge University Press.

PBL, 2007. The Netherlands in the future. Second Sustainability Outlook - part Physical Environment 

of the Netherlands. Netherlands Environmental Assessment Agency – PBL, Bilthoven/Den Haag, 

The Netherlands
Annex D 95

PBL, 2009. Roadmap to a climate-proof Netherlands. PBL publication number 500078003. 

Netherlands Environmental Assessment Agency – PBL, Bilthoven/Den Haag, The Netherlands

PBL, 2009. News in Climate Science. Netherlands Environmental Assessment Agency – PBL, 

Bilthoven/Den Haag, The Netherlands.

PBL, 2010. Zeespiegelstand langs de Nederlandse kust en mondiaal, 1891-2008. Versie 7. 

Compendium voor de Leefomgeving. http://www.compendiumvoordeleefomgeving.nl/

indicatoren/nl022907-Zeespiegelstand-Nederland.html?i=9-54. Netherlands Environmental 

Assessment Agency – PBL, Bilthoven/Den Haag. Statistics Netherlands, Den Haag/Heerlen, 

Wageningen UR, Wageningen, The Netherlands . 

Rahmstorf, S., 2007. A semi-empirical approach to projecting future sea-level rise. Science315: 367-

370
96 Assessing an IPCC assessment 

Annex E Abbreviations

ASTER

Advanced Spaceborne Thermal Emission and 

Reflection Radiometer (a remote sensing 

instrument)

CLA 

Coordinating Lead Author

CO2 

Carbon dioxide

CRU 

Climate Research Unit (at the University of East 

Anglia, United Kingdom)

EHP 

Elevated Heat Pump

FAO 

Food and Agriculture Organization

FOD 

First Order Draft

GCM 

Global Circulation Model (a type of climate 

model)

GDP 

Gross Domestic Product

GNP 

Gross National Product

H-K 

Himalaya-Karakoram

IPCC 

Intergovernmental Panel on Climate Change

IUCN 

International Union for Conservation of Nature

KNAW 

Royal Netherlands Academy of Arts and 

Sciences

KNMI 

Royal Netherlands Meteorological Institute

NAP

Amsterdam Ordnance Datum

NCAR 

National Center for Atmospheric Research 

(Boulder, USA)

NGO 

Non Governmental Organisation

SOD 

Second Order Draft

SPM 

Summary for Policymakers

SRES 

IPCC Special Report on Emission Scenarios

SYR 

IPCC Synthesis Report (of Working Groups I, II 

and III reports)

TAR 

Third Assessment Report of the IPCC

TS

Technical Summary

UN

United Nations

UNFCCC

United Nations Framework Convention on 

Climate Change

UNMSM

Universidad Nacional Mayor de San Marcos 

(Lima, Peru)

URL

Uniform Resource Locator (e.g. an internet 

address)

WG

Working Group

WG I

IPCC Working Group I (on the Science of Climate 

Change)

WG II

IPCC Working Group II (on Impacts, Adaptation 

and Vulnerability)
Annex E 97

WG III

IPCC Working Group III (on Mitigation of Climate 

Change)

WWF

World Wide Fund for Nature
98 Assessing an IPCC assessment 
Colophon 99

Colophon 

Responsibility

Netherlands Environmental Assessment Agency

Editors

L.A. Meyer, A.C. Petersen

Steering group

M.A. Hajer, J.A. Oude Lohuis, L.A. Meyer, A.C. Petersen, G. de Lange, J. Timmerhuis, 

E. Buitelaar 

Research

J.R.M. Alkemade, M. Bakkenes, G.J. van den Born, A.F. Bouwman, J.J.M. van Grinsven, 

H.B.M. Hilderink, P.H.M. Janssen, W. Ligtvoet, A.J.G. Manders, J.G. van Minnen, B.J. Strengers, 

H. Visser, M. Vonk, M. de Vos (PBL), R.J. Swart (WUR)

Translation and editing

A. Righart (PBL), C. Ettema (Shades of Green - Natural Translations)

Other contributors

F.S. de Blois, C. Campman, J.H.J. Dolmans, S.A. van Esch, M. Hartman, A.M. Oosterhuis, 

J.F. de Ruiter, C. Spelbos 

Design and layout

Uitgeverij RIVM

Corresponding Author

L. Meyer, leo.meyer@pbl.nl 
snijlijn 

Hier komt bij voorkeur een flaptekst te staan, in plaats 

van deze algemene tekst

Het Planbureau voor de Leefomgeving is hét nationale 

instituut voor strategische beleidsanalyses op het gebied 

van milieu, natuur en ruimte. 

Het PBL draagt bij aan de kwaliteit van het strategische overheidsbeleid door een brug te vormen tussen 

wetenschap en beleid en door gevraagd en ongevraagd, 

onafhankelijk en wetenschappelijk gefundeerd, verkenningen, analyses en evaluaties te verrichten waarbij een 

integrale benadering voorop staat. 

Planbureau voor de Leefomgeving

Locatie Bilthoven

Postbus 303

3720 AH Bilthoven

T: 030 274 274 5

F: 030 274 4479

E: info@pbl.nl

www.pbl.nl

Planbureau voor de Leefomgeving, <maand> 200X

Assessing an IPCC assessment

The Netherlands Environmental Assessment Agency (PBL) has investigated the scientific foundations for the IPCC summary conclusions 

of the Fourth Assessment Report of 2007 on projected regional 

climate-change impacts, at the request of the Dutch Minister for the 

Environment. Overall the summary conclusions are considered well 

founded and none were found to contain any significant errors. 

The Working Group II contribution to the Fourth Assessment Report 

shows ample observational evidence of regional climate-change 

impacts, which have been projected to pose substantial risks to most 

parts of the world, under increasing temperatures. However, in some 

instances the foundations for the summary statements should have 

been made more transparent. While acknowledging the essential role 

of expert judgment in scientific assessments, the PBL recommends to 

improve the transparency of these judgments in future IPCC reports. 

In addition, the investigated summary conclusions tend to single out 

the most important negative impacts of climate change. Although 

this approach was agreed to by the IPCC governments for the Fourth 

Assessment Report, the PBL recommends that the full spectrum of 

regional impacts is summarised for the Fifth Assessment Report, 

including the uncertainties. The PBL believes that the IPCC should 

invest more in quality control in order to prevent mistakes and shortcomings, to the extent possible.

Netherlands Environmental Assessment Agency, July 2010
The Independent 

Climate Change 

E-mails Review 

July 2010

Chair: Sir Muir Russell

Review team:

Professor Geoffrey Boulton

Professor Peter Clarke

David Eyton

Professor James Norton
2

INDEX OF CONTENTS 

 

GLOSSARY 8

Chapter 1 EXECUTIVE SUMMARY 10

1.1 Introduction 10

1.2 The Review Process 11

1.3 Findings 11

1.3.1 Land Station Temperatures 12

1.3.2 Temperature Reconstructions from Tree Ring Analysis 12

1.3.3 Peer Review and Editorial Policy 13

1.3.4 Misuse of IPCC Process 13

1.3.5 Compliance with the Freedom of Information Act (FoIA) and the 

Environmental Information Regulations (EIR)

14

1.3.6 Other Findings on Governance 14

1.4 Recommendations 14

1.5 Broader Issues 14

Chapter 2 INTRODUCTION 18

2.1 Background 18

2.2 The Review 19

Chapter 3 TERMS OF REFERENCE AND METHOD OF ENQUIRY 22

3.1 Terms of Reference 22

3.2 Method of Enquiry 22

Chapter 4 CONTEXT OF THE E-MAILS 26

4.1 Characterising the E-mails 26

4.2 The Timeline 27

4.3 What the E-mails Tell Us 32

Chapter 5 THE CHANGING CONTEXT OF MODERN SCIENCE 36

5.1 The Scientific Process 36

5.2 The Nature of Climate Science 36

5.3 Data Accessibility in the Digital Age 37

5.4 Handling Uncertainty 38

5.5 Scientific Journals and the Peer Review Process 39

5.6 The Responsibilities of Scientists in Communicating in the Public Domain 40

5.7 Communicating to Policymakers 41

5.8 The Changing Forum for Debate and the Blogosphere 41

Chapter 6 LAND STATION INSTRUMENTAL TEMPERATURE DATA 44

6.1 Background 44

6.2 The Allegations 44

6.3 The Approach Adopted by the Review Team 45

6.4 The Results of the Analysis 46

6.5 Checking Specific Details in the CRUTEM Analysis 49

6.5.1 Identification of Data Sources 49

6.5.2 The Availability of Computer Codes 49

6.6 Use of Local temperature Data from China

6.7 Conclusions and Recommendations

51

53
3
4

Chapter 7 TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 54

7.1 Background 54

7.2 The Allegations 54

7.3 Findings 56

7.3.1 IPPC Reports 56

7.3.2 Divergence 59

7.3.3 Withholding Data 60

7.3.4 Mishandling Data 61

7.4 Conclusions and Recommendations 62

Chapter 8 PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC 

JOURNALS

64

8.1 Background: Peer Review, Testing and Verification 64

8.2 The Allegations 64

8.3 The Soon and Baliunas Affair & Climate Research 64

8.4 The Conflict with Dr Boehmer-Christiansen 66

8.5 Peer Review and Professor Briffa‘s Editorship of Holocene 67

8.6 Conclusions 68

Chapter 9 COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 70

9.1 Background 70

9.2 The Allegations 70

9.3 The CRUTEM Temperature Series 71

9.3.1 The Scientific Challenge 71

9.3.2 The Allegations 71

9.3.3 Evidence in Support of the Allegations 71

9.3.4 Jones‘ Response 73

9.3.5 Evidence from IPCC Review Editor for Chapter 3 (Professor Sir Brian 

Hoskins)

74

9.3.6 Findings 75

9.4 The Tree Ring Proxy Temperature Series 77

9.4.1 The Scientific Challenge 77

9.4.2 The Allegations 77

9.4.3 Evidence in Support of the Allegations 78

9.4.4 Responses from Briffa 80

9.4.5 Evidence from IPCC Review Editor for Chapter 6 (Professor John 

Mitchell)

82

9.4.6 Findings 83

9.5 Conclusions 84

Chapter 10 COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION

REGULATIONS

86

10.1 Introduction and Method of Enquiry 86

10.2 The Allegations 86

10.3 General Context 86

10.4 Investigation 89

10.5 Findings 91

10.6 Recommendations 94
5
6

Chapter 11 GOVERNANCE 98

11.1 Introduction and Method of Enquiry 98

11.2 Research Management Systems 98

11.2.1 Background 98

11.2.2 Funding Management 99

11.2.3 Funders‘ Requirements 99

11.2.4 Good Research Practice 99

11.2.5 Financial Controls 100

11.2.6 Risk Management 100

11.2.7 Findings on Research Management Systems 100

11.3 Software, Data Management and Data Security 101

11.3.1 General Context 101

11.3.2 Issues and the Investigation 102

11.3.3 Findings on Software, Data Management and Data Security 103

11.4 Recommendations 103

ACKNOWLEDGEMENTS 106

APPENDIX 1: REVIEW TEAM MEMBERS 108

APPENDIX 2: APPROACH AND WORK PLAN 110

APPENDIX 3: ISSUES FOR EXAMINATION 112

APPENDIX 4: INDEX OF MEETINGS, INTERVIEWS, SUBMISSIONS, 120

FOLLOW UP ENQUIRIES AND RESPONSES

APPENDIX 5: PEER REVIEW 126

APPENDIX 6: DATA MINING – ACCESS TO THE ORIGINAL CRU E-MAIL 146

ARCHIVE

APPENDIX 7: LAND STATION TEMPERATURE DATA 150

APPENDIX 8: SOFTWARE AND DATA STANDARDS 160
7
8

GLOSSARY 

AR4 Fourth Assessment Report of the Intergovernmental Panel on 

Climate Change in 2007

CLA Coordinating Lead Author

COP Conference of the Parties

COPE Committee on Publication Ethics

CRU Climatic Research Unit

CRUTEMX Land air temperature anomalies on a 5° by 5° grid-box basis, 

version X

DEFRA UK Government Department for Environment, Food and Rural 

Affairs

DPA Data Protection Act 1998

E&E Energy and Environment

EC European Community

EIR Environmental Information Regulations

ENSO El Niño-Southern Oscillation

ENV University of East Anglia School of Environmental Sciences

FOI Freedom of Information

FoIA Freedom of Information Act 2000

GHCN Global Historical Climatology Network

GISS Goddard Institute for Space Studies

GISTEMP Goddard Institute for Space Studies Surface Temperature Analysis

GWPF Global Warming Policy Foundation

HadCRUTX Combined land and marine temperature anomalies on a 5° by 5° 

grid-box basis, version X

IAC Inter Academy Council

ICCER Independent Climate Change E-Mails Review

ICO Information Commissioner‘s Office

ICT Information Communications Technology

IDL Interactive Data Language

IEC International Electrotechnical Commission

IPCC Intergovernmental Panel on Climate Change

IPCM Information Policy and Compliance Manager

IS Information Systems

ISSC Information Systems Strategy Committee

IT Information Technology

JANET The United Kingdom‘s Education and Research Network

JISC Joint Information Systems Committee

LA Lead Authors

LIA Little Ice Age

MBH Mann, Bradley and Hughes

MM McKitrick and Michaels
9

M&M McIntyre and McKitrick

MWP Medieval Warm Period

NAO North Atlantic Oscillation

NASA National Aeronautics and Space Administration

NCAR National Center for Atmospheric Research

NCDC National Climatic Data Center

NMO National Meteorological Office

NOAA National Oceanic and Atmospheric Administration

PERL Practical Extraction and Reporting Language

PI Principal Investigator

REE Research, Enterprise and Engagement

S&B Soon and Baliunas

SAP Scientific Assessment Panel

SPM Summary for Policy Makers

TAR Third Assessment Report of the IPCC in 2001

UEA University of East Anglia

UHI Urban Heat Island

UKRIO United Kingdom Research Integrity Office

UN United Nations

UNEP United Nations Environment Programme

WMO World Meteorological Organization

WWR World Weather Records
CHAPTER 1: EXECUTIVE SUMMARY 

10

CHAPTER 1: EXECUTIVE SUMMARY 

1. The main findings of the Independent Climate Change E-mails Review (―the 

Review‖) are set out in Section 1.3 below, and the main recommendations in 

Section 1.4. We comment in Section 1.5 on some of the more general issues 

raised by the Review that we think are important about the context in which 

scientists operate and in which science contributes to public policy. 

1.1 Introduction 

2. In November 2009, approximately 1000 e-mails from the Climatic Research Unit 

(CRU) of the University of East Anglia (UEA) were made public without 

authorisation. 

3. CRU is a small research unit which over the last 30 years has played an important 

role in the development of climate science, in particular in their work on 

developing global temperature trends. 

4. The e-mails fuelled challenges to the work of CRU, to the reliability of climate 

science generally, and to the conclusions of the Intergovernmental Panel on 

Climate Change (IPCC). All this happened shortly before the Copenhagen 

Summit, and was extensively referred to there. 

5. In response, the UEA commissioned two inquiries. The first led by Lord Oxburgh, 

into the science being undertaken at CRU, has already reported. This document is 

the report of the second inquiry – The Independent Climate Change E-mails 

Review – which examines the conduct of the scientists involved and makes 

recommendations to the University of East Anglia. Our inquiry addresses a 

number of important allegations that were made following the e-mail release. 

6. The allegations relate to aspects of the behaviour of the CRU scientists, such as 

their handling and release of data, their approach to peer review, and their role in 

the public presentation of results. 

7. The allegations also include the assertion that actions were taken to promote a 

particular view of climate change by improperly influencing the process of 

advising policy makers. Therefore we have sought to understand the significance 

of the roles played by those involved from CRU and of the influence they had on 

the relevant outcomes. 

8. The Review examines the honesty, rigour and openness with which the CRU 

scientists have acted. It is important to note that we offer no opinion on the 

validity of their scientific work. Such an outcome could only come through the 

normal processes of scientific debate and not from the examination of e-mails or 

from a series of interviews about conduct.
CHAPTER 1: EXECUTIVE SUMMARY 

11

1.2 The Review Process

9. The approach taken by the Review was to identify and investigate the allegations 

to which the e-mails gave rise. This reflected our reading of the emails and the 

comments made on them. An online consultation was undertaken to ensure that 

the Team‘s initial analysis of the allegations and concerns was sound. The method 

of investigation is explained in the relevant Chapters and Appendices to the 

report. The Review‘s evidence base is published on the website, which it intends 

to archive. 

10. In addressing the allegations about CRU‘s impact on climate science, we sought 

evidence to place these into perspective: 

On handling global temperature data, we went to global primary sources and 

tested how data was handled.

On tree-ring temperature reconstructions, we looked at the overall picture 

painted in Chapter 6 of the Fourth Assessment Report of the IPCC in 2007 

(AR4) and examined the influence of CRU.

On peer review, we sought independent input (from the Editor of The Lancet) 

on how the system works, to provide a context for our judgement.

On influencing the IPCC process, we sought advice from the Review editors 

on the role individual contributors can play.

11. This work provided a context in which we considered the evidence about the 

specific allegations made in the submissions and identified in our interviews with 

CRU and others. 

12. Reflecting this approach, the report and conclusions are set out as follows. The 

heart of the report lies in Chapters 6 through 10 where the important allegations 

arising from the e-mail release are examined. Chapters 2 and 3 contain 

introductory material, 4 deals with the body of e-mails and 5 presents important 

contextual material. The report concludes with Chapter 11 on other governance 

issues. 

1.3 Findings 

13. Climate science is a matter of such global importance, that the highest standards 

of honesty, rigour and openness are needed in its conduct. On the specific 

allegations made against the behaviour of CRU scientists, we find that their 

rigour and honesty as scientists are not in doubt. 

14. In addition, we do not find that their behaviour has prejudiced the balance of 

advice given to policy makers. In particular, we did not find any evidence of 

behaviour that might undermine the conclusions of the IPCC assessments.

15. But we do find that there has been a consistent pattern of failing to display 

the proper degree of openness, both on the part of the CRU scientists and on the 

part of the UEA, who failed to recognise not only the significance of statutory 
CHAPTER 1: EXECUTIVE SUMMARY 

12

requirements but also the risk to the reputation of the University and, indeed, to 

the credibility of UK climate science. 

1.3.1 Land Station Temperatures 

16. On the allegation of withholding temperature data, we find that CRU was not 

in a position to withhold access to such data or tamper with it. We 

demonstrated that any independent researcher can download station data directly 

from primary sources and undertake their own temperature trend analysis. 

17. On the allegation of biased station selection and analysis, we find no evidence 

of bias. Our work indicates that analysis of global land temperature trends is 

robust to a range of station selections and to the use of adjusted or unadjusted 

data. The level of agreement between independent analyses is such that it is highly 

unlikely that CRU could have acted improperly to reach a predetermined outcome. 

Such action would have required collusion with multiple scientists in various 

independent organisations which we consider highly improbable. 

18. On the allegation of withholding station identifiers we find that CRU should 

have made available an unambiguous list of the stations used in each of the 

versions of the Climatic Research Unit Land Temperature Record 

(CRUTEM) at the time of publication. We find that CRU‟s responses to 

reasonable requests for information were unhelpful and defensive.

19. The overall implication of the allegations was to cast doubt on the extent to 

which CRU‟s work in this area could be trusted and should be relied upon 

and we find no evidence to support that implication. 

1.3.2 Temperature Reconstructions from Tree Ring Analysis 

20. The central implication of the allegations here is that in carrying out their work, 

both in the choices they made of data and the way in which it was handled, CRU 

scientists intended to bias the scientific conclusions towards a specific result and 

to set aside inconvenient evidence. More specifically, it was implied in the 

allegations that this should reduce the confidence ascribed to the conclusions in 

Chapter 6 of the IPCC 4th Report, Working Group 1 (WG1). 

21. We do not find that the way that data derived from tree rings is described 

and presented in IPCC AR4 and shown in its Figure 6.10 is misleading. In 

particular, on the question of the composition of temperature reconstructions, we 

found no evidence of exclusion of other published temperature reconstructions 

that would show a very different picture. The general discussion of sources of 

uncertainty in the text is extensive, including reference to divergence. In this 

respect it represented a significant advance on the IPCC Third Assessment 

Report (TAR). 

22. On the allegation that the phenomenon of “divergence” may not have been 

properly taken into account when expressing the uncertainty associated 

with reconstructions, we are satisfied that it is not hidden and that the 
CHAPTER 1: EXECUTIVE SUMMARY 

13

subject is openly and extensively discussed in the literature, including CRU 

papers. 

23. On the allegation that the references in a specific e-mail to a „trick‟ and to 

„hide the decline‟ in respect of a 1999 WMO report figure show evidence of 

intent to paint a misleading picture, we find that, given its subsequent iconic 

significance (not least the use of a similar figure in the IPCC Third 

Assessment Report), the figure supplied for the WMO Report was 

misleading. We do not find that it is misleading to curtail reconstructions at 

some point per se, or to splice data, but we believe that both of these procedures 

should have been made plain – ideally in the figure but certainly clearly 

described in either the caption or the text. 

24. On the allegations in relation to withholding data, in particular concerning 

the small sample size of the tree ring data from the Yamal peninsula, CRU 

did not withhold the underlying raw data (having correctly directed the 

single request to the owners). But it is evidently true that access to the raw 

data was not simple until it was archived in 2009 and that this delay can rightly 

be criticized on general principles. In the interests of transparency, we believe 

that CRU should have ensured that the data they did not own, but on which their 

publications relied, was archived in a more timely way. 

1.3.3 Peer Review and Editorial Policy

25. On the allegations that there was subversion of the peer review or editorial 

process we find no evidence to substantiate this in the three instances 

examined in detail. On the basis of the independent work we commissioned 

(see Appendix 5) on the nature of peer review, we conclude that it is not 

uncommon for strongly opposed and robustly expressed positions to be taken up 

in heavily contested areas of science. We take the view that such behaviour does 

not in general threaten the integrity of peer review or publication. 

1.3.4 Misuse of IPCC Process 

26. On the allegations that in two specific cases there had been a misuse by 

CRU scientists of the IPCC process, in presenting AR4 to the public and 

policy makers, we find that the allegations cannot be upheld. In addition to 

taking evidence from them and checking the relevant records of the IPCC 

process, we have consulted the relevant IPCC review Editors. Both the CRU 

scientists were part of large groups of scientists taking joint responsibility for the 

relevant IPCC Working Group texts, and were not in a position to determine 

individually the final wording and content. 
CHAPTER 1: EXECUTIVE SUMMARY 

14

1.3.5 Compliance with the Freedom of Information Act 

(FoIA) and the Environmental Information Regulations 

(EIR) 

27. On the allegation that CRU does not appear to have acted in a way 

consistent with the spirit and intent of the FoIA or EIR, we find that there 

was unhelpfulness in responding to requests and evidence that e-mails 

might have been deleted in order to make them unavailable should a 

subsequent request be made for them. University senior management should 

have accepted more responsibility for implementing the required processes for 

FoIA and EIR compliance. 

1.3.6 Other Findings on Governance

28. Given the significance of the work of CRU, UEA management failed to 

recognise in their risk management the potential for damage to the 

University‟s reputation fuelled by the controversy over data access. 

1.4 Recommendations 

29. Our main recommendations for UEA are as follows: 

Risk management processes should be directed to ensuring top management 

engagement in areas which have the potential to impact the reputation of the 

university. 

Compliance with FoIA/EIR is the responsibility of UEA faculty leadership 

and ultimately the Vice-Chancellor. Where there is an organisation and 

documented system in place to handle information requests, this needs to be 

owned, supported and reinforced by University leadership. 

CRU should make available sufficient information, concurrent with any 

publications, to enable others to replicate their results. 

1.5 Broader Issues

30. Our work in conducting the Review has led us to identify a number of issues 

relevant not only to the climate science debate but also possibly more widely, on 

which we wish to comment briefly. 

31. The nature of scientific challenge. We note that much of the challenge to 

CRU‘s work has not always followed the conventional scientific method of 

checking and seeking to falsify conclusions or offering alternative hypotheses 

for peer review and publication. We believe this is necessary if science is to 

move on, and we hope that all those involved on all sides of the climate science 

debate will adopt this approach. 

32. Handling Uncertainty – where policy meets science. Climate science is an 

area that exemplifies the importance of ensuring that policy makers –
CHAPTER 1: EXECUTIVE SUMMARY 

15

particularly Governments and their advisers, Non-Governmental Organisations 

and other lobbyists – understand the limits on what scientists can say and with 

what degree of confidence. Statistical and other techniques for explaining 

uncertainty have developed greatly in recent years, and it is essential that they 

are properly deployed. But equally important is the need for alternative 

viewpoints to be recognized in policy presentations, with a robust assessment of 

their validity, and for the challenges to be rooted in science rather than rhetoric. 

33. Peer review - what it can/cannot deliver. We believe that peer review is an 

essential part of the process of judging scientific work, but it should not be overrated as a guarantee of the validity of individual pieces of research, and the 

significance of challenge to individual publication decisions should be not 

exaggerated. 

34. Openness and FoIA. We support the spirit of openness enshrined in the FoIA 

and the EIR. It is unfortunate that this was not embraced by UEA, and we make 

recommendations about that. A well thought through publication scheme would 

remove much potential for disruption by the submission of multiple requests for 

information. But at the level of public policy there is need for further thinking 

about the competing arguments for the timing of full disclosure of research data 

and associated computer codes etc, as against considerations of confidentiality 

during the conduct of research. There is much scope for unintended 

consequences that could hamper research: US experience is instructive. We 

recommend that the ICO should initiate a debate on these wider issues. 

35. Handling the blogosphere and non traditional scientific dialogue. One of the 

most obvious features of the climate change debate is the influence of the 

blogosphere. This provides an opportunity for unmoderated comment to stand 

alongside peer reviewed publications; for presentations or lectures at learned 

conferences to be challenged without inhibition; and for highly personalized 

critiques of individuals and their work to be promulgated without hindrance. 

This is a fact of life, and it would be foolish to challenge its existence. The 

Review team would simply urge all scientists to learn to communicate their work 

in ways that the public can access and understand. That said, a key issue is how 

scientists should be supported to explain their position, and how a public space 

can be created where these debates can be conducted on appropriate terms, 

where what is and is not uncertain can be recognised. 

36. Openness and Reputation. An important feature of the blogosphere is the 

extent to which it demands openness and access to data. A failure to recognise 

this and to act appropriately, can lead to immense reputational damage by 

feeding allegations of cover up. Being part of a like minded group may provide 

no defence. Like it or not, this indicates a transformation in the way science has 

to be conducted in this century. 

37. Role of Research Sponsors. One of the issues facing the Review was the 

release of data. At various points in the report we have commented on the 

formal requirements for this. We consider that it would make for clarity for 

researchers if funders were to be completely clear upfront in their requirements 

for the release of data (as well as its archiving, curation etc). 
CHAPTER 1: EXECUTIVE SUMMARY 

16

38. The IPCC. We welcome the IPCC‘s decision to review its processes, and can 

only stress the importance of capturing the range of viewpoints and reflecting 

appropriately the statistical uncertainties surrounding the data it assesses. Our 

conclusions do not make a judgement on the work of IPCC, though we 

acknowledge the importance of its advice to policy makers. 
CHAPTER 1: EXECUTIVE SUMMARY 

17
CHAPTER 2: INTRODUCTION 

18

CHAPTER 2: INTRODUCTION 

2.1 Background 

1. Prior to the 1960‘s there had been little investigation of past climatic changes and 

variability, except by geologists and botanists and on geological timescales. The 

Climatic Research Unit (CRU) in the UEA, established in 1972, was one of the 

first institutes in the world to address the science of climate change on more recent 

timescales. Its objective was ―to establish the past record of climate over as much 

of the world as possible, as far back in time as was feasible, and in enough detail 

to recognise the basic processes, interactions, and evolutions in the earth‘s fluid 

envelopes and those involving the earth‘s crust and its vegetation cover‖. A useful 

history of its work is available on its website: 

http://www.cru.uea.ac.uk/cru/about/history. 

2. CRU is a small organisation: at present there are around 16 staff, with 3.5 

established posts and the rest postgraduate students and post-doctoral researchers. 

Two of the main areas of its work are the focus of this review: the development of 

methodologies for calculating the extent to which the average temperature of the 

earth‘s land masses is changing, using instrumental temperature measurements 

made over the past 160 years (the period for which reliable measurements have 

been available); and the estimation of the earth‘s temperature over the last 

millennium, using tree ring data as a proxy for temperature. 

3. One of CRU‘s most important contributions to climate science is the production of 

a land based, gridded temperature data set showing how the temperature has 

varied year by year since 1850 relative to the 1961 to 1990 average. This work 

was started in 1978 and continues today. CRU is now one of a number of 

organisations working in this area. Others carrying out similar development of 

temperature records include the Goddard Institute for Space Studies (GISS) and 

the National Climatic Data Center (NCDC). 

4. Over the period that CRU has been in existence and especially over the last 20 

years, there has been a transformation in the importance attached to climate 

science. One clear indication of this was the creation of the Intergovernmental 

Panel on Climate Change (IPCC) in 1988, a body established by the United 

Nations Environment Programme (UNEP) and the World Meteorological 

Organization (WMO) ―to provide the world with a clear scientific view on the 

current state of climate change and its potential environmental and socioeconomic consequences‖. It assesses the most recent scientific, technical and 

socio-economic information produced worldwide that is relevant to the 

understanding of climate change. Members of CRU have played several 

significant roles on the IPCC. In its successive assessment reports the IPCC has 

sought to achieve a scientific consensus, but many continue to challenge the basis 

of its work and its conclusions. 

5. The pioneering work conducted by CRU therefore began to assume a great deal of 

significance in the international debate that surrounded climate science. Not 

surprisingly, given the enormous level of public interest in the subject, the debate 
CHAPTER 2: INTRODUCTION 

19

soon moved out of the confines of climate scientists and became highly polarized 

in websites, journals and conferences across the world. As a result, the work 

conducted by CRU became the focus of intense scrutiny and challenge with 

multiple demands from both fellow scientists and laymen for background 

information and data. 

6. In November 2009, the nature of the debate and challenge took on a whole new 

significance when approximately 1000 of CRU‘s e-mails were made public 

without authorization. This material fuelled more challenges to their work, to the 

reliability of climate science generally, and to the conclusions of the IPCC. All 

this happened shortly before the Copenhagen Summit, and was extensively 

referred to there. 

2.2 The Review 

7. The material in the e-mails led to a set of allegations against the leading members 

of CRU, focusing on whether CRU had operated in accordance with best scientific 

practice at the relevant time, and in particular: whether data had been manipulated 

or suppressed; whether peer review and the dissemination of findings had been 

properly handled; and whether CRU had complied with the Freedom of 

Information Act. 

8. In response to this, the Vice-Chancellor of the UEA established the Independent 

Climate Change E-mails Review, to be led by Sir Muir Russell. Sir Muir was 

given complete freedom to develop the terms of reference as necessary and to 

assemble an appropriate team and appropriate support; and was asked if possible 

to report in spring 2010. The terms of reference are in Chapter 3 and details of the 

team members are in Appendix 1. 

9. Two other formal inquiries have addressed the matter. First, the House of 

Commons Science and Technology Committee held a hearing on 1 March 2010, 

reporting on 31 March 20101. Secondly, the Vice-Chancellor commissioned 

Lord Oxburgh to review CRU‘s scientific output. The Oxburgh Scientific 

Assessment Panel and this Review proceeded entirely independently, though the 

latter took steps to ensure, following the House of Commons Science and 

Technology Committee‘s Report, that Lord Oxburgh was aware of the approach it 

was taking to issues that might bear on his work. Lord Oxburgh‘s report was 

published on 14 April 20102

.

10. In addition investigations have been initiated by the police, looking at the 

circumstances of the unauthorised disclosure of the e-mails; and by the ICO, 

looking at compliance with the Data Protection Act, the FoIA and the EIR. 

11. Responding to a joint request by the Chairman of the IPCC and the Secretary-

 

1

 Science and Technology Committee - Eighth Report: The disclosure of climate data from the Climatic 

Research Unit at the University of East Anglia 

2 Report by Lord Oxburgh‘s Science Assessment Panel 

http://www.uea.ac.uk/mac/comm/media/press/CRUstatements/SAP
CHAPTER 2: INTRODUCTION 

20

General of the United Nations, an independent review of the IPCC‘s processes 

and procedures is now under way by the InterAcademy Council (IAC), chaired by 

economist Harold T. Shapiro, former president of Princeton University. 

12. Although the focus of this Review is quite clearly on the behaviour of a number of 

scientists, it is also apparent that there are many wider issues raised by the events 

described above. These include issues such as the workings of the peer review 

system, the reporting of uncertainty in the translation of scientific findings into 

policy, the handling and release of data, and the role of research funders. As a 

contribution to the wider scientific debate about the conduct of science in the 21st

century, this Review also considers these questions. 
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

21
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

22

CHAPTER 3: TERMS OF REFERENCE AND METHOD 

OF ENQUIRY 

3.1 Terms of Reference 

1. The Review‘s terms of reference are as follows:

―The Independent Review will investigate the key allegations that arose from a 

series of hacked* e-mails from CRU. The Review will: 

Examine the hacked e-mail exchanges, other relevant e-mail exchanges and 

any other information held at CRU to determine whether there is any evidence 

of the manipulation or suppression of data which is at odds with acceptable 

scientific practice and may therefore call into question any of the research 

outcomes. 

Review CRU‘s policies and practices for acquiring, assembling, subjecting to 

peer review and disseminating data and research findings, and their 

compliance or otherwise with best scientific practice. 

Review CRU‘s compliance or otherwise with the University‘s policies and 

practices regarding requests under the Freedom of Information Act ( ̳the 

FoIA‘) and the Environmental Information Regulations ( ̳the EIR‘) for the 

release of data. 

Review and make recommendations as to the appropriate management, 

governance and security structures for CRU and the security, integrity and 

release of the data it holds.‖

*Note: The word  ̳hacked‘ as contained in the Review‘s terms of reference 

has been challenged in submissions to the Review, on the basis that the means 

by which the unauthorized disclosure of the e-mails was made has not been 

established. This matter is subject to police enquiries and the Review has 

made no judgment on the question. 

3.2 Method of Enquiry 

2. The Review sought to operate in an open and transparent way. This is described in 

the  ̳Approach and Work Plan‟ paper (Appendix 2) made public at the launch at 

the Science Media Centre in February 2010. The Review called for submissions, 

which it undertook to publish on its website: http://www.ccereview.org/Evidence.php 

3. To provide a focus for submissions the Review produced and placed on its 

website an  ̳Issues for Examination‟ paper (Appendix 3) which addresses the main 

allegations made against the members of CRU. This paper set out our initial 

understanding based on a preliminary reading of the e-mails. Approximately 100 

submissions were then received, including one from CRU itself. Some provided 

further background considered relevant by the authors, but there was no 

suggestion in the submissions that the original statement of the issues was 

fundamentally misdirected. As its work progressed, the team kept the issues paper 

constantly under review and gradually developed a sharper focus on the key 
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

23

allegations. 

4. Thus this report is structured to address the principal allegations. It is important to 

recognise that this is not a detailed inquiry into the precise meaning of every email. 

5. The team proceeded to investigate the allegations by interviewing members of 

CRU and others from the University. We considered that the nature of our inquiry 

was such that holding public hearings to gather evidence, as some had urged, 

would be unlikely to add significant value over and above the written record. Nor 

have we produced transcripts of the interviews. This is because our conclusions 

are founded on information given in submissions and at interviews relating to 

facts that can be checked and referenced, rather than on interview testimony as 

such. The team found that this process helped it follow up key points, leading to 

supplementary submissions and references. 

6. The team did not carry out interviews other than with CRU and other UEA staff 

(apart from preliminary discussions with ICO and the police and interviews with 

two relevant IPCC Review Editors). We recognise that natural justice requires 

that those in respect of whom findings will be made should have an opportunity to 

be heard: this does not apply to the authors of submissions and other parties, in 

respect of whom the Review has made no findings. 

7. Under our publication policy notes of the team‘s meetings and interviews have 

been placed on the website. Interview notes have been checked for factual 

accuracy with those involved. Only those that are potentially sensitive, such as 

notes of discussions with the police, have been withheld. The Team has sought to 

publish all the submissions received unless they were explicitly described as 

confidential, subject only to the redaction, on legal advice, of content the 

publication of which could have led to legal action. In addition, one submission 

has been withheld on legal advice. The Review has invited the author to make it 

available directly to enquirers. It is a matter for the author whether to proceed in 

this way. This submission has been given full consideration. 

8. A full list of submissions and details of meetings can be found in Appendix 4. 

9. It is important to note that the allegations relate to aspects of the behaviour of the 

CRU scientists and not to the content of their work and hence this Review 

addressed scientific questions only to the extent necessary to place this behaviour 

in context. 

10. On the subject of peer review, in addition to investigating the specific allegations 

the team commissioned a contextual paper from a distinguished Journal editor –

Dr Richard Horton of The Lancet – on which Elizabeth Wager, Chair of the 

Committee on Publication Ethics (COPE), was invited to comment in a personal 

capacity. This material forms Appendix 5 and is referred to Chapters 5 and 8. 

11. The Review maintained contact with the police and the ICO. It was important to 

ensure that the work of the team did not compromise their statutory 

responsibilities, and also to avoid duplication of effort. The team‘s findings and 
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

24

recommendations respect this division of responsibilities and the fact that 

investigations are continuing. 

12. The Review intends that the website and the submissions and evidence published 

on the website should be archived by the British Library Web Archive. 
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

25
CHAPTER 4: CONTEXT OF THE E-MAILS 

26

CHAPTER 4: CONTEXT OF THE E-MAILS 

4.1 Characterising the E-mails 

1. This Chapter gives an overview of the e-mails and seeks to relate their content to 

key events happening in the climate change world, thus giving context to the 

criticism of CRU. 

2. The information released comprises a very small (less than 0.3%) subset of files 

which were held on the back-up server at CRU, which include e-mails and other 

documents – such as text files, Word documents, Excel spreadsheets, PDF 

documents, and computer code. 

3. The focus of this Chapter is e-mails which spanned the period 7th March 1996 to 

12th November 2009. The  ̳primary‘ e-mails number 1073 in total with 166 

authors. There are more e-mails and authors if the associated e-mail chains are 

included. When printed on A4 paper the e-mails run to 3,375 pages and contain 

many embedded duplicates. Self-evidently each of the primary e-mails was either 

sent by or received by CRU members, but this is not the case for many of the 

associated e-mail chains. Those who authored the largest numbers of primary emails are as follows. 

Author Number Role

Philip Jones 174 Director, CRU, UEA and Coordinating Lead Author 

IPCC 4th Assessment Report

Michael 

Mann

140 Director, Earth System Science Centre, Pennsylvania 

State University (from 2005), and Lead Author IPCC 

3

rd Assessment Report

Keith Briffa 117 Professor, CRU, UEA and Lead Author IPCC 4th

Assessment Report

Jonathan 

Overpeck

90 Institute Director, University of Arizona and 

Coordinating Lead Author IPCC 4th Assessment 

Report

Tim Osborn 59 Academic Fellow, CRU, UEA and Contributing 

Author IPCC 4th Assessment Report

Ben Santer 51 Researcher, Lawrence Livermore National Lab, US

and Contributing Author IPCC 4th Assessment Report

Tom 

Wigley

35 Scientist, University Corporation for Atmospheric 

Research, Contributing Author IPCC 4th Assessment 

Report and a former Director of CRU

4. The e-mails relate to a number of the major developments in recent climate 

science from the Kyoto Summit in December 1997, through the 4th Assessment 

Report of the IPCC in 2007, to the run-up to the 15th Conference of the Parties 

(COP) in Copenhagen. Since CRU played a significant role in providing scientific 

input to these events, the release of the emails resulted in reduced global 
CHAPTER 4: CONTEXT OF THE E-MAILS 

27

confidence in climate science and more specifically in the findings of the IPCC. 

4.2 The Timeline 

5. The  ̳story‘ which underlies the selected e-mails, and hence is reflected in them, is 

summarised in the table and text which follow. 

Year Event

1997 Kyoto Summit

1998 MBH981including what has become known as the ―Hockey Stick‖

1999 MBH992reconstruction extended to 1000 AD

WMO Statement on the Status of the Global Climate in 19993

2001 IPCC 3rd Assessment Report4

2002 Esper et al 20025

2003 Soon & Baliunas 20036, McIntyre &McKitrick 20037criticism of MBH

Mann et al 20038,9

2004 von Storch 200410 questioning the statistical methods used in MBH

Launch of RealClimate11 website

 

1 M.E. Mann, R.S. Bradley and M.K. Hughes, ―Global-scale temperature patterns and climate forcing 

over the past six centuries‖, Nature 392, 779-787 23 April (1998) 

2 M.E. Mann, R.S. Bradley and M.K. Hughes, ―Northern hemisphere temperatures during the past 

millennium: Inferences, uncertainties, and limitations‖, Geophysical Research Letters, Volume 26. No. 

6, pp. 759-762 (1999) 

3

 http://www.wmo.ch/pages/prog/wcp/wcdmp/statemnt/wmo913.pdf 

4

 http://www.grida.no/publications/other/ipcc_tar/ 

5

 J. Esper, E.R. Cook, F.H. Schweingruber, ―Low-Frequency Signals in Long Tree-Ring Chronologies 

for Reconstructing Past Temperature Variability‖, Science, Volume 295, 22 March (2002) pp. 2250-

2253

6

 W. Soon, S. Baliunas, ―Proxy climatic and environmental changes of the past 1000 years‖, Climate 

Research Volume 23: 89-110, January 2003 

7

 S. McIntyre, R. McKitrick, ―Corrections to the Mann et. al. (1998) proxy data base and Northern 

hemispheric average temperature series‖, Energy & Environment, Volume 14, No. 6 (2003) pp. 751-

771

8 M.E. Mann and P.D. Jones, ―Global surface temperatures over the past two millennia‖, Geophysical 

Research Letters Volume 30, No. 15, 1820, August 2003 

9

 M.E Mann, C.M Ammann, R.S. Bradley, K.R. Briffa, T.J. Crowley, M.K. Hughes, P.D. Jones, M. 

Oppenheimer, T.J. Osborn, J.T. Overpeck, S. Rutherford, K.E. Trenberth, T.M.L. Wigley, ―On past 

temperatures and anomalous late 20th century warmth‖, Eos, 84, 256-258, 2003. 

10 H. von Storch, E. Zorita, J.M. Jones, Y. Dimitriev, F. González-Rouco, S.F.B. Tett, ―Reconstructing 

Past Climate from Noisy Data‖, Science, 22 October 2004, Volume 306. no. 5696, pp. 679 – 682 

11 http://www.realclimate.org/ 
CHAPTER 4: CONTEXT OF THE E-MAILS 

28

Year Event (continued)

2005 UK Freedom of Information Act12

Launch of ClimateAudit13 website

McIntyre &McKitrick 200514 criticism of MBH

Rutherford et al 200515 defence of MBH

2006 US National Research Council review16

US Committee on Energy & Commerce (Wegman) statistical review17

2007 IPCC 4th Assessment Report18

Wahl & Ammann 200719defence of MBH

2009 15th COP in Copenhagen

CRU‘s work assumed progressively greater significance as it became 

apparent that both the global temperature record and tree ring data it was 

collecting and analysing were being cited prominently in the work of the 

IPCC. 

Building in part on the work of CRU, Mann, Bradley & Hughes (MBH) 

published a paper in Nature in 199820 which sought to reconstruct historic 

temperatures back to 1400 AD, well before significant numbers of 

instrumental climate records began in around 1850. They used 

measurements including those from tree rings, ice cores and corals, socalled proxy data, which can reflect local temperature changes. This 

paper was updated in 199921 with the reconstruction extended back to 

1000 AD in the Northern Hemisphere. The resulting temperature profile 

had a  ̳hockey stick‘ shape. In particular it reduced the significance of 

what is termed the Medieval Warm Period (MWP), that is believed to 

have reached its peak between about 950 and 1100 AD, and the Little Ice 

Age (LIA) (from about 1500 to 1850 AD, and centered around 1700 AD), 

relative to previous representations. Temperatures over the last 50 years 

appeared unprecedented in the past thousand years. These data were used 

to create an iconic representation of anthropogenic global warming, as 

supplied by Jones, in the 1999 World Meteorological Organisation for its 

 

12 http://www.ico.gov.uk/what_we_cover/freedom_of_information.aspx 

13 http://climateaudit.org/ 

14 S. McIntyre, R. McKitrick, ―Hockey sticks, principal components, and spurious significance‖, 

Geophysical Research Letters, Volume 32, No. 3 (2005) 

15 S. Rutherford, M.E. Mann, T.J. Osborn, R.S. Bradley, K.R. Briffa, M.K. Hughes, P.D. Jones, 

―Proxy-based Northern Hemisphere surface temperature reconstructions: sensitivity to methodology, 

predictor network, target season and target domain‖, Journal of Climate (2005); 18: 2308-2329

16 Surface Temperature Reconstructions for the Last 2,000 Years 

http://www.nap.edu/catalog.php?record_id=11676 

17 Ad Hoc Committee Report on the  ̳Hockey Stick‘ Global Climate Reconstruction (2006) for the US 

Committee on Energy and Commerce 

18 http://www.ipcc.ch/ 

19 E.R. Wahl, C.M. Ammann, ―Robustness of the Mann, Bradley, Hughes reconstruction of Northern 

Hemisphere surface temperatures: examination of criticisms based on the nature and processing of 

proxy climate evidence‖ Climatic Change, Volume 85, Numbers 1-2, pp.33-69, November 2007 

20 Ibid. 1 

21 Ibid. 2 
CHAPTER 4: CONTEXT OF THE E-MAILS 

29

Statement on the Status of the Global Climate in 199922. They were also 

referenced in the Third Assessment Report (TAR) of the IPCC in 200123. 

The IPCC process occurs over a number of years, with drafts being 

produced periodically, leading up to the final report and summary for 

policy makers (SPM). The process is clearly documented24 with extensive 

voluntary involvement of many eminent scientists in authoring and 

reviewing chapters of the report. Key players in CRU were involved in 

the development of both the TAR and AR4 in 2007 and many of the emails relate to their involvement in the process. 

Questions were increasingly asked about MBH. In 2002 Esper published 

a paper25 which suggested that MBH had underestimated the strength of 

the MWP, but it was Soon & Baliunas‘s (S&B) papers published in 

Climate Research26 and with minor amendments in Energy & 

Environment (E&E)27 which first challenged MBH‘s results directly. An 

e-mail by Osborn circulating S&B‘s 2003 paper28 is one of the most 

duplicated in the released e-mails triggering nine of the subsequently 

selected e-mail chains. S&B‘s papers were followed in 2003 by another 

paper by McIntyre and McKitrick (M&M)29 also critical of MBH. At 

issue in the whole exchange of e-mails were the following key scientific 

issues:- 

 (i) the selection of tree ring data (particularly those from Yamal 

and the Polar Urals); 

(ii) the statistical methods for extracting information; 

(iii) the record of local and hence global temperatures (especially 

the impact of the urban heat island (UHI) effect); 

(iv) relating proxy records to instrumental temperature records; and 

(v) uncertainties in comparing recent warmth with that over the last 

2000 years. 

Prior to the publication of S&B‘s papers in 2003 those critical of MBH 

had not had a paper published in a mainstream journal. The publications 

in Climate Research and Energy & Environment were significant not only 

because they challenged MBH but also because they had been peer 

reviewed. Not only were rebuttals published in 2003 by Mann, Jones, 

Briffa, Osborn et al30,31

, but also the process of peer review at Climate 

Research was questioned. The editor, de Freitas, sought initially to 

 

22 Ibid. 3 

23 Ibid. 4 

24 http://www.ipcc.ch/organization/organization_procedures htm 

25 Ibid. 5 

26 Ibid. 6 

27 W. Soon, S. Baliunas, C. Idso, S. Idso. D.R. Legates, ―Reconstructing climatic and environmental 

changes of the past 1000 years: a reappraisal‖, Energy & Environment, Volume 14: Nos. 2 & 3, 2003, 

pp. 233-296

28 Ibid. 6 

29 Ibid. 7 

30 Ibid. 8 

31 Ibid. 9 
CHAPTER 4: CONTEXT OF THE E-MAILS 

30

defend himself as the e-mail extract below shows but he ultimately 

resigned as did members of the editorial board of Climate Research. The 

matter is discussed in detail in Section 8.3 of Chapter 8. 

In an e-mail dated 18/6/03 (1057944829.txt) de Freitas wrote: 

“I have spent a considerable amount of my time on this matter and had 

my integrity attacked in the process. I want to emphasize that the people 

leading this attack are hardly impartial observers. Mike himself refers to 

"politics" and political incitement involved. Both Hulme and Goodess are 

from the Climate Research Unit of UEA that is not particularly well 

known for impartial views on the climate change debate. The CRU has a 

large stake in climate change research funding as I understand it pays the 

salaries of most of its staff.”

In 2004 von Storch32 questioned the statistical methods used in MBH and, 

at around the same time, the RealClimate33 and Climate Audit34 websites 

were launched, the former defending and the latter critical of majority 

climate science. This marked a new phase of more public climate science 

debate and criticism, and was seen as an unwelcome development by 

those in CRU, who wanted criticism to follow the normal channels of 

private peer review and publication. This view remained relatively 

unchanged for over a decade, as shown by the following: 

In an e-mail dated 6/5/99 (926031061.txt) Jones wrote: 

“I must admit to having little regard for the Web. Living over here makes 

that easier than in the US - but I would ignore the so-called skeptics until 

they get to the peer-review arena. I know this is harder for you in the US 

and it might become harder still at your new location. I guess it shows 

though that what we are doing in [sic] important. The skeptics are 

fighting a losing battle.”

In an e-mail dated 30/9/09 (1254323180.txt) Jones wrote: 

“Another issue is science by blog sites - and the then immediate response 

mode. Science ought to work through the peer-review system..... sure 

you've said all these things before. We're getting a handful of nasty emails 

coming and requests for comments on other blog sites.”

Critics of CRU claimed that they were unable to reproduce CRU‘s work 

due to a lack of access to data. The UK EIR and FoIA35 should have 

encouraged the release of information, but this does not seem to have 

been the initial result. The volume of requests grew over time, more 

formally under the FoIA starting in 2007 and spectacularly in 2009, 

particularly related to details of the CRUTEM land temperature datasets 

 

32 Ibid. 10 

33 Ibid. 11 

34 Ibid. 13 

35 Ibid. 12 
CHAPTER 4: CONTEXT OF THE E-MAILS 

31

and the role CRU played in the development of IPCC AR4. 

In an email dated 7/5/04 (1083962601.txt) Jones wrote: 

“Many of us in the paleo field get requests from skeptics (mainly a guy 

called Steve McIntyre in Canada) asking us for series. Mike and I are not 

sending anything, partly because we don't have some of the series he 

wants, also partly as we've got the data through contacts like you, but 

mostly because he'll distort and misuse them. Despite this, Mike and I 

would like to make as many of the series we've used in the RoG [Reviews 

of Geophysics] plots available from the CRU web page.”

We do not suggest that the allegations made against McIntyre are correct. 

In the run up to IPCC AR4 further papers were published criticising 

(McIntrye & McKitrick 200536) and defending MBH (Rutherford 200537

, 

Wahl and Ammann 200738). In parallel, political interest in AR4 grew 

and in the United States of America two reviews were commissioned, one 

by the National Research Council39 into surface temperature 

reconstructions for the last 2000 years and the other by the Committee on 

Energy and Commerce (Wegman)40 into the quality of the statistical 

analysis used in MBH. These reviews were critical of MBH; nevertheless 

the AR4 SPM41 contained the following statement. 

“Average Northern Hemisphere temperatures during the second half of 

the 20th century were very likely higher than during any other 50-year 

period in the last 500 years and likely the highest in at least the past 1300 

years.”

In the aftermath of AR4, attempts were made to demonstrate that the 

IPCC process had been abused by CRU members. Jones was accused of 

attempting to exclude reference to a 2004 paper by McKitrick and 

Michaels42, and Briffa of including reference to Wahl 200743 before it was 

in publication, contrary to IPCC rules. 

These accusations were accompanied by Freedom of Information FOI 

requests. In an email dated 10/12/08 (1228922050.txt) Jones wrote: 

“Anyway requests have been of three types - observational data, paleo 

data and who made IPCC changes and why. Keith has got all the latter –

 

36 Ibid. 14 

37 Ibid. 15 

38 Ibid. 19 

39 Ibid. 16 

40 Ibid. 17 

41 AR4 WG1 Summary for policymakers: 

http://www.ipcc.ch/publications_and_data/ar4/wg1/en/spmsspm-a-palaeoclimatic.html 

42 R. McKitrick and P.J Michaels, ―A test of corrections for extraneous signals in gridded surface 

temperature data‖, Climate Research 26, 159-173, 2004 

43 Ibid. 19 
CHAPTER 4: CONTEXT OF THE E-MAILS 

32

and there have been at least 4. We made Susan aware of these - all came 

from David Holland. According to the FOI Commissioner's Office, IPCC 

is an international organization, so is above any national FOI. Even if 

UEA holds anything about IPCC, we are not obliged to pass it on, unless 

it has anything to do with our core business - and it doesn't!”

4.3 What the E-mails Tell Us

6. Since the communication was assumed to be private, it was generally informal, 

using slang, jargon and acronyms. Now that the e-mails have become public, 

some are doubtless regretted by their authors. Indeed, some submissions have 

characterised them as  ̳unprofessional‘, or as evidence of CRU‘s contribution to a 

 ̳poisoned atmosphere‘ in climate science. The question is whether this was 

unusual, or simply characteristic of the normal mode of expression in e-mail 

communication. 

7. This is a well known issue in social psychology and communication. Early 

authoritative works44,45,46, reinforced by later authors, explore normative language 

in electronic communication in comparison with other written and oral forms. 

They demonstrate that e-mail communication is less inhibited than other written 

and spoken forms, and suggest reasons why. Extreme forms of language are 

frequently applied to quite normal situations by people who would never use it in 

other communication channels. 

8. It is also clear from the submissions that it is possible to place different 

interpretations on the same phrase. In such circumstances, only the original 

author can really know what their intentions were. For example, on 16/11/99 

(942777075.txt) Jones wrote his now infamous e-mail including the following: 

“I've just completed Mike's Nature trick of adding in the real temps to each series 

for the last 20 years (ie. from 1981 onwards) amd [sic] from 1961 for Keith's to 

hide the decline.”

McIntyre in his submission47 to the Review Team, states 

“The IPCC “trick” was not a “clever” mathematical method – it was merely the 

deletion of inconvenient data after 1960.”

This compares with Jones‘s own commentary in the UEA submission48 to the 

Review Team. 

“The email was written in haste and for a limited and “informed” audience (the 

people that had provided data). The word “trick” was not intended to imply any 

 

44 Sproull, L. and Kiesler, Sara. 1986. Reducing Social Context Cues: Electronic Mail in Organisational 

Communications. Management Science, 32, p.1492-1512

45 Sproull, L. and Kiesler, S. 1991. Connections. MIT Press, Cambridge MA. 

46 Shapiro, N and Anderson, R. 1985. Towards and Ethics and Etiquette for Electronic Mail. Rand 

Corporation. 

47 McIntyre Submission (No.23), 1 March 2010, paragraph 22 

48 Climatic Research Unit Submission (No.5), 1 March 2010, 3.1 on p.36 
CHAPTER 4: CONTEXT OF THE E-MAILS 

33

deception, simply the “best way of doing or dealing with something”. The 

reconstruction from the tree-ring density network was not shown after 1960, and 

thus in this sense it is “hidden” – but justifiably so: excluding the anomalous 

tree-ring density data is justified if the purpose is to illustrate the most likely 

course of temperature based on a combination of proxy and measured 

temperatures. Again, no deception was intended or practised.”

9. E-mails are rarely definitive evidence of what actually occurred. They are open to 

interpretation, but they are also indicative. Having identified specific allegations 

against CRU, based on the e-mails, the Review then sought to obtain evidence to 

substantiate or refute these allegations, as described in the subsequent Chapters of 

this report. 

10. The presumption is that e-mails were selected to support a particular viewpoint. 

Recognising that they were a tiny fraction of those archived, the Review Team 

sought to learn more about the full contents of the back-up server. This attempt, 

summarised in Appendix 6, was largely unsuccessful due to the sheer scale of the 

task and ongoing police investigation. 

11. In some instances the leaked e-mails contain statements which might be viewed as 

supporting the behavior of CRU and countering the allegations in paragraph 9 

above. The following extracts from e-mails by Briffa are examples. 

E-mail dated 22/9/99 (938018124.txt): 

“the early (pre-instrumental) data are much less reliable as indicators of global 

temperature than is apparent in modern calibrations that include them and when 

we don't know the precise role of particular proxies in the earlier portions of 

reconstruction it remains problematic to assign genuine confidence limits at 

multidecadal and longer timescales.”

E-mail dated 3/2/06 (1138995069.txt): 

“we are having trouble to express the real message of the reconstructions - being 

scientifically sound in representing uncertainty , while still getting the crux of the 

information across clearly...We have settled on this version (attached) of the 

Figure which we hoe [sic] you will agree gets the message over but with the rigor 

required for such an important document.”

12. On the other hand there are some e-mails which show that CRU members may 

have gone out of their way to make life difficult for their critics, just as they 

perceived the critics to be frustrating their work. Some examples follow: 

 

Jones e-mail dated 27/4/05 (1114607213.txt): 

“I got this email from McIntyre a few days ago. As far as I'm concerned he has 

the data - sent ages ago. I'll tell him this, but that's all - no code. If I can find it, it 

is likely to be hundreds of lines of uncommented fortran ! I recall the program 

did a lot more that just average the series. I know why he can't replicate the 

results early on - it is because there was a variance correction for fewer series.”
CHAPTER 4: CONTEXT OF THE E-MAILS 

34

Jones e-mail dated 3/12/08 (1228330629.txt):

“The inadvertent email I sent last month has led to a Data Protection Act request 

sent by a certain Canadian, saying that the email maligned his scientific 

credibility with his peers! If he pays 10 pounds (which he hasn't yet) I am 

supposed to go through my emails and he can get anything I've written about him. 

About 2 months ago I deleted loads of emails, so have very little - if anything at 

all.”

Osborn e-mail dated 23/6/08 (1214228874.txt):

“I've just had a quick look at CA. They seem to think that somehow it is an 

advantage to send material outside the formal review process. But *anybody* 

could have emailed us directly. It is in fact a disadvantage! If it is outside the 

formal process then we could simply ignore it, whereas formal comments had to 

be formally considered. Strange that they don't realise this and instead argue for 

some secret conspiracy that they are excluded from! I'm not even sure if you sent 

me or Keith anything, despite McIntyre's conviction! But I'd ignore this guy's 

request anyway. If we aren't consistent in keeping our discussions out of the 

public domain, then it might be argued that none of them can be kept private. 

Apparently, consistency of our actions is important.”

13. During the work of the Review an additional concern surfaced, namely that of 

financial controls, for example see extract below. This is addressed in Chapter 11 

(paragraph 18). In an e-mail dated 7/3/96 (826209667.txt) Shiyatov (A Russian 

tree ring researcher) wrote: 

“It is important for us if you can transfer the ADVANCE money on the personal 

accounts which we gave you earlier and the sum for one occasion transfer (for 

example, during one day) will not be more than 10,000 USD. Only in this case we 

can avoid big taxes and use money for our work as much as possible.”

14. Finding: The extreme modes of expression used in many e-mails are 

characteristic of the medium. Crucially, the e-mails cannot always be relied upon 

as evidence of what actually occurred, nor indicative of actual behaviour that is 

extreme, exceptional or unprofessional. 

15. The Chapters which follow address specific behavioural allegations based in 

particular on a comparatively small number of e-mails. Despite the fact that all 

the e-mails released are unlikely to be representative of the larger set on the CRU 

back-up server, there nevertheless would appear to be a pattern of behaviour. 

There is little doubt about the polarisation of views in the world of climate 

science, which has overstepped the line dividing heated scientific debate from 

outright hostility. One camp comprises the main authors of the e-mails in 

paragraph 3, who are acknowledged leaders in majority climate science as 

indicated by the roles they all played in the IPCC. The other camp is their critics, 

for whom pejorative terms such as ―prat, dishonest, appalling, rubbish and crap‖ 
CHAPTER 4: CONTEXT OF THE E-MAILS 

35

were used by some CRU members to refer to them or their work. More generally 

the majority climate scientists appear to have been united in their defence against 

criticism. Whilst perhaps understandable, given the nature and methods of 

criticism, some of which impugns their personal integrity as well as challenging 

their work, this may have blinded some CRU members to the possibility of merit 

therein. Such denial then fuelled yet further antagonism. There needs to be better 

communication, as well as greater openness enabling more scientific debate. We 

comment on this in Chapters 5 and 10. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

36

CHAPTER 5: THE CHANGING CONTEXT OF MODERN 

SCIENCE 

1. In this Chapter we review the changing context of modern science as a framework 

against which CRU‘s practices can be judged. 

2. When CRU was created in the early 1970s, climate change was a relatively 

obscure area of science. But it has developed into an area of great political and 

public concern. CRU found itself ―in the eye of the storm‖ in generating much of 

the data necessary to address one of the most important issues of the century. 

3. This has come at a time of transformation in the need for openness in the culture 

of publicly-funded science. It is being driven by a range of pressures including 

new recommendations from national academies addressing the timely release of 

data, increased demands for scientific input to public policy, important changes in 

the law and challenge from the blogosphere. 

5.1 The Scientific Process

4. Scientific hypotheses and theories are presumed to be provisional: they can be 

refuted by testing but they cannot be verified as correct or true in an absolute 

sense. Verification is however possible of the results of the experiments upon 

which theories are built or that attempt to test a theory. If they can be repeated, 

and produce the same results, they are said to be validated. It is important to 

recognise that science progresses by substantive challenges based on rigorously 

logical, published arguments that present a different view of reality from that 

which they challenge. Criticising and attacking process and behaviour is not the 

same as an attack on a scientific hypothesis. Failure in one is not necessarily 

failure in the other. 

5. Given the nature of our remit, our concern is not with science, whether data has 

been validated or whether the hypotheses have survived testing, but with 

behaviour; whether attempts have been made to misrepresent, or ―cherry pick‖ 

data with the intention of supporting a particular hypothesis, or to withhold data so 

that it cannot be independently validated, or to suppress other hypotheses to 

prevent them being put to the test. 

5.2 The Nature of Climate Science 

6. Modern climate science is largely a creation of the last 30-40 years. It involves a 

wide range of disciplines, including geography, physics, chemistry, a wide variety 

of earth sciences, palaeo-biology and computer modelling. The synthesis of such 

diverse sources of evidence generated through wide variety of scientific 

approaches and traditions has been a major task, in which the IPCC has played an 

important role. 

7. One of the main contributions of CRU has been in the laborious assembly of 

datasets of instrumental measures of climate change and of data relating to tree 

rings that are proxies for climatic variables. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

37

5.3 Data Accessibility in the Digital Age

8. Modern digital technologies permit the acquisition and manipulation of very much 

larger datasets than formerly. To enable proper validation of the conclusions, 

such datasets must be made freely available, along with details of the associated 

computational manipulation. It is often difficult and time-consuming to describe 

this  ̳meta-data‘ in ways that permit others to verify experimental results and test 

the inferences drawn from them. The preservation of such data also presents a 

novel problem of archiving and stewardship compared with the archives of former 

times. 

9. An important shift in attitude is represented by a recent report by the US National 

Academies1

, which highlights these issues. It recommends a new approach to the 

integrity, accessibility, and stewardship of data in publicly-funded science, 

arguing that researchers should make all research data, methods, and other 

information underlying the results publicly accessible in a timely manner. These 

recommendations would require a substantial shift of behaviour amongst many 

scientists. The e-mails suggest that this would be true for CRU – for example: 

Jones: “We have 25 or so years invested in the work. Why should I make the data 

available to you, when your aim is to try and find something wrong with it”

2

.

10. Digital datasets in specific areas of science are increasingly being placed in 

international archives. The World Data System, run by the International Council 

for Scientific Unions3, for example, covers a very wide range of data types 

including the International Tree-Ring Data Bank. The majority of the source data 

used to create the CRUTEM gridded product is available through the Global 

Historical Climatology Network (GHCN) managed by the NCDC4

 and the US 

National Center for Atmospheric Research (NCAR)5.

11. Some funders of research require that applications for funding include plans for 

data archiving. The Natural Environment Research Council (NERC), the UK 

research council most relevant to the work of CRU, has a policy that requires 

recipients of grants to offer to deposit with NERC a copy of datasets resulting 

from the supported research6

. 

12. Over the period under review, data accessibility was also being driven by major 

changes in the law that had important implications for research scientists working 

in the UK. Whilst the precursors of the Environmental Information Regulations 

date back to the 1990s, the current EIR came into force on 1st January 2005, on the 

same date as the general right of access under the Freedom of Information Act 

2000 (FoIA). These are described in more detail in Chapter 10. The EIR applies 

exclusively within a rather broad definition of environmental and related 

 

1

 National Academy of Sciences. 2009. Ensuring the Integrity, Accessibility, and Stewardship of 

Research Data in the Digital Age. 188pp. 

2

 Referred to in Sherrington submission (No. 38), 17 February 2010, page 10) 

3

 http://www.ukssdc.ac.uk/wdcmain/guide/wdcguide html 

4

 http://www.ncdc noaa.gov/oa/ncdc.html 

5

 http://www.ncar.ucar.edu/ 

6

 NERC Data Policy (Version February 2010) 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

38

information; this area is then excluded from the FoIA. The extent to which both 

regimes apply to access to information from publicly funded (or part-funded) 

research is still an evolving area. Whilst it is broadly recognised that these do 

apply to data and supporting metadata, algorithms, etc. at the point when a 

research paper is published, how to interpret the legislation and regulations in 

terms of data held for a long period prior to publication and also pre-publication 

drafts and other correspondence is work in progress. The  ̳public interest test‘, 

applicable to both EIR and FoIA, again described in Chapter 10, remains key to 

such interpretation. 

5.4 Handling Uncertainty 

13. There is a widespread misconception that science produces unequivocal and 

absolutely precise answers. It does not, and cannot. All scientific results contain 

uncertainties, and it is important that these are made clear to and are understood 

by those who use them. There are two fundamental sources of uncertainty in 

science: uncertainty in measuring a phenomenon and uncertainty in determining 

causes and causal relationships. 

14. Over the period in question, the increasing amount of quantitative data available 

on climate change has made it easier to assess and represent uncertainty. This can 

clearly be seen in the representation of what is termed the Medieval Warm Period. 

Prior to the 1980s this was shown as a single line, without error bars7. Subsequent 

quantitative palaeo-temperature work on tree rings, in which CRU has been a 

leader, has permitted errors to be assessed as a consequence of the statistical 

correlations involved (see Chapter 7). Although the recent Oxburgh Scientific 

Assessment Panel8

 has been critical of some of the statistical work of CRU in 

relation to the tree ring series, that issue lies beyond our remit. Our concern is 

whether tree ring series have been improperly selected and whether the 

uncertainties have been properly presented. 

15. The processes of hypothesis, experiment, testing and refutation generally reduce 

uncertainties about causes and relationships but do not entirely remove them. 

Some of the conflicts involving CRU have been scientific disagreements about 

causes and relationships that in principle should be resolved by new data which 

exposes existing hypotheses to rigorous testing. One of the allegations against 

members of CRU has been that they have attempted improperly to prevent ideas 

with which they disagreed from being effectively discussed, for example in 

relation to the publication of papers (Chapter 8) and in parts of the IPCC process 

(Chapter 9). 

 

7

 e.g. Houghton, J.T., Jenkins, G.J. and Ephraums J.J. 1990. Climate Change: the IPCC Scientific 

Assessment. Cambridge University Press. 

8 Report of the International Panel set up by the University of East Anglia to examine the research of 

the Climatic Research Unit. 2010. Available at: 

http://www.uea.ac.uk/mac/comm/media/press/CRUstatements/oxburgh 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

39

5.5 Scientific Journals and the Peer Review Process 

16. Allegations against CRU include improper attempts to influence the editorial 

policy of scientific journals and the peer review process. To help it understand the 

context, the Review commissioned a contextual paper from a distinguished 

Journal editor – Dr Richard Horton of The Lancet – on which Elizabeth Wager, 

Chair of COPE, was invited to comment in a personal capacity. (Appendix 5). 

17. Peer review is important because of the way that scientific journals provide the 

frame of reference for most formal scientific debate. Scientific ideas that are not 

expressed in a scientific journal are rarely addressed. Edward Wilson, for 

example, commented that  ̳a discovery does not exist until it is safely reviewed 

and in print‟.

9

18. Access to publication in scientific journals is therefore a crucial issue. Editors 

have used peer review by acknowledged experts of papers submitted for 

publication to advise them whether they contain errors, are important, trivial or 

simply repeat what has previously been published. 

19. Richard Horton‘s essay explores typical patterns of behaviour relating to journals 

and peer review in contentious areas of science, and what behaviour should be 

regarded as improper. His conclusions are summarised as follows: 

What an editor seeks from a reviewer is a powerful critique of the manuscript 

– testing each assumption, probing every method, questioning all results, 

sceptically challenging interpretations and conclusions and ensuring that 

uncertainties are fully acknowledged, measured, and reported. 

Armed with such a critique, the editors decide, and take full responsibility for 

deciding, whether to publish. 

There is always the risk of group-think among experts which resists alternative 

perspectives. Editors try to reduce the risk of group-think by sending papers to 

different and widely dispersed reviewers, deliberately seeking or even 

provoking critical reviews. 

Editors send manuscripts to reviewers based on a principle of confidentiality. 

The author expects the editor to maintain a covenant of trust between the two 

parties. Disclosure to a third party without the prior permission of the editor 

would be a serious violation of the peer review process and a breach of 

confidentiality. 

Many who are far from the reality of the peer review process would like to 

believe that peer review is a firewall between truth on the one hand and error 

or dishonesty on the other. It is not. It is a means of sieving out evident error, 

currently unacceptable practices, repetition of previously published work 

without acknowledgement, and trivial contributions that add little to 

 

9

 Wilson, E.O. 1998. Consilience: The Unity of Knowledge. New York: Knopf. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

40

knowledge. 

It does not and cannot guarantee veracity. Many published papers have proved 

deeply flawed: many good ones have been rejected. Nor has it been efficient in 

identifying fraud, which has usually come to light by different routes. 

However, journals, as the gatekeepers of scientific publication, have come to 

exert an increasing influence on the reputations of scientists, research units and 

universities. Many measures of academic success, promotion, tenure, grants, 

fame, and personal wealth depend upon journal publication. It is not surprising 

therefore that journals, and peer review, are the subject of constant tension and 

occasionally explosive controversy. 

20. In conclusion, it is common for editors to have multiple, intense, and sometimes 

sharp and passionate interactions with authors and reviewers. The tone of their 

exchanges and communications with editors can be attacking, accusatory, 

aggressive, and even personal. If a research paper is especially controversial and 

word of it is circulating in a particular scientific community, third-party scientists 

or critics with an interest in the work may get to hear of it and decide to contact 

the journal. They might wish to warn or encourage editors. This kind of 

intervention is entirely normal. It is the task of editors to weigh up the passionate 

opinions of authors and reviewers, and to reflect on the comments (and 

motivations) of third parties. Occasionally, a line might be crossed into highly 

improper behaviour leading, for example, to censorship of ideas that might 

normally pass peer review. Defining that line is the crucial task when judging the 

role of CRU scientists, and determining whether, as has been alleged, they acted 

to subvert peer review by slowing or blocking the publication of research which 

disagreed with their own views. Was their activity part of the normal, robust 

hurly-burly surrounding publication in important highly contended fields, or was 

an important line crossed? We address this in Chapter 8. 

5.6 The Responsibilities of Scientists in Communicating in 

the Public Domain 

21. The scientific literature is relatively opaque to non-specialists. Scientific 

understanding that is transmitted into the public domain must be comprehensible 

to non-specialists, make appropriate and not excessive claims, and include careful 

statements of the uncertainties surrounding that understanding. These principles 

apply universally to scientific advice, and are embedded in the codes of practice 

for scientific advice that have been adopted by many governments10. One of the 

allegations against CRU is that they have not been sufficiently frank in 

communicating uncertainties about their reconstructions into the public domain. 

This is particularly relevant to graphical presentation such as the  ̳hockey stick‘ 

which has taken on iconic significance. Images have a great power to persuade, 

and this is particularly true when complex issues are faced by lay audiences, who 

may often infer a level of certainty that does not in fact exist. The danger is 

 

10 For example, the ―Code of Scientific Ethics‖ published in 2007 by the UK Chief Scientific Advisor, 

and accepted by UK Government, includes the responsibility to: ―present and review scientific 

evidence, theory or interpretation honestly and accurately.‖
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

41

obviously heightened where an image is being used to support arguments for 

policy change. Therefore, if images are likely to be used in this way, it is essential 

that qualifications such as uncertainties are given a closely coupled prominence 

and explanation. 

5.7 Communicating to Policymakers 

22. The interface between science and public policy is a crucial one in matters of great 

public importance such as climate change. The IPCC was set up to provide just 

such an interface. Its job was to draw on and synthesise the diverse strands that 

contribute to modern climate science (see paragraph 6) and to make this accessible 

to the public, policymakers and other stakeholders in a way that is comprehensible 

and that does justice to underlying uncertainties. The importance of this process is 

underlined by the potential magnitude of the economic and social consequences of 

governmental decisions in the domain of global climate change. It has been 

alleged that CRU scientists subverted IPCC processes by minimising uncertainties 

and blocking ideas that disagreed with their established views. 

23. The IPCC produces assessments of the current state of understanding of climate 

change, its causes and implications. Its approach is to produce the most probable 

account of these issues; together with their uncertainties, and to identify where 

there is insufficient evidence to discriminate between different interpretations of a 

phenomenon. Its purpose is to produce a  ̳best estimate‘ of what is currently 

understood, through the work of a group of scientists chosen for their expertise 

and experience to make reasoned assessments on the balance of evidence. It is not 

to produce a review of the scientific literature. 

24. The IPCC assessment reports were published in 1990, 1996, 2001 and 2007. AR4, 

published in 2007, consisted of three components: The Physical Science Base

(Working Group 1 – WG1); Impacts, Adaptation and Vulnerability (WG2); and 

Mitigation of Climate Change (WG3). The issues that concern CRU are a 

consequence of their involvement in WG1. The core activities of these WGs are 

undertaken by writing teams consisting of Coordinating Lead Authors (CLAs) and 

Lead Authors (LAs). There are two drafting cycles prior to the production of the 

Final Draft. The first draft is submitted to Expert Reviewers for comment. The 

second draft is submitted for both expert and governmental review. Review 

Editors are appointed for each of the review cycles to ensure that critical 

comments are properly dealt with. The final draft is the source of a Summary for 

Policymakers, which is also reviewed by governments prior to publication. 

5.8 The Changing Forum for Debate and the Blogosphere 

25. The development in recent years of the internet as a vehicle for easy, 

instantaneous transmission of news and opinion has changed the nature of the 

debate about scientific issues. Prior to these developments, scientific debate 

largely took place in journals and conferences that effectively excluded the public 

from active engagement. Experts tended to introduce their conclusions to the 

public in ways that were difficult to challenge. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

42

26. The mode has now changed and the field of climate change exemplifies this. 

There continues to be a scientific debate about the reality, causes and uncertainties 

of climate change that is conducted through the conventional mechanisms of peerreviewed publication of results, but this has been paralleled by a more vociferous, 

more polarised debate in the blogosphere and in popular books. In this the 

protagonists tend to be divided between those who believe that climate is 

changing and that human activities are contributing strongly to it, and those that 

are sceptical of this view. This strand of debate has been more passionate, more 

rhetorical, highly political and one in which each side frequently doubts the 

motives and impugns the honesty of the other, a conflict that has fuelled many of 

the views expressed in the released CRU emails, and one that has also been 

dramatically fuelled by them. It is difficult at the moment to predict whether and 

how the necessary cooler, rigorous scientific debate and the vital public policy 

interface will develop, or the effect that it will have on scientific publication or 

peer review. 

27. Arguably the most significant change produced by the blogosphere is a 

transformation in the degree of openness now required of scientists whose work 

directly affects policy making. Without such openness, the credibility of their 

work will suffer because it will always be at risk of allegations of concealment 

and hence mal-practice. The extent to which this change was fully recognised by 

both CRU and UEA administration is an important issue for the Review. 

28. Therefore, the Review would urge all scientists to learn to communicate their 

work in ways that the public can access and understand; and to be open in 

providing the information that will enable the debate, wherever it occurs, to be 

conducted objectively. That said, a key issue is how scientists should be 

supported to explain their position, and how a public space can be created where 

these debates can be conducted on appropriate terms, where what is and is not 

uncertain can be recognised. The learned societies may have an expanded role to 

play here in encouraging debate. We would also commend the work of bodies 

such as the Science Media Centre at the Royal Institution for encouraging and 

helping scientists to take their work to lay audiences through the media, and 

advising them on how best to do this. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

43
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

44

CHAPTER 6: LAND STATION INSTRUMENTAL 

TEMPERATURE DATA

6.1 Background 

1. Terrestrial temperature data is recorded at more than seven thousand land stations 

across the world. This is referred to as primary data. 

2. The steps involved in producing CRU‘s analysis of global temperature trends 

involve assembling and quality checking the primary data from approximately 

four thousand stations, determining the monthly ―anomalies‖1for each station, and 

then assembling averages based on a uniform 5x5 degree grid covering the globe. 

3. This work, requiring considerable experience and man-years of effort, was started 

by CRU in 1978 and it continues today. The results were published in a series of 

CRUTEM datasets over a period of 20 years between 1986 and 2006. 

4. In assembling a gridded temperature dataset, some adjustment of primary data 

may be needed in order to allow for what are referred to as  ̳non-climatic effects‘. 

A simple example might involve adjusting temperatures at a particular station to 

allow for obvious recording errors, or the elimination of an obvious discontinuity 

in results if a station had been moved (referred to as homogenization). Such 

adjustments are both necessary and ubiquitous in scientific research. 

5. This Chapter deals with the availability of these data to anyone wishing to carry 

out an equivalent study, the effect of adjustments, the comparison with 

independent studies and the availability of the information required to explicitly 

check the CRUTEM analysis. Finally, it addresses specific allegations about 

Chinese data used in a paper about the effect of urbanization on temperature. 

6.2 The Allegations 

6. The broad allegations which are prevalent in the public domain are: 

That CRU prevented access to raw data. 

That CRU adjusted the data without scientific justification or adequate 

explanation. Some allegations imply that this was done to fabricate evidence 

for recent warming. 

That CRU inappropriately withheld data and computer code, thus inhibiting 

others from checking the conclusions of the CRUTEM analysis. 

The overall implication of these allegations is to cast doubt on the extent to which 

CRU‘s work in this area can be trusted and should be relied upon. 

 

1 A station  ̳normal‘ is the average recorded temperature over a defined time period. In the case of 

CRUTEM this is 1961-1990. The temperature ―anomaly‖ is defined as the difference between the 

recorded temperature and the normal. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

45

7. While very few of the submissions to this Inquiry make the allegations above 

explicitly, they are nevertheless implied. The full text of the submissions is 

contained on the team website. It is not the intent here to review each one, but 

only to highlight the most relevant and significant. 

8. The most comprehensive and substantive submission critical of CRU is from 

McKitrick2. He addresses the question of data adjustments and also clarifies that 

the issue is not generally one of data availability, but more specifically the 

availability of a list of additional stations used in the CRUTEM analysis since 

1986. He draws attention to the series of requests to CRU for station identifiers 

over the period 2002-2007, eventually culminating in a series of FOI requests, 

also referred to by Matthews3

 and Cockroft4. Two submissions5 allege that CRU 

has withheld access to primary data and by implication stopped others from 

repeating analyses, but present no evidence. A common theme in the submissions 

from the Global Warming Policy Foundation (GWPF), McKitrick and McIntyre6

is the lack of cooperation exhibited by CRU in their dealings with selected third 

parties. 

9. In support of CRU, several submissions7,8,9

 point out that the data is in the public 

domain and can be accessed by anyone wishing to repeat a temperature trend 

analysis. Santer also defends the work done by CRU on regional adjustments, 

maintaining that it made very little difference to the global results. He also 

highlights that studies conducted independently from CRU, based on different 

choices and adjustment protocols, produce very similar results – a point also made 

by Joos10 and Tett. 

10. In its own submission under issue 4, CRU11 defends its position with regard to 

adjustments and openness. The website also contains the minutes of the meeting 

which took place on the subject of data sets on March 4th, 2010 between members 

of CRU and the Review Team. 

6.3 The Approach Adopted by the Review Team 

11. In order to test the principal allegations of withholding data and making 

inappropriate adjustments, the Review undertook its own trial analysis of land 

station temperature data. The goal was to determine whether it is possible for an 

independent researcher to (a) obtain primary data and (b) to analyse it in order to 

produce independent temperature trend results. This study was intended only to 

test the feasibility of conducting such a process, and not to generate scientific 

conclusions. The process followed is described in Appendix 7. 

 

2

 McKitrick submission (no.15), 26 February 

3

 Matthews submission (no.16), 1 March 

4

 Cockroft submission (no. 6), 28 February 

5

 Global Warming Policy Foundation submission (no. 22), 28 February, and Cockroft (Ibid.) 

6

 McIntyre submission (no. 23), 2 March 

7

 Tett submission (no. 21), 1 March 

8

 Santer submission (no. 44), 28 February 

9

 Barnes submission (no. 1), 6 March 

10 Joos submission (no. 12), 1 March 

11 CRU submission (no. 5), 1 March 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

46

12. The Team then went on to address a further question, namely the extent to which 

an independent researcher could check the CRUTEM analysis. In particular, this 

considered: 

whether the data sources were properly explained 

the availability of software required in the analysis 

6.4 The Results of the Analysis 

13. To carry out the analysis we obtained raw primary instrumental temperature 

station data. This can be obtained either directly from the appropriate National 

Meteorological Office (NMO) or by consulting the World Weather Records 

(WWR) available through the National Centre for Atmospheric Research 

(NCAR), or the Global Historical Climatology Network (GHCN): 

NCAR: http://www.ncar.ucar.edu/tools/datasets/ 

WWR: http://dss.ucar.edu/datasets/ds570.0/ 

GHCN: http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php. 

14. Anyone working in this area would have knowledge of the availability of data 

from these sources. There are also other sources but we have not investigated 

these.

15. The Review extracted a comprehensive set of global temperature records from 

both GHCN and NCAR websites to test ease of availability from either. The 

stations included in these sources have much overlap as would be expected. In 

addition, as a test case, primary data was also obtained directly from the Japanese 

NMO. 

16. In a true analysis the researcher would at this point be able to make whatever 

quality control, station selection and adjustments they considered appropriate, 

which should be justified in the published record of research. 

17. Figure 6.1 below gives the results of the Team‘s trial analysis to produce three 

different global temperature anomaly average series. These use a 5x5 degree grid 

and a 5-year smoothing (for details see Appendix 7). In each case we started with 

the full dataset, requiring only that a station had enough data to construct the 

relevant normals when constructing the gridded averages. Apart from this we 

made no selection or other adjustments of our own. We show the result obtained 

using the GHCN unadjusted data set (blue), the GHCN adjusted data set (yellow) 

and the NCAR dataset which is unadjusted (green). On the same figure we show 

as a black line the global average series obtained from the land air temperature 

anomalies on a 5x5 degree grid-box basis (CRUTEM3) gridded analysis (which 

uses adjusted data). The green and blue lines are both from unadjusted datasets. 

The yellow and black lines are from adjusted datasets. 

18. All the lines, whether from adjusted or unadjusted datasets, agree very well. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

47

Figure 6.1: Temperature anomaly time series created by the Review Team‘s own trial analysis using a 

5x5 degree grid with 5 year smoothing. Shown are results obtained from GHCN (blue), GHCNadjusted (yellow) and NCAR (green). Also shown is the CRUTEM3 line (black). The Y-axis is 10 x 

the anomaly in degrees. The X-axis is year. 

19. In Figure 6.2 we show the results of independent scientific studies, where 

individual researchers have brought to bear their own judgment on station 

selection and adjustments to the data. Such studies have been carried out by 

NASA-GISS and NOAA-NCDC using the GHCN dataset. The figure from the 

IPCC 4th Report Chapter 3 is reproduced below to show these in comparison to 

CRUTEM312. Once again there is good agreement. 

 

12 The reader is referred to the comprehensive submission from the UK Met Office (CRU 54) to the 

House of Commons Committee on Science and Technology Inquiry which shows a similar unsmoothed 

figure. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

48

Fig 6.2 Reproduced from the IPCC 4th Report Chapter 3. 

20. Finding: This simple analysis and the comparisons in figures 6.1 and 6.2 give rise 

to the following findings: 

Any independent researcher may freely obtain the primary station data. It is 

impossible for a third party to withhold access to the data. 

It is impossible for a third party to tamper improperly with the data unless they 

have also been able to corrupt the GHCN and NCAR sources. We do not 

consider this to be a credible possibility, and in any case this would be easily 

detectable by comparison to the original NMO records or other sources such 

as the Hadley Centre. 

The steps needed to create a global temperature series from the data are 

straightforward to implement. 

The required computer code is straightforward and easily written by a 

competent researcher. 

The shape of the temperature trends obtained in all cases is very similar: in 

other words following the same process with the same data obtained from 

different sources generates very similar results. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

49

21. By performing this simple test one determines easily that the results of the 

CRUTEM analysis follow directly from the published description of the method, 

and that the resultant temperature trend is not significantly different from the other 

results regardless of stations used or adjustments made. The test is therefore 

sufficient to demonstrate that, with respect to the declared method, the CRUTEM 

analysis does not contain either error or adjustments which are responsible for the 

shape of the resultant temperature trend. 

22. A researcher can evidently produce a study which would test the CRUTEM 

analysis quite precisely, without requiring any information from CRU to do so. 

23. Finding: The high level trial analysis shows that adjustments make little difference 

to the overall conclusions on temperature trends. However, this has been a matter 

about which much comment has been made and it is therefore discussed in more 

detail in Appendix 7. Adjustments were made by CRU to the data from 10% of 

stations. The Review Team is satisfied that this had no significant effect upon 

results. Very strong evidence for this is furnished by the level of agreement 

between CRUTEM and the other lines shown in Figure 6.1, as well as other 

studies detailed in the Appendix, showing the effect of de-adjusting and of 

removing the adjusted stations. 

24. It should be noted that in making these findings, the Review Team is making no 

statement regarding the correctness of any of these analyses in representing global 

temperature trends. We do not address any alleged deficiencies such as allowance 

for non climatic effects or the significant drop in station number post 1991. We do 

not address any possible deficiencies of the method. These are entirely matters for 

proper scientific study and debate and lie outside the scope of this Review. 

6.5 Checking Specific Details in the CRUTEM Analysis 

25. The goal here is to consider the requirements for carrying out an exact replication 

of the CRUTEM work to produce an identical gridded analysis. The work above 

demonstrates that this procedure is not necessary if the goal is simply to produce 

an alternative temperature trend analysis. As discussed in Appendix 7, and as 

follows from the close agreement between the lines in Figures 6.1 and 6.2, it is 

also not necessary to test the robustness of the CRUTEM3 analysis. However, the 

Review upholds the important principle that any scientific result should be open to 

such scrutiny for whatever reason. It therefore considered whether all the 

necessary information was made available. 

6.5.1 Identification of Data Sources 

26. In order to reproduce exactly a CRUTEM study, the independent researcher 

requires the exact list of stations used in order to source the primary data 

themselves. 

27. The original version of the CRUTEM gridded averages was published in 1986 (in 

this report we call this CRUTEM1986) using 3276 stations (314 of which were 

adjusted). Subsequent revisions were produced in 1994, 2003 and 2006, referred 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

50

to as CRUTEM1, 2 and 3. CRUTEM3 used 4138 stations (298 were adjusted). 

Whereas the 3276 stations used in CRUTEM1986 are listed in the respective 

references13, this is not the case for the additional approximately 1000 stations 

included in CRUTEM314. The additional stations are described only in general 

terms in the respective publication via references to the sources. 

28. For some years prior to the coming into force of the general right of access to 

information under FoIA, CRU had received requests for data, including station 

identifiers. An example of the attitude to these requests is given in the following 

e-mail extract: 

Jones to Mann on 2nd February 2005 (1107454306.txt): 

“And don't leave stuff lying around on ftp sites - you never know who is trawling 

them. The two MMs have been after the CRU station data for years. If they ever 

hear there is a Freedom of Information Act now in the UK, I think I'll delete the 

file rather than send to anyone. Does your similar act in the US force you to 

respond to enquiries within 20 days? - our does ! The UK works on precedents, 

so the first request will test it. We also have a data protection act, which I will 

hide behind. Tom Wigley has sent me a worried email when he heard about it - 

thought people could ask him for his model code. He has retired officially from 

UEA so he can hide behind that. IPR should be relevant here, but I can see me 

getting into an argument with someone at UEA who'll say we must adhere to it !”

29. A request was made to CRU for both station identifiers and access to the raw data 

for these stations. This was formally logged by the UEA Information Policy & 

Compliance Manager (IPCM) in January 2007 and was dealt with under the FoIA. 

The request was initially refused entirely on the grounds that raw station data was 

publicly available (from GHCN, NCAR and the NMOs), but without reference to 

providing station identifiers. Following further correspondence, an internal UEA 

appeal process and further prompting by the applicant, the list of the 4138 stations 

used in CRUTEM3 was finally released on 1st October. The issues that this raises 

 

13 Bradley, R.S., Kelly, P.M., Jones, P.D., Goodess, C.M. and Diaz, H.F., 1985: A Climatic Data Bank 

for Northern Hemisphere Land Areas, 1851-1980, U.S. Dept. of Energy, Carbon Dioxide Research 

Division, Technical Report TRO17, 335 pp. 

Jones, P.D., Raper, S.C.B., Santer, B.D., Cherry, B.S.G., Goodess, C.M., Kelly, P.M., Wigley, T.M.L., 

Bradley, R.S. and Diaz, H.F., 1985: A Grid Point Surface Air Temperature Data Set for the Northern 

Hemisphere, U.S. Dept. of Energy, Carbon Dioxide Research Division, Technical Report TRO22, 251 

Jones, P.D., Raper, S.C.B., Cherry, B.S.G., Goodess, C.M. and Wigley, T.M.L., 1986: A Grid Point 

Surface Air Temperature Data Set for the Southern Hemisphere, 1851-1984, U.S. Dept. of Energy, 

Carbon Dioxide Research Division, Technical Report TR027, 73 pp. 

Jones, P.D., Raper, S.C.B., Bradley, R.S., Diaz, H.F., Kelly, P.M. and Wigley, T.M.L., 1986: Northern 

Hemisphere surface air temperature variations: 1851-1984. Journal of Climate and Applied 

Meteorology 25, 161-179. 

Jones, P.D., Raper, S.C.B. and Wigley, T.M.L., 1986: Southern Hemisphere surface air temperature 

variations: 1851-1984. Journal of Climate and Applied Meteorology 25, 1213 1230 

14 Brohan, P., Kennedy, J., Harris, I., Tett, S.F.B. and Jones, P.D., 2006: Uncertainty estimates in 

regional and global observed temperature changes: a new dataset from 1850. J. Geophys. Res. 111, 

D12106, doi:10.1029/2005JD006548. 

Jones, P.D. and Moberg, A., 2003: Hemispheric and large-scale surface air temperature variations: An 

extensive revision and an update to 2001. J. Climate 16, 206-223. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

51

on compliance with the FoIA are dealt with in Chapter 10. 

30. The Review Team verified that matching of stations to a simple list of identifiers 

is not a straightforward process. The identification numbers are not unique across 

data sets, station names differ, and in the case of GHCN several replicas of the 

same data exist. This is primarily a problem caused by the lack of standardisation 

of metadata within the global climate science community. We make a 

recommendation on this below. 

31. The Review Team was able to match 90% of stations given in the CRU list to 

GHCN (see Appendix 7). CRU has stated in a written submission15 that that the 

remaining 10% can be obtained from other sources including the NMOs. Thus 

substantial work is required to take the CRU published list and assemble 100% of 

the primary station data from global repositories and NMOs. We make a 

recommendation for the future below. 

32. Finding: The Review finds that as a matter of good scientific practice, (and having 

established the precedent with CRUTEM1986) CRU should have made available 

an unambiguous list of the stations used in each of the versions of CRUTEM at 

the time of publication. In the absence of this, CRU was unhelpful and defensive 

and should have responded throughout to requests for this information in a more 

timely way. 

6.5.2 The Availability of Computer Codes 

33. Finding: The computer code required to read and analyse the instrumental 

temperature data is straightforward to write based upon the published literature. It 

amounts a few hundred lines of executable code (i.e. ignoring spaces and 

comments). Such code could be written by any research unit which is competent 

to reproduce or test the CRUTEM analysis. For the trial analysis of the Review 

Team, the code was written in less than two days and produced results similar to 

other independent analyses. No information was required from CRU to do this. 

 

34. The Met Office Hadley centre has published Practical Extraction and Reporting 

Language (PERL) codes which are a few hundred lines long. The submission by 

Barnes is also very helpful and illustrative in this context. 

6.6 Use of Local Temperature Data from China 

35. The above account of our work has focused on global data, its handling and 

accessibility. We also considered the implication in a submission to us by Dr 

Benny Peizer of the Global Warming Policy Foundation16 that Jones was 

complicit in malpractice in failing to respond appropriately to allegations of fraud 

made against a climate scientist at the State University of New York, Albany, 

Professor Wei-Chyung Wang in relation to more local data. An important paper 

 

15 Follow up request (20 April) to Professor Jones and response in relation to raw instrument station 

availability for each CRUTEM set 

16 Peizer/Global Warming Policy Foundation submission (no. 22), 28 February
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

52

on the effect of urbanization on temperature by Jones et al (1990)17 included 

data from China (Wang et al, 1990)18 that contained the statement that only 

instrument stations were selected where there had been few, if any, changes 

in instrumentation, location or observation times. It has been claimed by 

Keenan (2007)19 that this latter statement was knowingly untrue 

(“fabricated”), that stations had been moved, thus that their record of 

temperature change would be unreliable, and that hard copies of details of 

station histories had been lost or withheld. A subsequent paper by Jones et 

al. (2008)20 verified the original conclusions for the Chinese data for the 

period 1954–1983, showing that the precise location of weather stations was 

unimportant to the outcome 

36. Peizer, as Editor of Energy and Environment, to which the Keenan paper had 

been submitted, had contacted Jones asking him to comment on the Keenan 

paper, which Jones did, by denying its criticisms and advising rejection of the 

paper21. Jones was encouraged by colleagues to respond in writing to the 

journal, but chose not to do so "until the SUNY process has run its course."22

(The State University of New York, Albany, investigated the accusation of 

fraud against Wang, which it rejected23). Peizer has included some detail of 

email traffic on this issue in his submission to us.24

37. The allegation against Jones that is relevant to the Review25 is that since 

significant doubts about the reliability of Chinese climate data were raised, Jones 

has taken no public steps to clear up the discrepancies regarding Wang's claims 

and data, and that “it is unacceptable that the scientist who disseminates a data 

product on which international treaties are based, as well as IPCC reports and 

countless government policies, should actively seek to suppress information that 

calls the quality of the data into question”.

38. We note that, in at least a Nature interview referring to the loss of records26, Jones 

is reported as acknowledging “It‟s not acceptable...not best practice ...the stations 

„probably did move‟...I thought it was the right way to get the data. I was 

specifically trying to get more rural station data that wasn‟t routinely available in 

real time from [meteorological] services.” Although Jones chose not to respond in 

 

17 Jones P.D., Groisman P.Y., Coughlan M., Plummer N., Wang W.-C., Karl T.R. (1990), ―Assessment 

of urbanization effects in time series of surface air temperature over land‖, Nature, 347: 169–172. 

18 Wang W.-C., Zeng Z., Karl T.R. (1990), ―Urban heat islands in China‖, Geophysical Research 

Letters, 17: 2377–2380.

19 Keenan, D.J. Fraud allegation against some climatic research of Wei-Chyung Wang. Energy and 

Environment, 18 (7-8), 985-995. 

20 Jones, P.D., Lister, D.H. and Li, Q. 2008. Urbanization effects in large-scale temperature records, 

with an emphasis on China. Journal of Geophysical Research, 113 (D16122), 12pp. 

21 Ibid. 16 

22 Ibid. 16 

23 http://www.informath.org/apprise/a5620.htm. This site included a letter to Keenan from the State 

University of New York, Albany, that states that ―The Investigation Committee finds no evidence of 

the alleged fabrication of results and nothing that rises to the level of research misconduct having been 

committed by Dr. Wang.‖ We are not aware of a published version of this report. 

24 Ibid. 16 

25 Ibid. 16 

26 Nature News, published online 15 February 2010 

http://www.nature.com/news/2010/100215/full/news.2010.71.html 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

53

writing when Keenan‟s claim was made, he did respond through an analysis in a 

paper in a front rank peer-reviewed journal only one year later that verified the 

conclusions of the earlier work27. By that time the SUNY investigation had 

concluded in Wang‟s favour.

6.7 Conclusions and Recommendations 

39. In summary, with regard to the allegations concerning the temperature data, the 

conclusions of the Review Team are as follows: 

Regarding data availability, there is no basis for the allegations that CRU 

prevented access to raw data. It was impossible for them to have done so. 

Regarding data adjustments, there is no basis for the allegation that CRU made 

adjustments to the data which had any significant effect upon global averages 

and through this fabricated evidence for recent warming. 

We find that CRU was unhelpful in dealing with requests for information to 

enable detailed replication of the CRUTEM analysis. 

Crucially, we find nothing in the behaviour on the part of CRU scientists that 

is the subject of the allegations dealt with in this Chapter to undermine the 

validity of their work. 

40. Reflecting the analysis in Appendix 7, the Review has the following 

recommendations: 

It would benefit the global climate research community if a standardised way 

of defining station metadata and station data could be agreed, preferably 

through a standards body, or perhaps the WMO. We understand that this is not 

straightforward and could be a lengthy process, but the process should start 

now. As example an xml based format would make the interpretation use, 

comparison, and exchange of data much more straightforward. 

Without such standardisation there will always be problems in issuing 

unambiguous lists, and assembling primary data from them. It would be in the 

public interest if CRU and other such groups developed a standard process to 

capture and publish a snapshot of the data used for each important publication.

 

 

27 Ibid. 20 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

54

CHAPTER 7: TEMPERATURE RECONSTRUCTIONS 

FROM TREE RING ANALYSIS

7.1 Background 

1. This Chapter considers the criticisms that have been made of CRU scientists in 

relation to their work on obtaining and analysing tree ring data to reconstruct land 

temperature records back over thousands of years, and in relation to the use made 

of that work in the IPCC. 

2. Data is recorded from tree core samples from many sites around the globe. These 

data comprise measurements of width and density taken from the cores and do not 

in themselves represent temperature directly. The processes of assembling and 

standardising the tree data, into what are termed tree ring chronologies, are 

separate from the later stage of producing temperature reconstructions. 

3. Temperature reconstructions are obtained through regression models which are 

calibrated against instrument temperature data during periods of overlap. A 

temperature reconstruction from any specific region does not in itself indicate 

global temperature patterns. Tree and other proxy reconstructions are combined 

to produce reconstructions pertaining to a wider area, and such reconstructions 

from many authors may be aggregated in order to provide an indication of global 

patterns. Even so the coverage is not as extensive as that of instruments. 

4. The uncertainty associated with temperature reconstructions in the distant past is 

much larger than that of present day instrument records. There is a wide and 

healthy debate in the published literature on all aspects of this. 

5. CRU scientists have developed temperature reconstructions based on a range of 

tree ring series. The current leader in CRU is Professor Briffa. Their work is 

extensively described in the relevant scientific literature. It also contributes to the 

development of a global picture such as that produced by IPCC in its 4th

Assessment Report (Working Group 1, Chapter 6, Palaeoclimate). In particular 

this contained conclusions on the likelihood that the second half of the 20th

Century was the warmest 50 year period in the last 500 or 1300 years in the 

Northern Hemisphere (different statements of probability are made for each – see 

paragraph 20).

7.2 The Allegations 

6. The release of the e-mails triggered the expression of a range of criticisms (many 

of long standing) of the work of CRU scientists. There was a clear implication 

that in carrying out their work, both the choices they made of data and the way it 

was handled, were intended to bias the scientific conclusions towards a specific 

result and to set aside inconvenient evidence. The overall implication is that this 

work is misleading and should not be trusted. 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

55

7. We address the following specific allegations which were explicit or can be 

inferred from the comments made on the e-mails and from the submissions we 

received. 

8. In relation to IPCC:

That the CRU work was ―flawed‖ and yet had a distinct influence on the 

balance of judgements made by IPCC Working Group 1 in Chapter 6 of their 

4

th Assessment Report, and therefore that less confidence should be ascribed 

to the conclusions reached by the authors, mentioned in paragraph 5 above. 

The criticism here is often captured by the proposition that today‘s 

temperatures are not unusual compared to the MWP. 

That Yamal and other chronologies constructed by CRU are unrepresentative 

of temperature trends (in recent years), and had an undue influence on all of 

the lines appearing in Chapter 6 of the 4th IPCC Report. 

That a majority of the reconstructions would look significantly different if 

certain component series were replaced with others, and that if this were done 

then the conclusions reached in respect of the likelihood associated with 

ranking of recent warmth with respect to the past would be significantly 

different. 

9. In relation to divergence:

That the phenomenon of ―divergence‖ (see discussion below) may not have 

been properly taken into account when expressing the uncertainty associated 

with reconstructions; and that this was not addressed with sufficient openness 

in IPCC reports. 

That the reference in a specific e-mail to a ―trick‖ and to ―hide the decline‖ in 

respect of a 1999 WMO report figure show evidence of intent to paint a 

misleading picture. 

10. In relation to withholding data:

That access to tree ring data from the Yamal peninsula in Siberia (used by 

Professor Briffa) was withheld, and that as a particular consequence of this the 

small sample size of the Yamal series in the most recent years was not 

acknowledged. 

11. In relation to mishandling data:

That improper manipulations have been performed – specifically with respect 

to the Yamal and Tornetrask tree series. 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

56

7.3 Findings 

7.3.1 IPCC Reports 

12. The IPCC reports are at the centre of allegations of presenting misleading 

messages produced by CRU. We concentrate upon AR4 as it is the most recent 

and most comprehensive. 

13. On the replacement of series in reconstructions, much of the critical material 

submitted presents several examples of differing results obtained from a tree 

series (A) compared to another tree series (B) in the same region. This is to be 

expected, and taken in isolation has no implication for the validity of either. To 

know whether replacing one with another would have a significant effect upon 

statements regarding Northern Hemisphere temperature series, would require the 

fulfilment of several conditions: 

That it has been proven that series B is more representative over the whole 

period than series A. 

That the difference between series A and B is important in a temperature 

reconstruction in which it is included. 

That if A were replaced with B in each of the reconstructions in which A is 

included then the conclusions drawn in respect of the likely ranking of the 

past to the present would change significantly when considering all 

reconstructions together (including those in which it does not appear). 

To support an allegation of improper conduct would require that the scientists 

involved, knowing the above, still use and promote non-representative series A in 

order to reach improperly a false conclusion. 

14. Finding: We are unaware of any analysis to demonstrate that any of the above 

conditions are fulfilled for Yamal or any of the series cited in relation to CRU 

work (i.e. Tornetrask, Taymir). The Review is naturally aware that partial studies 

and comments referring to CRU‘s published work appear elsewhere. However 

these criticisms of CRU‘s work are not in peer reviewed journals, and we have not 

found that these are anywhere assembled into a coherent, comprehensive and 

scrutinised case which demonstrates the proposition in respect of any of the series 

cited. 

15. Finding: To make the case that replacing series in reconstructions calls in question 

the validity of the picture painted in Chapter 6 of the IPCC report (or to sustain a 

charge of impropriety on the part of its many authors) would require that all the 

conditions in paragraph 13 were met. No evidence of this has been presented to 

the Review. 

16. Finding: The influence of the Yamal series of Briffa 20001

 is often called into 

question. The conditions in paragraph 13 above have not been demonstrated 

 

1

 Briffa, K. R. 2000. Annual climate variability in the Holocene: interpreting the message of ancient 

trees. Quaternary Science Reviews 19:87-105
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

57

anywhere which has been brought to our attention. This series is used in none of 

the IPCC TAR and only 42out of 12 reconstructions in AR4. 

17. We now turn to the question of uncertainty. There are multiple sources of 

uncertainty in respect of proxy temperature reconstructions and they are the 

subject of an ongoing and open scientific debate. 

18. Elements of that debate are touched on in the e-mails. For example, the following 

e-mails from Briffa show some concern to ensure that uncertainty was properly 

recognised. 

E-mail dated 29/4/03 (1051638938.txt): 

“Can I just say that I am not in the MBH camp - if that be characterized by an 

unshakable "belief" one way or the other , regarding the absolute magnitude 

of the global MWP. I certainly believe the " medieval" period was warmer 

than the 18th century - the equivalence of the warmth in the post 1900 period, 

and the post 1980s, compared to the circa Medieval times is very much still an 

area for much better resolution. I think that the geographic / seasonal biases 

and dating/response time issues still cloud the picture of when and how warm 

the Medieval period was . On present evidence , even with such uncertainties I 

would still come out favouring the "likely unprecedented recent warmth" 

opinion - but our motivation is to further explore the degree of certainty in this 

belief - based on the realistic interpretation of available data.”

E-mail dated 3/2/06 (1138995069.txt): 

“we are having trouble to express the real message of the reconstructions - 

being scientifically sound in representing uncertainty , while still getting the 

crux of the information across clearly...We have settled on this version 

(attached) of the Figure which we hoe [sic] you will agree gets the message 

over but with the rigor required for such an important document.”

19. What is clear is that the uncertainty associated with any estimate of past 

temperatures from reconstructions is much larger than that of recent instrument 

temperature data. This is demonstrated in the figure below taken from IPCC AR4 

Figure 6.10. The middle plot shows the variability both within and between 

different reconstructions each using an aggregation of proxy data (i.e. several tree 

and non-tree proxies). The lower plot gives an indication of uncertainty bands 

from those reconstructions. Simply looking at any individual reconstruction line 

alone makes only a partial statement about Northern Hemisphere temperatures 

with a large uncertainty. It is obviously even less meaningful to look at an 

individual tree series in isolation. 

 

2

 A fifth reconstruction is disputed. 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

58

Figure 7.1: From IPCC AR4 (Figure 6.10) 

20. Understanding requires proper statistical interpretation, i.e. to determine the 

confidence level associated with a statement such as “the present is likely warmer 

than the past”. To do this as objectively as possible would require a complex (and 

difficult) study to perform hypothesis testing in a mathematically rigorous way, 

taking proper account all of the uncertainties and their correlations. We are not 

aware that this has been done in the production of IPCC reports to date, but 

instead qualitative statements have been made based on definitions of ―likely‖, 

―very likely‖ etc according to criteria laid down by the IPCC ( ̳Likely‘ means a 

probability greater than 66%, and  ̳Very Likely‘ means a probability greater than 

90%). 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

59

21. Finding: We do not find that the data described in IPCC AR4 and shown in Figure 

6.10 is misleading, and we do not find that the question marks placed over the 

CRU scientists‘ input casts doubt on the conclusions. In particular: 

The variation within and between lines, as well as the depiction of 

uncertainty is quite apparent to any reader. 

It presents all relevant published reconstructions we are aware of, i.e. there 

has been no exclusion of other published temperature reconstructions 

which would show a very different picture. 

The general discussion of sources of uncertainty in the text is extensive, 

including reference to divergence and it therefore cannot be said that 

anything has been suppressed. Presenting uncertainty in this way is a 

significant advance on the TAR. 

7.3.2 Divergence 

22. Divergence: The phenomenon of divergence is an ongoing research area which is 

discussed widely in the literature and the submissions. Tree core data yields both 

width and density measurements and both of these are used to reconstruct 

temperature records. In some cases the temperature reconstruction from one or 

other of these will track instrumental temperature measurements on a short 

timescale, but diverges on a longer timescale. The phenomenon of divergence is 

not observed in all tree series. There is as yet no complete understanding of the 

source of divergence although studies are suggesting possible contributory effects. 

23. Finding: The question of whether divergence has been adequately taken into 

account when estimating uncertainties is a complex and ongoing question of 

science and is therefore outside the scope of this Review. We have however 

investigated this matter enough to be satisfied that it is not hidden and that the 

subject is openly and extensively discussed in the literature, including CRU 

papers. 

24. The WMO 1999 front cover figure: The allegation is that the figure appearing on 

the front cover of the WMO report of 1999 (Statement on the Status of Global 

Climate, 19993

) did not disclose that it combined proxy and temperature records 

and failed to show a declining proxy line due to divergence; and that this was 

done to deceive. 

Jones‘ e-mail on 16/11/99 (942777075.txt) 

“I've just completed Mike's Nature trick of adding in the real temps to each series 

for the last 20 years (ie from 1981 onwards) and from 1961 for Keith's to hide the 

decline.”

is perhaps the prime example of the fact that opposing interpretations can be 

obtained from the same statement. The word ―trick‖ has been widely taken to 

 

3

 http://www.wmo.ch/pages/prog/wcp/wcdmp/statement/wmo913.pdf 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

60

confirm the intention to deceive, but can equally well, when used by scientists, 

mean for example a mathematical approach brought to bear to solve a problem. It 

is the latter explanation that Jones has given the Review, quoted in paragraph 6 of 

Chapter 4. 

25. The WMO report is a short document produced annually. It does not have the 

status or importance of the IPCC reports. The figure in question was a frontispiece 

and there is no major discussion or emphasis on it in the text. The caption of the 

figure states: ―Northern Hemisphere temperatures were reconstructed for the past 

1000 years (up to 1999) using palaeoclimatic records (tree rings, corals, ice 

cores, lake sediments, etc.), along with historical and long instrumental records”.

26. Finding: In relation to ―hide the decline‖ we find that, given its subsequent iconic 

significance (not least the use of a similar figure in the TAR), the figure supplied 

for the WMO Report was misleading in not describing that one of the series was 

truncated post 1960 for the figure, and in not being clear on the fact that proxy and 

instrumental data were spliced together. We do not find that it is misleading to 

curtail reconstructions at some point per se, or to splice data, but we believe that 

both of these procedures should have been made plain – ideally in the figure but 

certainly clearly described in either the caption or the text. 

7.3.3 Withholding Data

27. Yamal - withholding data and information on sample size: This issue concerns 

the Briffa 20004 and Briffa 20085 publications. The underlying raw data were not 

archived at the time of publication. Since it was not owned by CRU, they had no 

right to archive it. The data were owned by others including Hantemirov and 

Shiyatov. CRU has stated that it directed the request it received for data to the 

owners. Whether it was a result of this, or otherwise, the requester was given a 

copy

6

 of the data in 2004 by Hantemirov. Later, following publication of Briffa 

20087

, and again following requests for the data, CRU has stated that it asked the 

owners to archive the data, which they did. 

28. Because the raw data was not archived, the information on the number of core 

counts used in each year was not easily available. As a result the reader could not 

know that the core count in the most recent years had dropped substantially. This 

fact grew in importance as the need to understand uncertainty grew. The 

information was actually published and in the public domain in 20028

 but it is 

arguable that a user of the CRU Yamal chronology would not have known to look 

 

4

 Ibid. 1 

5

 Briffa, K. R., Shishov, V.V., Melvin, T. M., Vaganov, E. A., Grudd, H., Hantemirov, R. M., Eronen, 

M., and Naurzbaev, M. M. 2008. Trends in recent temperature and recent tree growth spanning 2000 

years across Northwest Eurasia. Philosophical Transactions of the Royal Society B-Biological Sciences 

363:2271-2284 

6

 It has also been pointed out to us by CRU that McIntyre acknowledged this in a comment (number 

61) made underneath a Climate Audit post which states that McIntyre did not realize that this was the 

series used by Briffa when he received it. http://climateaudit.org/2009/10/05/yamal-and-ipcc-ar4-

review-comments/. 

7

 Ibid. 5 

8

 Hantemirov, R. M., and Shiyatov, S. G. 2002. A continuous multimillenial ring-width chronology in 

Yamal, northwestern Siberia. Holocene 12:717-726 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

61

there unless explicitly directed. Hantemirov and Shiyatov 2002 was referenced in 

Briffa 20089.

29. Finding: It is evidently true that access to the raw data was not simple until it was 

archived in 2009 and this can rightly be criticized on general principles of 

transparency, although it may have been common practice at the time of the 

original publication. We find that CRU has not withheld the data (having correctly 

directed the single request to the owners). However, in the interests of 

transparency, we believe that CRU should have ensured that the data it did not 

own, but had relied upon in publications, was archived in a more timely way. 

This is another example where a more open approach would have been helpful. 

7.3.4 Mishandling Data 

30. We have not focussed upon disagreements over comparisons of results using 

individual tree series for two reasons. First, because this is a science question and 

should be addressed through the normal channels of peer reviewed publication. 

Secondly, because as we have indicated above, it requires much more work to 

show that one is unequivocally more representative than the other, and even then 

requires the conjunction of several additional conditions to lead to an allegation of 

wrongdoing. We have addressed these conditions above, and we have found no 

evidence to substantiate them. Nevertheless we comment briefly upon Yamal as it 

has received so much attention and the Tornetrask series as it is subject of much 

misunderstanding. 

31. Finding on Yamal: The Briffa 2000

10 paper presents a collection of many 

chronologies (Yamal is simply one of them) assembled from many authors. It is

quite clear that Briffa simply reprocessed data from other authors (Hantemirov 

and Shiyatov in this case) and no selection was performed. In their submission to 

the Review CRU included a copy of work they had made available on the web11. 

This was posted in response to criticisms published at Climate Audit12 which 

described examples of variability between specific tree series in the region. This 

CRU response demonstrates the consistency of the original Yamal publication 

with that obtained using an updated analysis incorporating a comprehensive set of 

contemporary data. The Review is not aware of any equivalent comprehensive 

analysis which demonstrates that the conclusions of Briffa 2000 are inconsistent 

with the best analysis available today. 

32. Finding on ―Bodging‖ in respect of Tornetrask. The term ―bodging‖ has been 

used, including by Briffa himself, to refer to a procedure he adopted in 199213. 

The  ̳bodge‘ refers to the upward adjustment of the low-frequency behaviour of 

the density signal after 1750, to make it agree with the width signal. This ad hoc 

process was based on the conjecture that the width signal was correct. There is 

nothing whatsoever underhand or unusual with this type of procedure, and it was 

 

9

 Ibid. 5 

10 Ibid. 1 

11 http://www.cru.uea.ac.uk/cru/people/briffa/yamal2009/ 

12 http://climateaudit.org/2009/09/27/yamal-a-divergence-problem/ 

13 Briffa, K. R., Jones, P. D., Bartholin, T. S., Eckstein, D., Schweingruber, F. H., Karlen, W., 

Zetterberg, P., and Eronen, M. 1992. Fennoscandian Summers from AD 500: temperature changes on 

long and short timescales. Climate Dynamics 7:111-119. 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

62

fully described in the paper. The interpretation of the results is simply subject to 

this caveat. The conjecture was later validated14 when it was shown to be an effect 

due to the standardisation technique adopted in 1992. Briffa referred to it as a 

―bodge‖ in a private e-mail in the way that many researchers might have done 

when corresponding with colleagues. We find it unreasonable that this issue, 

pertaining to a publication in 1992, should continue to be misrepresented widely 

to imply some sort of wrongdoing or sloppy science. 

7.4 Conclusions and recommendations 

33. We do not find that the data described in AR4 and shown in Figure 6.10 is 

misleading, and in particular we do not find that the question marks placed 

over the CRU scientists’ input cast doubt on the conclusions. 

34. The variation within and between lines, as well as the depiction of uncertainty is 

quite apparent to any reader. All relevant published reconstructions of which we 

are aware are presented, and we find no evidence of exclusion of other published 

temperature reconstructions which would show a very different picture. The 

general discussion of sources of uncertainty in the text is extensive, including 

reference to divergence and it therefore cannot be said that that anything has been 

suppressed. Presenting uncertainty in this way is a significant advance on the 

TAR.

35. We have seen no evidence to sustain a charge of impropriety on the part of CRU 

staff (or the many other authors) in respect of selecting the reconstructions in AR4 

Chapter 6. This would require that all the conditions in paragraph 13 were met in 

respect of tree chronologies either used by, or created by, CRU. No evidence of 

this has either been presented to the Review, nor has it been assembled as a 

scientific study published elsewhere and subjected to scrutiny. For the same 

reasons we found no evidence that there is anything wrong with the CRU 

publications using the Yamal or other tree series. 

36. We find that divergence is well acknowledged in the literature, including CRU 

papers. 

37. In relation to ―hide the decline‖ we find that, given its subsequent iconic 

significance (not least the use of a similar figure in the IPCC TAR), the figure 

supplied for the WMO Report was misleading in two regards. It did not make 

clear that in one case the data post 1960 was excluded, and it was not explicit on 

the fact that proxy and instrumental data were spliced together. We do not find 

that it is misleading to curtail reconstructions at some point per se, but that the 

reason for doing so should have been described. 

38. We find that CRU has not withheld the underlying raw Yamal data (having 

correctly directed the single request to the owners). But it is evidently true that 

access to the raw data was not simple until it was archived in 2009 and this can 

rightly be criticised on general principles. In the interests of transparency, we 

believe CRU should have ensured that the data they did not own, but had relied 

upon in publications, was archived in a more timely way. 

 

14 Briffa KR and Melvin TM (2010 in press) 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

63

39. It is a matter for the IPCC Review to determine whether the conclusions were in 

line with IPCC processes and guidelines for levels of likelihood. In respect of that 

Review we offer the suggestion that putting the combination of different 

reconstructions upon a more rigorous statistical footing would help in the future to 

make confidence levels more objective. 
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

64

CHAPTER 8: PEER REVIEW AND INFLUENCING 

EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

8.1 Background: Peer Review, Testing and Verification 

1. The rigorous development of scientific ideas leans heavily on peer-reviewed 

publication in scientific journals that are subject to testing and challenge by other, 

published, scientifically derived evidence. 

2. The processes whereby new scientific concepts are peer-reviewed and published 

can be complex, contentious and partial (see Chapter 5 and Appendix 5). Peer 

review is not a ―gold standard‖ that ensures validity, as some claim. The processes 

of testing, validation and replication are the means whereby error is discarded, 

uncertainty reduced and robust scientific progress made. Peer review attempts 

initial sieving-out of the demonstrably erroneous or trivial, thereby setting a high 

standard that encourages rigour. But it is not infallible. Many well-founded 

concepts are rejected and many erroneous ideas accepted. 

3. If peer review is subverted to exclude apparently well-founded alternative views, 

or if journal editors are intimidated from considering their publication, progress on 

an important issue such as climate change can be seriously slowed or skewed. 

8.2 The Allegations 

4. The broad allegation is that CRU made improper attempts to influence the peer 

review system, pressuring journals to reject submitted articles that did not support 

a particular view of climate change. 

5. This can only be properly assessed by analysis of the individual cases cited. 

Specific examples used to support this allegation are the events surrounding the 

publication of a paper by Soon and Baliunas (2003)1, the issues relating to Dr 

Boehmer-Christiansen, editor of the journal Energy and Environment, and certain 

actions of Professor Briffa when he was editor of the Journal Holocene.

8.3 The Soon and Baliunas Affair & Climate Research

6. The paper by Soon and Baliunas (2003) entitled Proxy climatic and environmental 

changes of the past 1000 years, published in the journal Climate Research, 

reviewed 240 previously published papers on temperature trends during the last 

millennium. It challenged the conclusion of Mann et al (1998, 1999)2 that the late 

20th century was the warmest period of the last millennium on a hemispheric 

scale, and claimed that recent temperatures were by no means unprecedented over 

 

1

 Soon, W. and Baliunas, S. 2003. Proxy climatic and environmental changes of the past 1000 years. 

Climate Research 23, 89-110.

2 Mann ME, Bradley RS, Hughes MK (1998) Global-scale temperature patterns and climate forcing 

over the past six centuries. Nature 392:779–787. 

Mann ME, Bradley RS, Hughes MK (1999) Northern Hemisphere temperatures during the past 

millennium: inferences, uncertainties, and limitations. Geophys Res Lett, 26:759 – 762.
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

65

this period. It was greeted with enthusiasm by those sceptical of the hypothesis of 

anthropogenic global warming3. However it received a negative reception from 

many other climate scientists4 on scientific grounds, viz. that it conflated 

qualitative data on temperature and precipitation from many sources that could not 

be combined into a consistent proxy record. That hostility is reflected in the 

released CRU e-mails (e.g. CRU 1051156418.txt, 1051202354.txt), and the 

following e-mail from Jones to Mann on 11.3.03 (1047388489.txt): 

“I think the skeptics will use this paper to their own ends and it will set paleo back 

a number of years if it goes unchallenged. I will be emailing the journal to tell 

them I'm having nothing more to do with it until they rid themselves of this 

troublesome editor (de Freitas), a well-known skeptic in NZ. A CRU person is on 

the board but papers get dealt with by the editor assigned by Hans von Storch." 

7. The S&B paper had been seen by four reviewers, none of whom had 

recommended rejection5, and had been accepted by de Freitas (one of 10 Climate 

Research review editors; papers could be submitted to any one of them). A 

number of review editors resigned as a reaction against the publication of what 

they regarded as a seriously flawed paper. The journal‘s publisher admitted that 

the journal should have requested appropriate revisions of the manuscript prior to 

publication6

. The Editor in Chief resigned on being refused permission by the 

publisher to write an editorial about what he regarded as a failure of the peer 

review system. De Freitas was said to have described these events as ―a mix of a 

witch-hunt and the Spanish Inquisition‖7

.

8. These events, and the e-mail quote in paragraph 6, have led to the allegation that 

normal procedures of publications were being improperly undermined by a group 

that included Jones8.

9. Jones has responded in evidence9 to us that the reaction to the S&B paper was not 

improper or disproportionate given what he saw as the self evident errors of the 

paper. The arguments presented in the Eos article, if correct, are strongly put, and 

suggest that the reaction was based on a belief, for which evidence was adduced, 

that the science was poor. In light of the reaction of the Journal‘s publisher, we do 

not believe that any criticism of Jones can be justified in this regard. 

10. Finding: This was clearly a bruising experience for all concerned. But Richard 

Horton‘s paper (Appendix 5) and the comments on it in Chapter 5 suggest to the 

Team that this scale of reaction is not unusual in contested areas, and the peer 

review process does not provide insulation from it. The Review makes no 

judgement about the correctness or otherwise of the Soon and Baliunas paper, but 

we conclude that the strong reaction to it was understandable, and did not amount 

to undue pressure on Climate Research. 

 

3 New York Times, 5 August 2003. 

4

 e.g. Eos, Vol. 84, No. 27, 8 July 2003.

5

 http://www.sgr.org.uk/climate/StormyTimes_NL28 htm 

6

 A statement from Inter-Research, 2003. Climate Research, 24, 197-198

7

 http://www.sgr.org.uk/climate/StormyTimes_NL28 htm 

8

 Montford, A.W. 2010. The Hockey Stick Illusion. Stacey International, London. 

9

 See the note of the CCER meetings held on 9 April 2010 
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

66

8.4 The Conflict with Dr Boehmer-Christiansen 

11. Dr Boehmer-Christiansen is Reader Emeritus in Geography at the University of 

Hull and the editor of the scientific journal Energy and Environment. As part of 

her evidence to us, she submitted a copy of her evidence to the House of 

Commons Science and Technology Committee Select Committee for its inquiry 

into ―The disclosure of climate data from CRU at the UEA‖. In it she wrote10

: 

 

“As editor of a journal which remained open to scientists who challenged the 

orthodoxy, I became the target of a number of CRU manoeuvres. The hacked 

emails revealed attempts to manipulate peer review to E&E‟s disadvantage, and 

showed that libel threats were considered against its editorial team. Dr Jones 

even tried to put pressure on my university department. The emailers expressed 

anger over my publication of several papers that questioned the „hockey stick‟ 

graph and the reliability of CRU temperature data. The desire to control the peer 

review process in their favour is expressed several times”.

12. Apart from the allegation of  ̳attempts to manipulate peer review‘ for which 

Boehmer-Christiansen presents no evidence in relation to the journal that she 

edits, her statement11 to us implies that pressure on her University was designed to 

undermine her role as editor of Energy and Environment. She comments that ―a 

message was send to my head of department late last year by Phil Jones in 

relation to some other matter, which suggested that the University of Hull should 

dissociate itself from me as editor of Energy & Environment (Multi-Science) as I 

was causing difficulties for CRU.‖ 

13. This episode appears to have been precipitated by an email sent on 2 October 

2009 (1254746802.txt) from Boehmer-Christiansen to Stephanie Ferguson at the 

UK Climate Impacts Programme, and copied to a number of others. Its title was: 

“Please take note of potetially [sic] serious scientific fraud by CRU and Met 

Office.” The email was brought to the attention of Jones by the ―help desk‖ of 

UKCP09 in the UK Department for Environment, Food and Rural Affairs 

(DEFRA). He then wrote an e-mail on 27 October 2009 (1256765544.txt) to 

Professor Graham Haughton, Boehmer-Christiansen‘s Head of Department, 

complaining about the e-mail as ―very malicious”. Haughton responded 

sympathetically, but commented that “I‟d want to protect another academic‟s 

freedom to be contrary and critical”, to which Jones replied: “I don‟t think there 

is anything more you can do. I have vented my frustration and have had a 

considered reply from you”. 

14. Finding: We see nothing in these exchanges or in Boehmer-Christiansen‘s 

evidence that supports any allegation that CRU has directly and improperly 

attempted to influence the journal that she edits. Jones‘ response to her accusation 

of scientific fraud was appropriate, measured and restrained. 

 

10 Memorandum submitted by Dr Sonja Boehmer-Christiansen (CRU 26), at 4.1 

11 Dr Sonja Boehmer-Christiansen submission (no.43), 23 February 
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

67

8.5 Peer Review and Professor Briffa‟s Editorship of 

Holocene

15. An e-mail (1054748574.txt) was sent by Briffa, as Editor of the journal Holocene, 

on 4.06.03 to Ed Cook, a reviewer, as follows: 

“I am really sorry but I have to nag about that review – Confidentially I now need 

a hard and if required an extensive case for rejecting – to support Dave Stahle‟s 

and really as soon as you can.” 

This has been assumed to be an editor who wishes to have the paper rejected, for 

reasons that are not explicit, seeking help from a reviewer to do so. In general, 

reviewers are not, or should not be invited to reject, merely to review. This e-mail 

has been widely interpreted as Briffa perverting the purpose of peer review by 

asking a colleague for help in rejecting a paper that contained research findings 

contradictory to his own views12. 

16. However, put in context, the e-mail‘s significance changes. The reality is reflected 

in a series of emails provided by Briffa13 and which have been redacted to remove 

names, apart from that of Stahle, as follows: 

17.06.02 – Briffa requests references from Dr Dave Stahle and referee B for a 

paper submitted to Holocene on the use of a specific tree ring record. 

07.08.02 – Stahle submits a referee report to Briffa. It is strongly critical of 

some of the underlying analysis, though it comments that the paper is well 

written. Advises rejection but suggests that if substantial additional work were 

done, it could be suitable for publication. 

09.08.02 – Briffa to Stahle, thanking him for the review, and committing 

himself to reading the paper very carefully in anticipation of receipt of a 

second review. 

28.05.03 – Briffa to referee B, pressing to send a second review. 

04.06.03 – Briffa reiterates his request to referee B, sending the email copied 

in paragraph 15.

04.06.03 – Referee B sends review to Briffa. In the event, the review is not as 

negative as that of Stahle: the paper is ―marginal at best, could justifiably be 

rejected‖. This is an immediate reply to the email copied in paragraph 15.

23.07.03 – One of the authors e-mails to Briffa enquiring about a decision on 

the paper. 

24.07.03 – Briffa to author. Apologises for the delay (―the manuscript could 

have been dealt with much better‖) and encloses referees‘ comments. Offers 

possible fast-track publication if the referees comments could be dealt with. 

We can find no evidence that the article was subsequently published but the 

evidence above demonstrates that the possibility of publication was not 

rejected. 

 

12 e.g. Channel 4 News 3.2.10 

13 Copies of Communications relating to Keith Briffa‟s editorial treatment of a submitted manuscript. 

Available at www.cce-review.org/Evidence.php 
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

68

17. Finding: Although much has been made of the e-mail in paragraph 15 as evidence 

of an unprincipled approach to the role of editor, we see nothing in these 

exchanges that supports the interpretations of subverting the peer review process 

that have been placed upon it. It appears to reflect an Editor with a strongly 

negative review in hand, and who presumably has read the paper, asking for 

confirmation that the paper should be rejected, possibly to reduce one of the many 

complications that assail an editor; and in view of the delay in communicating to 

authors, hoping for a strong decision from the referee. On receiving a second, 

more equivocal review, he offers the authors the opportunity to re-submit. These 

exchanges illustrate some of the complications of an Editor‘s life as referred to in 

Chapter 5 and in Appendix 5. They do not provide evidence of subversion of 

process in rejecting contradictory ideas as has been alleged. 

8.6 Conclusions 

18. In our judgement none of the above instances represents subversion of the peer 

review process nor unreasonable attempts to influence the editorial policy of 

journals. It might be thought that this reflects a pattern of behaviour that is partial 

and aggressive, but we think it more plausible that it reflects the rough and tumble 

of interaction in an area of science that has become heavily contested and where 

strongly opposed and aggressively expressed positions have been taken up on both 

sides. The evidence from an editor of a journal in an often strongly contested area 

such as medicine (Appendix 5) suggests that such instances are common and that 

they do not in general threaten the integrity of peer review or publication. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

69
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

70

CHAPTER 9: COMMUNICATING INTO THE PUBLIC 

DOMAIN THROUGH THE IPCC 

9.1 Background

1. Where scientific results and concepts are relevant to issues of public or policy 

interest, their complexities are often such that they need to be communicated in 

simpler, comprehensible language that includes explicit statements about 

uncertainties and errors. 

2. The IPCC process and its assessment reports have been the prime routes through 

which the complexities of climate change are transmitted into the international 

governmental domain. Actions that undermined the rigour, honesty and 

expressions of scientific uncertainty in such communications would be serious 

infringements of good scientific practice. 

9.2 The Allegations 

3. The above principles are the frame for allegations levelled at CRU that relate to 

the Review‘s remit. In broad terms the claim is that CRU attempted to prevent 

proper consideration of views which conflicted with their own through their roles 

in the IPCC. 

4. If this is the case it would represent a failure to represent current scientific 

understanding impartially at the vital interface between science and policy. If this 

has been done, it has the potential to understate uncertainty, and to undermine the 

rigour of risk-based evaluations by government, with the potential for severe 

social and economic consequences. 

5. There are two specific, analogous allegations. They primarily relate to papers that 

challenge important elements of IPCC work; the first about the interpretation of 

the CRUTEM instrumental temperature series and the second about the 

reconstruction of tree ring proxy temperature series, discussed earlier in different 

contexts in Chapters 6 and 7 respectively. It is alleged that these papers were not 

properly considered by the relevant IPCC writing groups, and that members of 

CRU played the major role in ensuring that this was not done. 

6. The following approach is adopted for each of these two sets of allegations: 

summary of the scientific basis of the challenge; 

allegations about the improper treatment of the papers by CRU members in 

their roles in IPCC writing groups; 

evidence in support of the allegations; 

the response of the CRU members alleged to have been primarily involved; 

evidence from the IPCC Review Editors (see Chapter 5, section 5.7) about the 

procedures of the relevant IPCC writing groups; 

findings of the Review. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

71

9.3 The CRUTEM Temperature Series 

9.3.1 The Scientific Challenge 

7. A paper by McKitrick & Michaels (2004, hereafter MM2004)1

argued that a large 

proportion of the measured late 20th century warming was a consequence of 

increased economic activity and that many meteorological measurement sites had 

become increasingly influenced by the warming effect of urbanisation around 

them, the so-called ―heat island effect‖, and that the CRUTEM data set was not 

adequately adjusted for such non-climatic effects. They argued that removing the 

effects of this socio-economic transition would reduce the trend of land surface 

temperatures from 0.27 deg C/decade to 0.11 deg C/decade and possibly to 0.06 

deg C/decade. If this claim were correct it would, for many, fundamentally 

undermine the argument that the increasing concentration of atmospheric carbon 

dioxide as a consequence of human activity was a major driver of climate change. 

9.3.2 The Allegations 

8. The allegation that CRU withheld their data from scrutiny and adjusted it without 

justification is dealt with in Chapter 6. Here we are concerned with the specific 

allegation that CRU improperly dismissed contrary views about the interpretation 

of the CRUTEM series and prevented its serious consideration by the IPCC. 

9.3.3 Evidence in Support of the Allegations 

9. The reaction of Jones, who was then one of two CLAs for Chapter 3 

(Observations: Surface and Atmospheric Climate Change) of the Working Group 

1 report of IPCC AR4, to the MM2004 paper is revealed in an e-mail 

(1089318616.txt) sent on 8th July 2004 to Mann, that included: “The other paper 

by MM (McKitrick & Michaels, 2004) is just garbage. ... I can‟t see either of 

these papers being in the next IPCC report. Kevin (Trenberth, the other 

coordinating lead author for ch. 3) and I will keep them out somehow – even if we 

have to redefine what the peer-review literature is!” It indicates clearly his 

awareness of the paper prior to any drafts of IPCC AR4, and his determination to 

exclude it from consideration by IPCC.

10. It has been suggested2 that this determination to exclude is reflected in the fact 

that MM2004 was not referred to in either the first order draft of Chapter 3 in 

August 2005 or in the second order draft of March 2006. Nor was a paper drawing 

similar conclusions by de Laat and Maurellis (2006)3 referred to. It has been 

suggested to us by McIntyre4 that it was Jones who kept reference to these papers 

out of the first and second order drafts, with the implication that this was done 

improperly to prevent incorporation of conclusions contrary to those held by the 

CRU group. 

 

1

 McKitrick, R. and Michaels, P.J. 2004. A test of corrections for extraneous signals in gridded surface 

temperature data. Climate Research, 26, 159-173.

2

 McKitrick submission (no.15), 26 February 

3

 de Laat, A.T.J. and Maurellis, A.N. 2006. Evidence for influence of anthropogenic surface processes 

on lower tropospheric and surface temperature trends. International Journal of Climatology, 26, 897-

913.

4

 McIntyre submission (no. 23), 2 March 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

72

11. The omission of reference to MM2004 was criticised by expert reviewer Vincent 

Gray for Chapter 3, but this was rejected by the IPCC writing team for this 

Chapter (see Chapter 5 paragraph 24) with the argument that large scale patterns 

of atmospheric circulation over the continents and stronger warming of the 

continents would produce patterns of warming such as those recorded by 

temperature stations5. 

12. The final draft of Chapter 3 did however include reference both to McKitrick & 

Michaels (2004) and de Laat and Maurellis (2006), in the following paragraph: 

“McKitrick and Michaels (2004) and De Laat and Maurellis (2006) attempted to 

demonstrate that geographical patterns of warming trends over land are strongly 

correlated with geographical patterns of industrial and socioeconomic 

development, implying that urbanisation and related land surface changes have 

caused much of the observed warming. However, the locations of greatest 

socioeconomic development are also those that have been most warmed by 

atmospheric circulation changes (Sections 3.2.2.7 and 3.6.4), which exhibit largescale coherence. Hence, the correlation of warming with industrial and 

socioeconomic development ceases to be statistically significant. In addition, 

observed warming has been, and transient greenhouse-induced warming is 

expected to be, greater over land than over the oceans (Chapter 10), owing to the 

smaller thermal capacity of the land.”

13. It has been surmised in a submission to the Review by McKitrick6

 that Jones 

wrote the above paragraph and bears responsibility for its inclusion. The same 

submission alleges that it represents a ―fabricated conclusion‖ or ―invented 

evidence‖ and that only the derivation of a p-value7 from a statistical test that is 

compatible with the claim of statistical insignificance would rebut this allegation 

of fabrication. It alleges that this is evidence of bias, and that after attempts to 

exclude evidence that conflicted with the preferred CRU interpretation of the 

CRUTEM data series from drafts 1 and 2, reasons were contrived in the published 

draft for the specific purpose of rejecting this evidence. If this were correct, such 

actions would appear to violate the principles in Chapter 5, of the duty of 

scientists to ensure that uncertainties are clearly transmitted to those that have the 

responsibility for deciding on any contingent actions. 

 

5 Gray: “The „corrections‟ to the surface temperature record have always been based on very poor 

evidence. The many references to studies on individual or regional stations which find the need for 

much higher corrections than are currently applied, are ignored. Now you have ignored the persuasive 

evidence of McKitrick and Michaels 2004 Climatic Research 26 156-173 who have shown a significant 

influence on your „corrected‟ figures of a series of socioeconomic factors. You cannot just ignore this 

paper.”

Response of the IPCC writing team: “References are plentiful. Those of value are cited. Rejected.

The locations of socioeconomic development happen to have coincided with maximum warming, not 

for the reasons given by McKitrick and Michaels (2004) but because of the strengthening of the Arctic 

Oscillation and the greater sensitivity of land than ocean to greenhouse forcing owing to the smaller 

thermal capacity of land. Parker (2005) demonstrates lack of urban influences.”

http://pds.lib harvard.edu/pds/view/7795947?n=7&s=4&imagesize=1200&jp2Res=.25&rotation=0 

(comment 3-34). 

6

 McKitrick submission (no.15), 26 February 

7

 The p-value in statistics is the probability of obtaining a statistic at least as extreme as the one 

actually observed, assuming that there is no relationship between the measured phenomena. The lower

the p-value, the less likely that there is no relationship. McKitrick and Michaels (2004) obtained a pvalue of 0.002, suggesting a significant relationship between climatic and socio-economic trends. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

73

14. It has been further alleged8 that the changing response to the McKitrick and 

Michaels (2004) paper between the response to Gray and the statement in 

paragraph 13 of AR4, both assumed to have been by Jones, is evidence of a 

preparedness to accept any argument, irrespective of how well founded, that 

would refute conclusions to which he was opposed. 

9.3.4 Jones‟ Response 

15. These allegations were put to Jones. We summarise his oral and written responses 

as follows9,10: 

Jones comments that the e-mail of July 8, 2004 was sent on the spur of the 

moment and quickly forgotten. No pattern of behaviour with respect to his 

IPCC work could be construed from this one email. 

The reason for the strong response of the email and the justification for not 

including reference MM2004 in the early drafts is that it can be readily shown 

to be scientifically flawed. 

The basis for this statement is that if the CRUTEM3 trend is reduced by the 

factor claimed by MM2004, the land-based record then becomes incompatible 

with the ocean and the satellite record. MM2004 make no mention of this in 

their paper. In writing Chapter 3 of AR4 the author team were mindful of this. 

MM2004‘s analysis of the land surface temperature record is completely at 

odds with the rest of the surface and lower tropospheric temperature records. 

MM2004 also fails to take into account the effects of changes in the 

atmospheric circulation. 

In summary, the atmospheric circulation has been shown, in numerous studies, 

to account for large scale patterns of temperature change. Before undertaking 

the kind of analysis in MM2004, it is essential to account for known signals 

(i.e. the North Atlantic Oscillation (NAO)11 and El Nino Southern Oscillation 

(ENSO)12 and possibly others) and then examine the residuals. It does not 

make sense to calculate the p-value without allowing first for the atmospheric 

effects and their spatial autocorrelation in that calculation. 

In view of these arguments, it was reasonable to exclude MM2004, as the 

IPCC reports are assessments, not reviews of the literature. The author teams 

were chosen for their experience and expertise, and reliance was placed on this 

in determining which work should be included and discussed in the 

 

8

 McKitrick submission (no.15), 26 February 

9 Response from Professor Jones to questions from Professor Geoffrey Boulton (20 April) 

10 Clarification from Professor Jones to additional questions from Professor Geoffrey Boulton (3 May) 

11 The North Atlantic Oscillation (NAO) and the associated Arctic Oscillation are parts of the global 

wave structure in the northern hemisphere atmosphere. The Icelandic low and the Azores high are parts 

of this wave structure, and their east-west oscillations control the strength and direction of westerly 

winds and storm tracks across the North Atlantic. 

12 ENSO (El Niño-Southern Oscillation) is a 5-year (on average) oscillation between warm and 

upwelling cold waters in the tropical eastern Pacific, which has a major impact on atmospheric pressure 

in the region and knock-on effects on weather in many other parts of the world. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

74

assessment. 

Jones commented that the decision about which papers to include and which to 

exclude was a collective one of the author team of Chapter 3 of AR4. Jones 

stated that he did not write the relevant text in the final report as has been 

assumed: the lead on the relevant section (3.2) was another member of the 

writing team. The suggestion to make a response did not come from Jones, as 

he was not the responsible person for the section. He did, however, agree with 

its inclusion in the final draft as a part of the overall writing team and a CLA 

for Chapter 3. 

Jones explained that the decision to include MM2004 (and de Laat and 

Maurellis, 2006) was made at the final plenary team meeting in Bergen, and as 

stated the text was seen by the whole writing team. It had not been possible to 

include de Laat and Maurellis (2006) until then, as it was not published until 

after the third Lead Author‘s meeting. Discussion of MM2004 can be seen in 

comments numbered 3-283 to 3-289 of the Second Order Draft of Chapter 313. 

In two of these comments (3-284 and 3-285) it was stated that reference would 

be made to MM2004 in Section 3.2.2.2 with some text, which would point out 

that the papers by MM2004 and de Laat and Maurellis were biased. The fact 

that the Chapter author team had now read de Laat and Maurellis is referred to 

in response to comment 3-289. The comments were signed off by the two 

Review Editors for the Chapter. 

Jones also noted to us that there were three criticisms made of the draft of the 

Summary for Policy Makers (SPM) based on those of MM2004, but that these 

were rejected by the SPM writing team (which did not include any members 

of the Chapter 3 team). They regarded them as inconsistent with a large body 

of the climatological literature addressed in Chapter 3 (comments numbers 

482, 864 and 1005)14

.

9.3.5 Evidence from IPCC Review Editor for Chapter 3 

(Professor Sir Brian Hoskins) 

16. The role of Review Editors was to ensure that all comments from expert and 

government reviewers were given appropriate consideration, to advise lead 

authors how to handle contentious and controversial issues and to ensure that 

genuine controversies were reflected adequately in the text. Professor Hoskins 

was one of three Review Editors for Chapter 3. Telephone evidence was sought 

from Hoskins about the ways of working of the author team for Chapter 3. The 

full summary of the interview with Hoskins is available on the Review website15

.

17. Hoskins confirmed that LAs, working individually and as small groups, were 

responsible for the collation and primary assessment of material relevant to the 

 

13 Comments are available at http://hcl.harvard.edu/collections/ipcc/ by scrolling down to appropriate 

comments. 

14 Ibid. 13 

15 Evidence from Review Editors for Chapters 3 and 6 of the IPCC Fourth Assessment Report on “The 

Physical Science Basis” (June 2010) 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

75

topics for which they were responsible. The CLAs led the plenary meetings of the 

writing team prior to production of each of the drafts, led the process of overall 

collation of the Chapter material and the production of the initial drafts of the First 

and Second Draft Reports and the Final Draft Report of the Chapter. These drafts 

were discussed and agreed during plenary meetings of the whole writing team. 

18. There were a very large number of comments from reviewers, of which a majority 

were from a relatively small group. The Review Editors made sure that they were 

all given proper consideration, and that they were either responded to by a change 

in the text or by an adequate reason for omission that was recorded in the author 

responses to expert and governmental review comments16. Hoskins, as a Review 

Editor, took part in the Chapter 3 plenary discussions and ensured that conflicting 

views were addressed. 

19. Led by the two CLAs, Jones and Trenberth, the writing team for Chapter 3 was 

assiduous in dealing with comments. Hoskins was very impressed by Jones‘ 

attention to detail, and the rigour of the Chapter 3 process. 

20. The levels of confidence and uncertainty reflected in the drafts were based on the 

consensus of a group of CLAs and LAs who were chosen for their expertise and 

experience in relevant fields. Irrespective of whether a paper is published in a peer 

reviewed journal, it is the responsibility of the whole team to assess whether a 

paper‘s conclusions are robust and to justify whether its arguments should carry 

weight in the assessment. These decisions for each draft were taken in plenary 

sessions of the whole team. Hoskins said that it is inconceivable that a paper 

making significant claims relevant to the work of IPCC and the Chapter 3 team 

would not be considered by the team as a whole. The basis for rejecting one of the 

papers that is a focus of the allegation is included in IPCC records17. Decisions 

about the inclusion of the MM2004 paper would have been taken by the whole 

team. Jones‘ voice would have been one amongst many. 

9.3.6 Findings 

21. The essential issues in determining whether there is substance to the allegation 

that Jones misused his position to exclude well-founded conclusions that 

conflicted with his prior views and ―invented‖ an explanation in order to reject 

them are as follows: 

i) Was the decision to exclude the MM2004 paper from the first and second drafts 

an unreasonable one? 

ii) Was the reason given for the rebuttal of MM2004 in the final draft ―invented‖? 

iii) If the answer to either or both of these questions is ―yes‖, was Jones the major 

 

16 Available at: http://hcl.harvard.edu/collections/ipcc

17 The issue is discussed further in discussion of reviewers comments on the 2nd order draft of Chapter 

3 (footnote to comments 3-275 to 3-389), in part of which McKitrick objected to the exclusion of the 

MM2004 and de Laat and Michaelis 2004, to which the writing team responded by the addition of text 

to section 3.2.2.2 of the draft. This was included in the final draft of Chapter 3. The comments and 

responses can be found at: 

http://pds.lib harvard.edu/pds/view/7786376?n=40&s=4&imagesize=1200&jp2Res=.25&rotation=0 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

76

influence in excluding early consideration and/or in providing an ―invented‖ 

explanation? 

22. The answer to i) depends upon the scientific credibility of the article. The 

contemporary emails indicate a highly critical response to it, and Jones‘ email 

(paragraph 9) is very strongly negative. Was the absence of a discussion of 

MM2004 in the first and second drafts reasonable, or could it reflect suppression 

of a view merely because it conflicted with that of the writing team or of Jones? 

Was the analysis in MM2004, and later in de Laat and Maurellis (2006), so 

evidently flawed in an issue as important as the significance of the instrumental 

record of climate change (see Chapter 6), that it could be readily rejected in the 

AR4? Jones believes that it was (paragraph 15, bullets 2-4), although the 

persistence of a debate on the issue in peer reviewed journals, including on one 

side MM2004, de Laat and Maurellis (2006), McKitrick and Michaels (2007)18

and McKitrick (2010)19, and on the other side Benestad (2004)20, Parker (2010)21, 

Jones et al (2008)22 and Schmidt (2009)23, suggests at least a continuing margin of 

doubt. Those within the writing team took one view, and a group outside it took 

another. It is not in our remit to comment on the rights and wrongs of this debate, 

but those within the team had been entrusted with the responsibility of forming a 

view, and that is what they did. They initially rejected inclusion of reference to 

MM2004, and in the final draft included a commentary on it explaining why they 

disagreed with its conclusions. It may be that the conclusions of MM2004 

conflicted so strongly with a generally held view among climate scientists that 

rejection was made too easily; but in the absence of better evidence, this is mere 

speculation. The mechanisms of the IPCC did however ensure that a reference to 

the article (and to de Laat and Maurellis, 2006) was contained in the final draft. 

23. The answer to ii) depends upon the implication that the response to MM2004 in 

the published Chapter 3 was not scientifically credible. Having read most of the 

relevant papers however, we observe a consistence of view amongst those who 

disagree with MM2004 that has been sustained over the last 6 years, that the large 

scale organisation of atmospheric circulation produces a spatially integrated 

response to forcing. Although we do not comment on the relative merits of the 

two views, we see no justification of the view that that this response was 

―invented‖, or even that its various expressions in the response to reviewer Gray24

or the final text are fundamentally different. 

24. Irrespective of the above comments on issues i) and ii), the evidence of the 

Review Editor underlines the team responsibility for the text, and the unlikelihood 

 

18 McKitrick R and Michaels P. 2007. Quantifying the influence of anthropogenic surface processes 

and inhomogeneities on gridded global climate data. Journal of Geophysical Research 112: D24S09. 

19 McKitrick, R., 2010: Atmospheric circulations do not explain the temperature-industrtrialization 

correlation. Statistics, Politics and Policy (in press – summer 2010). 

20 Benestad, R. 2004. Are temperature trends affected by economic activity? Comment on McKitrick 

and Michaels. Climate Research, 27, 171-173.

21 Parker, D. E., 2010: Urban heat island effects on estimates of observed climate change, Wiley 

Interdisciplinary Reviews: Climate Change, 1(1), 123-133, doi:10.1002/wcc.21. 

22 Jones, P.D., Lister, D.H. and Li, Q., 2008: Urbanization effects in large-scale temperature records, 

with an emphasis on China. J. Geophys. Res. 113, D16122, doi:10.1029/2008/JD009916. 

23 Schmidt, G.A., 2009: Spurious correlations between recent warming and indices of local economic 

activity. Int. J. Climatol. 29, 2041-2048. 

24 Ibid. 5
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

77

that a single voice could dominate on an important issue. Jones‘ evidence is that 

McKitrick‘s ―surmise that Professor Jones ... wrote the paragraph (in Chapter 3) 

alone or in consultation with Trenberth, and bears responsibility for its 

inclusion”25 is false, and that the lead responsibility for the relevant section was 

another specified member of the writing team. 

25. We conclude that there is evidence that the text was a team responsibility. It is 

clear that Jones (though not alone) had a strongly negative view of the paper but 

we do not find that he was biased, that there was any improper exclusion of 

material or that the comments on the MM2004 paper in the final draft were 

―invented‖ given the (continuing) nature of the scientific debate on this issue. 

9.4 The Tree Ring Proxy Temperature Series 

9.4.1 The Scientific Challenge 

26. A paper by McIntyre and McKitrick (2003; hereafter referred to as M&M2003)26

argued that the so called ―hockey stick‖ plot (Mann, Bradley and Hughes, 1998; 

hereafter referred to as MBH98)27 contained both simple errors and serious 

statistical errors. It suggested that the ―hockey stick‖ shape of the MBH98 

reconstruction was largely an artefact of these errors and of the selection of 

specific tree ring series. 

27. This work posed a significant challenge to interpretation of the assessment by the 

IPCC of the history of climate of the last millennium prior to the instrumental 

record of the last 150 years. It cast doubt on the validity of the claim of MBH98 

and of Mann, Bradley and Hughes (1999; hereafter MBH99)28 that the northern 

hemisphere was warmer in the late 20th century than at any time during the last 

millennium, which has been assumed by many to be evidence in support of the 

argument for strong, human induced greenhouse warming over recent decades. 

 

9.4.2 The Allegations 

28. It has been alleged that Briffa, in his role as lead author for Chapter 6 in Working 

Group 1 for AR4, and as the member of the writing team with the most relevant 

expertise, attempted to bias the scientific conclusions towards those of the 

MBH98/99 and to set aside the inconvenient evidence of M&M2003. It is alleged 

this behaviour was calculated to favour one particular view of climate change and 

its causes, and to discredit or ignore opposing views, without, at the time, an 

adequate scientific reason for doing so. It would therefore represent a failure to 

discharge a scientist‘s responsibility impartially to represent current scientific 

understanding at the vital interface between science and policy. 

 

25 McKitrick submission (no. 15), 26 February 

26 McIntyre, S. and McKitrick, R. 2003. Corrections to the Mann et al (1998) proxy database and 

northern hemisphere average temperature series. Energy and Environment, 14, 751-771.

27 Mann, M.E., Bradley, R.S. and Hughes, M.K. 1998. Global-scale temperature patterns and climate 

forcing over the past six centuries. Nature, 392, 779-787. 

28 Mann, M.E., Bradley, R.S. and Hughes. M.K. 1999. Northern hemisphere temperatures during the 

last millenium: inferences, uncertainties and limitations. Geophysical Research Letters, 26(6), 759-762.
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

78

9.4.3 Evidence in Support of the Allegations 

29. The Chapter 6 writing team relied heavily on a paper that was in preparation by 

Wahl and Ammann (eventually published as Wahl and Ammann, 2007, hereafter 

referred to as WA2007) that purported to refute the arguments of M&M2003. It 

has been alleged that Briffa played a central role in improperly using WA2007 to 

refute M&M2003, to discredit a paper that conflicted with his core hypothesis, 

and in doing so was willing to break IPCC rules: 

The claim of refutation made in the second order and the published drafts of 

Chapter 6 was knowingly misleading, because the method used to support the 

conclusions of WA2007 was dubious, and in any case, relied upon material 

that was rejected for publication in Geophysical Research Letters, and was not 

made available until August 2008 in an online supplement to Ammann and 

Wahl 2007, hereafter referred to as AW2007, long after acceptance of the final 

draft of Chapter 6. Thus even if the paper did represent an effective refutation 

of the M&M2003 argument, it was not available prior to the acceptance of the 

final draft of Chapter 6 of AR4.

IPCC rules require papers that are to be referenced should be at least 

 ̳accepted‘ by journals by specific deadline dates. WA2007 missed these 

deadlines and should not have been quoted as evidence.

Briffa broke confidence by asking Wahl, who was not involved in the IPCC 

process, to comment on Chapter 6 text. 

30.The second order draft text of IPCC 2007 WGI Chapter 6, assumed in the 

allegation to have been written by Briffa, and sent late in March 2006 to the 

Government and Expert Reviewers, included on page 29 the following text that 

relied on WA2007 to rebut M&M2003: 

“McIntyre and McKitrick (2003) reported that they were unable to replicate the 

results of Mann et al. (1998). Wahl and Ammann (accepted) demonstrated that 

this was due to the omission by McIntyre and McKitrick of several proxy series 

used by Mann et al. (1998). Wahl and Ammann (accepted) were able to 

reproduce the original reconstruction closely when all records were included”. 

31. This text was criticised by the Reviewer for the Government of the United States 

of America, who wrote in comment 6-75029

: 

“The use of Wahl and Ammann (accepted) does not comply with WG1‟s deadlines 

and all text based on this reference should be deleted. WG1‟s rules require that 

all references be “published or in print” by December 16, 2005. Wahl and 

Ammann was “provisionally accepted” on that date, and not fully accepted until 

February 28, 2006, at which time no final preprint was available. Substantial 

changes were made in the paper between December 16, 2005 and February 28, 

 

29 Comments are available at http://hcl.harvard.edu/collections/ipcc/
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

79

2006, including insertion of tables showing that the MBH98 reconstruction failed 

verification with r-squared statistics, as had been reported by McIntyre and 

McKitrick in 2003. These tables were not available in the draft considered by 

WG1 when developing the second-order draft.” The response to this was:

“Rejected – the citation is allowed under current rules” (Comment 6-1158)30. 

32. It was alleged that the material derived from WA2007 that was the rationale for 

the text in the final version of Chapter 6 was based on material that was not 

published or openly available until after the last deadline for the final draft. Their 

evidence should therefore not have been included.

33.In an email dated 18 July 2006 (1153470204), Briffa wrote to Wahl, who was not 

an official Expert Reviewer, as follows: 

“Gene I am taking the liberty (confidentially) to send you a copy of the 

reviewers comments (please keep these to yourself) of the last IPCC draft 

chapter. I am concerned that I am not as objective as perhaps I should be and 

would appreciate your take on the comments from number 6-737 onwards, that 

relate to your reassessment of the Mann et al work. I have to consider whether 

the current text is fair or whether I should change things in the light of the 

sceptic comments. In practise this brief version has evolved and there is little 

scope for additional text, but I must put on record responses to these comments 

- any confidential help , opinions are appreciated . I have only days now to 

complete this revision and response. note that the sub heading 6.6 the last 2000 

years is page 27 on the original (commented) draft. Cheers Keith”

34. It is alleged that this e-mail is an appeal to a strong proponent of the ―hockey 

stick‖ plot for assistance in coping with the comments of reviewers sceptical about 

it, that it hands confidential material to Wahl for him to help rebut the comments 

from Expert Reviewers critical of the Wahl and Ammann paper, and that it breaks 

rules of confidentiality at a stage when even official reviewers were denied ready 

access to review comments. It is implied that Briffa was prepared to go to 

exceptional and improper lengths to bolster a case that he supported and to defend 

it against alternative views. 

35. Finally, it is alleged that the relevant paragraph on p.466 in Chapter 6 of the AR4 

Final Report leaves the last word to Wahl and Ammann, and the reader is left with 

the clear impression that the M&M2003 criticisms have been rebutted, although 

the work claimed to be the rigorous basis of this rebuttal had missed or was long 

after IPCC deadlines. 

 

30 Ibid. 29 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

80

9.4.4 Responses from Briffa 

36. These allegations were put to Briffa. We summarise his oral and written responses 

as follows31,32,33,34

: 

a) Evaluation of the M&M2003 work by the Chapter 6 writing team 

Briffa responded that the M&M2003 work was taken very seriously by the 

Chapter 6 writing group. There was genuine concern about whether its claim 

that the MBH98 reconstruction could not be replicated was correct. It was 

important to assess the merits of this, as a failure in replication would, if 

substantiated, be crucial in assessing confidence in the MBH99 reconstruction 

and the trend of past climate. Did M&M2003 provided sufficient grounds for 

dismissing the view of temperature change in MBH98? 

The evaluation of this issue was very similar in both the second order and 

final, published versions of Chapter 6. The final version is: 

“McIntyre and McKitrick (2003) reported that they were unable to 

replicate the results of Mann et al. (1998). Wahl and Ammann (2007) 

showed that this was a consequence of differences in the way McIntyre and 

McKitrick (2003) had implemented the method of Mann et al. (1998) and 

that the original reconstruction could be closely duplicated using the 

original proxy data. McIntyre and McKitrick (2005a,b) raised further 

concerns about the details of the Mann et al. (1998) method, principally 

relating to the independent verification of the reconstruction against 19thcentury instrumental temperature data and to the extraction of the 

dominant modes of variability present in a network of western North 

American tree ring chronologies, using Principal Components Analysis. 

The latter may have some theoretical foundation, but Wahl and Ammann 

(2006NOTE) also show that the impact on the amplitude of the final 

reconstruction is very small (~0.05°C; for further discussion of these 

issues see also Huybers, 2005; McIntyre and McKitrick, 2005c,d; von 

Storch and Zorita, 2005).”

The first sentence above reflects the concern that the results of MBH98 could 

not be replicated. 

Briffa noted however that M&M2003 had not followed the MBH98 method and 

that Rutherford et al. (2005)35 had pointed out that the M&M2003 approach to 

calculation resulted in the elimination of 77 out of the 95 pre-1500 tree ring 

 

31 Response to specific questions raised by Professor Geoffrey Boulton, in his letter of 6 May 2010, in 

his role as a member of the Muir-Russell Review team (19 May) 

32 Issues for discussion with Briffa and Jones on 9 April 2010 (April 10) 

33 Response to Additional Question regarding Keith Briffa‟s request to Eugene Wahl and his response 

(June 2010) 

34 Copies of Communications relating to Professor Briffa‟s editorial treatment of a submitted 

manuscript (June 2010)

35 Rutherford, S., M.E. Mann, T.J. Osborn, R.S. Bradley, K.R. Briffa, M.K. Hughes and P.D. Jones, 

2005: Proxy-based Northern Hemisphere surface temperature reconstructions: Sensitivity to 

methodology, predictor network, target season and target domain. Journal of Climate 18, 2308-232. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

81

proxy series used by MBH98. 

WA2007 had then shown that the results of MBH98 could be replicated very 

closely using their implementation of the MBH98 methods and using the same 

data. This is pointed out in the second sentence of the above paragraph from 

Chapter 6. 

Briffa pointed out that the AR4 text did not state that WA2007 had disproved 

the concerns of M&M2003. Instead it considered their possible impact on the 

final reconstruction, citing papers that had assessed this impact including, but 

not exclusively, WA2007. These results indicated that the impact on the final 

reconstruction might be relatively small, leading to the view, contained in the 

AR4 text, that the criticisms raised against the MBH98 reconstruction were not 

sufficient to discount the evidence that it represented. 

Briffa commented that he believed the above quoted paragraph from the 

published report represented a proper and balanced response to the issues raised 

by M&M2003, that the issues were important to the assessment and that the 

reference to WA2007 was a significant part of the debate. 

Briffa and his colleague Osborn36 commented that in any case the MBH98 was 

only one of 12 such reconstructions in figure 6.10 in Chapter 6, and does not 

therefore dominate the picture. The M&M2003 series is not presented in that 

figure as an alternative reconstruction, as McKitrick commented on the first 

order draft that, the draft ―trots out the straw man that we are selling an 

alternative climate history, despite our repeated and persistent statements that 

we are not trying to offer „our‟ climate history curve.‖

37

Briffa also rejected the implication that this text was his responsibility, asserting 

that it was the responsibility of the whole writing group, not of any one person. 

 b) Deadlines 

Briffa rejected the allegation that IPCC rules on deadlines were broken because 

of a determination to include reference to WA2007 (which was claimed to have 

been in press at the time). Details of these allegations and the responses to them 

from Briffa and Osborn are on the Review website38

.

Briffa pointed out that the scientific content in AW2007, referred to by 

WA2007, is equivalent to the content in the manuscript rejected by Geophysical 

Research Letters (this was not rejected for its scientific content, but for other 

editorial reasons). 

 

 c) Breaching confidentiality 

 

36 Response to specific questions raised by Professor Geoffrey Boulton, in his letter of 6 May 2010, in 

his role as a member of the Muir-Russell Review team (May 2010) 

37 Minute 6-1319 at http://hcl.harvard.edu/collections/ipcc/

38 Ibid. 33 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

82

Briffa responded to the allegation of having broken confidentiality in sending 

draft text to Wahl to comment on, that there is no proscription in the IPCC rules 

to prevent the author team seeking expert advice when and where needed. The 

Technical Support Unit (TSU) and the CLAs of Chapter 6 agree that the author 

team was allowed to seek such advice. Copies of communications from both 

CLAs (Jansen and Overpeck) and the IPCC WG1 TSU are provided by Briffa 

(and published on the website) to provide support to Briffa‘s claim that his 

actions did not contravene IPCC procedures. 

Briffa asserts that Wahl was asked for comment on text as a knowledgeable and 

objective arbiter and as such was a wholly reasonable judge of whether the 

responses were appropriate. Given his particular expertise on the details of the 

Mann et al. methodology and most importantly the implications for the 

character of the Mann et al. reconstruction, Briffa felt justified in seeking his 

advice and in using specific wording in a very few responses that were based on 

the text of a paper co-authored by Wahl (AW2007). Wahl did not write any of 

the main text, though he did make some suggestions for very minor edits. 

Briffa‘s evidence includes copies of the relevant email exchanges with Wahl39

, 

which also confirm that both Jansen and Overpeck, as CLAs, were aware of the 

approach to Wahl. 

9.4.5 Evidence from IPCC Review Editor for Chapter 6 

(Professor John Mitchell) 

37. Mitchell was one of two Review Editors for Chapter 6 of the AR4 Working Group 

1 report. A telephone interview with him was conducted to establish how 

unpublished and ―in press‖ material was handled and how decisions about the text 

of successive drafts were made. He commented as follows40: 

“I was not aware of the debate about whether the Wahl and Ammann paper had 

or had not met the deadline for the 2nd order draft for chapter 6, until after the 

event. The concentration on specific deadlines however misses the larger point. It 

must be recognised that if only published sources were used, the report would be 

two years old by the time of publication. In a fast-moving area such as climate 

change research, assessments could be significantly behind the times if important, 

but as yet unpublished, new results could not be used. The assessments for 

policymakers could also therefore be behind the times”. 

“In earlier assessments, there had been a relatively liberal regime in using 

unpublished material provided that there was a sound basis for regarding it as 

rigorous or reliable, although priority was always given to finding published 

sources. In AR4 however, the regime was tightened significantly, so that such 

material was only to be used under exceptional circumstances, but the use of 

unpublished material was not prohibited. „Hockey-stick‟ issues were regarded at 

 

39 Response to Additional Question regarding Keith Briffa‟s request to Eugene Wahl and his response 

(June 2010), available at: www.cce-review.org/Evidence.php 

40 Evidence from Review Editors for Chapters 3 and 6 of the IPCC Fourth Assessment Report on “The 

Physical Science Basis”
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

83

the time as sufficiently important to justify using new data. The dilemma between 

using only published material and being out of date, or using more recent 

unpublished material was increased in AR4 as the „latest publication date‟ was 

about 12 months earlier than in the process than in the previous assessment”.

“The email is problematic (para 33). On the one hand it appears to reflect an 

honest request to an expert for a comment about the extent to which the author is 

being balanced and fair. On the other hand, it stresses the need for confidentiality 

in three places, implying that the author realizes that the approach may be 

improper. There was also a leak of an early draft of the WG1 report to the press 

which led to IPCC emphasizing the need to maintain confidentiality in general 

which may have been at the back of the author‟s mind”. 

“In principle however there is nothing in IPCC rules that prevents an author from 

seeking external help, comment or judgement on text through consulting his peers. 

It is questionable whether expert reviewers‟ comments, in some* ways analogous 

to the comments of a peer reviewer for a journal, should be shared with an third 

party without their consent. (*Note that unlike most peer-reviewed journals, IPCC 

reviewers names and comments are made available at the end of the process).”

9.4.6 Findings 

38. The essential issues to be resolved in relation to the allegations are: 

Was the McIntyre and McKitrick (2003) paper dealt with in a reasonable 

fashion in the draft and final versions of Chapter 6? 

The evidence and narrative provided by Briffa is persuasive that these issues were 

dealt with in a careful and reasonable fashion that took into account the 

importance of the issues addressed in M&M2003. McIntyre or McKitrick may 

have wished to see them addressed in a different way, but they were addressed 

seriously and cogently. 

Is there any evidence of any personal desire from Briffa to protect a 

fundamentalist line in defending the conclusions of MBH98/99? 

The evidence of the Review Editor suggests that no one person in the writing team 

could have overridden the team responsibility for the text. Indeed, the evidence of 

a contemporary e-mail (1140039406; Feb 2006; to Overpeck) suggests that Briffa 

was unlikely to be an uncritical defender of the MBH view of the  ̳hockey stick‘, 

and wished to respect the view of the writing team as a whole: 

―Peck, you have to consider that since the TAR, there has been a lot of argument 

re„hockey stick‟ and the real independence of the inputs to most subsequent 

analyses is minimal. True, there have been many different techniques used to 

aggregate and scale data - but the efficacy of these is still far from established. 

We should be careful not to push the conclusions beyond what we can securely 

justify - and this is not much other than a confirmation of the general conclusions 

of the TAR.....Just need to show the "most likely"course of temperatures over the 

last 1300 years - which we do well I think. Strong confirmation of TAR is a good 

result, given that we discuss uncertainty and base it on more data. Let us not try 

to over egg the pudding. For what it worth, the above comments are my (honestly 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

84

long considered) views - and I would not be happy to go further. Of course this 

discussion now needs to go to the wider Chapter authorship, but do not let Susan 

[Solomon – co-chair of IPCC WG1 for AR4] (or Mike) [Mann] push you (us) 

beyond where we know is right.”

Is the inclusion of references to WA2007 allegedly against the rules of IPCC 

evidence of a determination to rebut M&M2003 at all costs? 

Taking into account the evidence of the Review Editor, IPCC papers, the 

statement from the CLA, and the importance of the issues raised by M&M2003, 

we consider it to be reasonable that work that might throw further light on these 

issues and to ensure that assessments were as up to date as possible should be 

included. We do not consider therefore that these were exceptional unwarranted 

efforts to defend a particular position, but reasonable attempt so use up to date 

information that might resolve an issue. They appear to be consistent with IPCC 

principles and to reflect a concern for objectivity. 

Was there breach of confidentiality in having Wahl comment on draft text of the 

report and does this reflect determination to sustain a predetermined line? 

Although Briffa‘s e-mail41 stressing confidentiality does imply an awareness of 

questionable conduct, the e-mail correspondence with Wahl42 stresses in several 

places Briffa‘s concern to be fair to sceptical views. We see no evidence in the 

correspondence of anything other than a detailed determination to resolve a 

scientific issue. Nor do the IPCC Review Editor‘s comments to us suggest that 

what would normally be regarded in the research community as conventional 

requests for advice and help, were ruled out. 

39. We conclude, in line with the comments made by Professor Mitchell, it was not 

unreasonable to include the WA2007 paper alongside the M&M2003 paper in 

presenting an up-to-date picture of the relevant scientific arguments. We do not 

find that Briffa acted improperly in the part he played in this, and note that in any 

case he did not have sole responsibility for the outcome. 

9.5 Conclusions 

40. In summary, we have not found any direct evidence to support the allegation that 

members of CRU misused their position on IPPC to seek to prevent the 

publication of opposing ideas. 

41. In addition to taking evidence from them and checking the relevant minutes of the 

IPCC process, we have consulted the relevant IPCC Review Editors. Both Jones 

and Briffa were part of large groups of scientists taking joint responsibility for the 

 

41 Briffa to Wahl email: 1140039406.txt; Feb 2006 "I am taking the liberty (confidentially) to send you 

a copy of the reviewers comments (please keep these to yourself) of the last IPCC draft chapter. I am 

concerned that I am not as objective as perhaps I should be and would appreciate your take on the 

comments from number 6-737 onwards, that relate to your reassessment of the Mann et al work. I have 

to consider whether the current text is fair or whether I should change things in the light of the sceptic 

comments. " 

42 Response to Additional Question regarding Keith Briffa‟s request to Eugene Wahl and his response 

(June 2010) 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

85

relevant IPCC Working Group texts and were not in a position to determine 

individually the final wording and content. We find that neither Jones nor Briffa 

behaved improperly by preventing or seeking to prevent proper consideration of 

views which conflicted with their own through their roles in the IPCC. 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

86

CHAPTER 10: COMPLIANCE WITH 

FoIA/ENVIRONMENTAL INFORMATION 

REGULATIONS 

10.1 Introduction and Method of Enquiry 

1. This Chapter addresses the third part of our remit: “Review the Climatic Research 

Unit‘s compliance or otherwise with the University of East Anglia‘s policies and 

practices regarding requests under the Freedom of Information Act 20001

( ̳the 

FoIA‘) and the Environmental Information Regulations2( ̳the EIR‘) for the release 

of data‖. It also reviews subject access requests under the Data Protection Act 

1998 – as amended3

. Whilst compliance with FoIA, EIR and DPA fall within the 

more general topic of  ̳Governance‘ (covered in the following Chapter), they are 

extracted and covered in detail here, given their very specific relevance to the 

Review. 

2. Interviews were carried out with the: 

Information Policy & Compliance Manager – IPCM; 

Science Faculty FoIA/EIR Contact; 

Director of Information Services; and 

Key staff within CRU. 

3. Discussions were also held with representatives of the ICO, both to co-ordinate 

the work of the Independent Review with that of the concurrent ICO investigation 

and to seek advice. 

4. Notes of all these interviews are available on the Review website. 

10.2 The Allegations 

5. It is alleged by a number of correspondents (for example the submission by 

Matthews4) and commentators that requests under the FoIA and the EIR were 

incorrectly denied. Other correspondents (for example the submission by Mann5) 

have suggested that a number of these FoIA requests were inappropriate or 

frivolous. Similarly it is alleged that subject access requests under the DPA for 

access to e-mails specifically referencing the applicant were not fully complied 

with. 

10.3 General Context

6. The Freedom of Information Act 2000 created new statutory rights of access, 

subject to a number of defined exemptions, to information held by a wide range of 

public bodies. It replaced the earlier  ̳Code of Practice on Access to Government 

Information‘ that was a non-statutory scheme. The general right of access under 

 

1

 See: http://www.opsi.gov.uk/Acts/acts2000/ukpga_20000036_en_1 

2

 See: http://opsi.gov.uk/si/si2004/uksi_20043391_en.pdf 

3

 See: http://www.ico.gov.uk/what_we_cover/data_protection/legislation_in_full.aspx 

4

 Matthews submission (no. 16), 1 March 

5

 Mann submission (no. 42), 28 February 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

87

this Act came into force on 1st January 2005. In cases where access to information 

is refused, the Act requires the public authority to give reasons for that refusal, 

including detailing any exemptions the public authority believes may apply. 

There is also a duty under the Act to offer reasonable advice and assistance to 

applicants seeking information. 

7. The current Environmental Information Regulations (EIR) also came into force on 

1

st January 2005. These contain a similar general right of access to that defined 

under the FoIA. Their genesis however dates back to initial regulations based on 

the European Community (EC) Council Directive 90/313 published in 1992, to 

which UEA would have been subject. It is not clear whether this was widely 

understood either by the University or by those seeking information. 

8. Both the FoIA and EIR: 

encourage the use of formal publication policies (―Schemes‖) and pro-active 

dissemination of information; 

have initial 20 working day time limits for the public authority to respond to 

requests; 

require public authorities to provide advice and assistance to applicants and to 

provide the information in the form or format requested wherever reasonably 

possible. 

use the  ̳Public Interest‘ test. Information can only be withheld if: ―in all the 

circumstances of the case, the public interest in maintaining the exception 

outweighs the public interest in disclosing the information‖;

have defined appeal processes (albeit with some subtle differences on time 

limits); and 

have common recourse to the Information Commissioner and to the 

Information Tribunal. 

9. Key differences6

 between the FoIA and EIR regimes are that, under the EIR: 

requests can be made verbally and do not have to be submitted in writing (or 

by e-mail) with a name and address; 

there is a clearer definition of  ̳information held‘ than under the FoIA: ―If the 

information is in the authority's possession and has been produced or received 

by the authority, or if it is held by another person on behalf of the authority‖; 

a request cannot be rejected purely on grounds of cost; 

the allowed initial response deadline can be extended to 40 working days for 

particularly complex requests; 

withholding of information under exemptions remains subject to the  ̳public 

interest‘ test;

there is an emphasis on improved decision making and participation; and 

there is no direct equivalent to the FoIA  ̳vexatious requests‘ provisions.

There are no guarantees of absolute confidentiality under the EIR. 

10. The EIRs only cover environmental information, albeit this is subject to a very 

broad definitioni (see endnote after paragraph 35). The FoIA covers all other 

 

6

 See: http://www.defra.gov.uk/corporate/policy/opengov/eir/slides-leaflets htm 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

88

information held by public authorities, but with a specific exemption for 

environmental information. 

11. The DPA 1998 (as amended) gives individuals the right to know what information 

is held about them. It provides a framework to ensure that personal information is 

handled properly. The Act states that anyone who processes personal information 

must comply with eight principles, which make sure that personal information is: 

Fairly and lawfully processed 

Processed for limited purposes 

Adequate, relevant and not excessive 

Accurate and up to date 

Not kept for longer than is necessary 

Processed in line with your rights 

Secure 

Not transferred to other countries without adequate protection 

12. More specifically in the context of this Chapter, the Act provides individuals with 

important rights, including the right to find out what personal information is held 

on computer and most paper records. 

13. UEA‘s (and thus CRU‘s) formal processes are described in 3 documents:

―The Code of Practice for Responding to Requests for Information under the 

Freedom of Information Act 2000‖7

, which is based on the Lord Chancellor‘s 

Code and sets out the manner in which UEA processes requests. 

―Guidance for Staff‖8, which directs how UEA staff should respond to 

requests for information. 

―Requests for Information‖9, which explains how to make a request for 

information, and how the UEA expects to respond. 

14. UEA‘s FoIA/EIR processes are based on concentric circles. The IPCM is 

considered the central point. He: 

should always be the co-ordinator for FoIA/EIR requests; 

maintains the formal FoIA/EIR logs on requests received; 

establishes a case file for each request; 

makes the initial determination as to whether to treat a request under the FoIA 

or EIR regime; 

determines in cooperation with the Faculty contact and relevant staff whether 

the information sought is indeed ―held‖ in statutory terms; and 

identifies and approaches the key provider via the appropriate faculty or 

central unit contact. 

15. The IPCM is the key person within UEA with a detailed understanding of the 

FoIA and the EIR. He maintains liaison with the ICO. He is also the key training 

provider to others within UEA. 

 

7

 See: 

http://www.uea.ac.uk/is/strategies/infregs/FOIA+Code+of+Practice+for+Responding+to+Requests 

8

 See: http://www.uea.ac.uk/is/foi/guidance 

9

 See: http://www.uea.ac.uk/is/foi/foi_requests 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

89

16. The next circle comprises FoIA/EIR contacts at the Faculty or Central Unit level. 

There are FoIA/EIR contacts in place for each of the four University Faculties and 

for each of the central university units (such as Information Systems and the 

Registrar‘s Department). These receive initial training and then yearly updates. It 

is understood that participation in these training sessions has been good. Beyond 

the initial training, it is understood that much learning is ―on the job‖. 

17. The final circle comprises individual members of staff/researchers. An initial 

brochure was sent to all units in the University when the FoIA/EIR regimes came 

into force in 2005. Awareness training is made available on a voluntary basis. 

10.4 Investigation

18. As stated in this Review‘s  ̳Issues for Examination‟ document (Appendix 3), it is 

alleged that requests under the FoIA, EIR and DPA were incorrectly denied by 

UEA on advice from CRU. This is the subject of a separate inquiry by the ICO, 

but does fall within the terms of reference of the Review. The Review has 

remained in regular contact with the ICO to ensure that both investigations could 

continue in parallel (see paragraph 19). The Review is particularly concerned to 

address: 

What formal processes were in place both centrally and within CRU to ensure 

fair and impartial assessment of FoIA requests? 

Were there any processes in place centrally to review recommendations from 

CRU that information should not be released? 

19. The improperly released e-mails contain a number of references that raise specific 

concerns with respect to FoIA, EIR and the DPA. In particular, since CRU uses 

data obtained from other bodies, guidance was sought from the ICO on the extent 

to which a public authority that was not a primary repository of data might be 

expected to act as a secondary source of that data for the purposes of FoIA/EIR. 

The advice received was that: 

Neither FoIA nor EIR make a distinction as to whether a public body is a 

primary or secondary source of information or data. The point is simply 

whether they hold the information for the purposes of the legislation, i.e. 

section 3(2) of FoIA and Regulation 3(2) EIR. 

If the public authority holds the information for any reason or in any form 

they must provide it or rely on a provision in the legislation to refuse the 

request. 

If however a request is received and the information requested is publicly 

available, and reasonably accessible, the public body can rely on either 

section 21 FoIA or Regulation 6 EIR and point the requester to these 

sources. “Reasonably accessible” will normally mean the information is 

published or available on demand. There is no obligation for the public 

body to apply section 21 or Regulation 6: the public body is free to choose 

to supply the data.
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

90

20. The Review recognises three time periods into which information requests should 

be grouped: 

before 2005, where the FoIA was not in operation but a precursor regime to 

the current EIR was operating; 

from 2005 until the start of 2007, when responses to requests were being 

handled informally by CRU; and 

after 25th January 2007 when the first request was formally logged by the 

IPCM. 

21. The Review had access to the complete formal log of FoIA/EIR requests, 

compiled by the IPCM, with respect to CRU since both current regimes came into 

force at the start of 2005. CRU only started to treat such requests formally via the 

IPCM in 2007. It should be noted however that there are extensive indications, 

from e-mails now in the public domain, of requests for information and some 

release of information prior to 2007. An example is the e-mail from Jones on 21st

February 2005 (1109021312.txt), ―PS I'm getting hassled by a couple of people to 

release the CRU station temperature data. Don't any of you three tell anybody 

that the UK has a Freedom of Information Act!‖. Indeed, three e-mails from 2004 

also made various information requests. 

22. Formal FoIA/EIR requests were initially quite limited. 

In 2007 four requests were received, of which two were given full release of 

the requested information but two, despite appeals, were rejected. 

In 2008 two requests were received, one was granted full release, but the other 

was rejected, both initially and upon appeal. 

In the first half of 2009 only one request was received and this was responded 

to in full. 

23. But in the third quarter of 2009 a wave of requests was received. In the five days 

starting on 24th July, some 60 requests were logged by the IPCM. A further 10 

requests were logged between the 31st July and 14th August. Some related to the 

raw station data underpinning the CRUTEM data sets and the vast majority sought 

details of any confidentiality agreements related to this data. The wordings bear 

the hallmarks of an organised campaign. One applicant (UEA Log 09/97) appears 

to have forgotten to customise the request before dispatch. The text reads: ―I 

hereby make a EIR/FOI request in respect to any confidentiality agreements) 

restricting transmission of CRUTEM data to non-academics involving the 

following countries: [insert 5 or so countries that are different from ones already 

requested] 

the date of any applicable confidentiality agreements; 

the parties to such confidentiality agreement, including the full name of 

any organization; 

a copy of the section of the confidentiality agreement that "prevents further 

transmission to non-academics". 

a copy of the entire confidentiality agreement.”

24. In the final quarter of 2009 a further wave of 41 requests was received, starting on 

20th November. These were mostly related to the unauthorised release of e-mails 

and seeking information in the context of that release. However a number raised 

other issues such as: the extent of UEA staff training in FoIA/EIR issues; sources 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

91

of funding; and statistics on FoIA/EIR requests received. Still others harked back 

to previous themes of access to the CRUTEM data set and related confidentiality 

agreements. 

10.5 Findings

25. Education and culture. Whilst we did identify evidence that UEA had widely 

distributed initial guidance at the introduction of the FoIA/EIR regimes in 2005, 

we also found a lack of engagement by core CRU team in understanding 

EIR/FoIA legal requirements and how these might legitimately impact them. 

There was evident confusion within the CRU as to how these requirements might 

be applied for example to data, code, and personal correspondence. We found a 

tendency to assume that no action was required until precedents had been set. As 

an example, on 21st January 2005 Jones wrote (1106338806.txt): ―On the FOI Act 

there is a little leaflet we have all been sent. It doesn't really clarify what we might 

have to do re programs or data. Like all things in Britain we will only find out 

when the first person or organization asks. I wouldn't tell anybody about the FOI 

Act in Britain. I don't think UEA really knows what's involved.‖ There was 

insufficient priority given from the UEA centre to motivating staff and to 

prompting continuing education. Various requests received by the CRU between 

2005 and the start of 2007 had not been formally logged with the IPCM. 

26. Foresight lacking. We found a lack of recognition by both CRU, and the 

University‘s senior management of the extent to which earlier action to release 

information or give full guidance to locate primary sources and to provide station 

identifiers might have minimized the problems. There are many references in the 

e-mails now in the public domain to ―hiding information‖, ―finding ways around 

releasing‖, or finding excuses not to release information. There was a fairly swift 

shift towards a lack of sympathy with the requesters, as seen in an e-mail from 

Jones sent on 7th May 2005 (1083962092.txt): ―Mike and I are not sending 

anything, partly because we don't have some of the series he wants, also partly as 

we've got the data through contacts like you, but mostly because he'll distort and 

misuse them.‖ 

We do not suggest that the allegations made against McIntyre are correct. 

27. Unhelpful responses. We found a tendency to answer the wrong question or to 

give a partial answer. For example the very first formal FoIA request (UEA Log 

07-04) asked for the list of stations used in the preparation of HadCRUT3 and the 

raw data for these stations. The initial response refers simply to the availability of 

the raw data from other sources and not to the station list. The requester then 

enters into an extended correspondence trying to extract the station identifiers. An 

extract from 14th April 2007 is given below: 

“While it is good to know that the data is available at those two web sites, that 

information is useless without a list of stations used by Jones et al. to prepare 

the HadCRUT3 dataset. As I said in my request, I am asking for: 

1) A list of the actual sites used by Dr. Jones in the preparation of the 

HadCRUT3 dataset, and 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

92

2) A clear indication of where the data for each site is available. This is quite 

important, as there are significant differences between the versions of each 

site's data at e.g. GHCN and NCAR." 

Without knowing the name and WMO number of each site and the location of 

the source data (NCAR, GHCN, or National Met Service), it is not possible to 

access the information. Thus, Exemption 21 does not apply - I still cannot 

access the data. 

I don't understand why this is so hard. All I am asking for is a simple list of the 

sites and where each site's data is located. Pointing at two huge piles of data 

and saying, in effect, "The data is in there somewhere" does not help at all. 

To clarify what I am requesting, I am only asking for a list of the stations used 

in HadCRUT3, a list that would look like this: 

 WMO# Name Source 

 58457 HangZhou NCAR

 58659 WenZhou NCAR

 59316 ShanTou GHCN 

 57516 ChongQing NMS 

etc. for all of the stations used to prepare the HadCRUT3 temperature data. 

That is the information requested, and it is not available "on non-UEA 

websites", or anywhere else that I have been able to find. 

I appreciate all of your assistance in this matter, and I trust we can get it 

resolved satisfactorily.”

These station identifiers are finally promised on appeal, but not provided until 

some six months later following further prompting. 

28. Deliberate actions to avoid release. There seems clear incitement to delete emails, although we have seen no evidence of any attempt to delete information in 

respect of a request already made. Two e-mails from Jones to Mann on 2nd

February 2005 (1107454306.txt) and 29th May 2008 (in 1212063122.txt) relate to 

deletion: 

2

nd February 2005: ―The two MMs have been after the CRU station data for 

years. If they ever hear there is a Freedom of Information Act now in the UK, I 

think I'll delete the file rather than send to anyone”.

29th May 2008: ―Can you delete any emails you may have had with Keith re 

AR4? Keith will do likewise. He's not in at the moment - minor family crisis. 

Can you also email Gene and get him to do the same? I don't have his new 

email address. We will be getting Caspar to do likewise”. 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

93

There is a clear statement that e-mails had been deleted – for example, an e-mail 

from Jones to Santer sent on 3rd December 2008 (1228330629.txt ): ―About 2 

months ago I deleted loads of emails, so have very little - if anything at all.‖

It seems likely that many of these  ̳deleted‘ e-mails subsequently became public 

following the unauthorised release from the backup server. There is evidence that 

the IPCM did try to warn Prof. Jones about deliberate deletion of information; for 

example, an email from Jones to Santer (1228922050.txt) 10th December 2008 

states: ―I did get an email from the FOI person here early yesterday to tell me I 

shouldn't be deleting emails - unless this was 'normal' deleting to keep emails 

manageable!”. 

29. Imbalance of authority. The current structure for handling FoIA/EIR 

requests within UEA focuses very much on the IPCM. At interview he 

indicated that he felt: “Very much the bull’s eye at the centre of the target”. We 

believe that the UEA senior staff need to take more explicit responsibility for 

these processes, thus enhancing the authority and standing of the IPCM. We 

found that the IPCM may have lacked such standing within the University 

structure and the authority to challenge the assertions of senior professors. 

In this context, the IPCM and the Faculty FoIA contact may not have been 

empowered to be sufficiently rigorous: Jones to Santer sent on 3rd December 

2008 (1228330629.txt): When the FOI requests began here, the FOI person 

said we had to abide by the requests. It took a couple of half hour sessions - one 

at a screen, to convince them otherwise showing them what CA was all about. 

Once they became aware of the types of people we were dealing with, everyone 

at UEA (in the registry and in the Environmental Sciences school - the head of 

school and a few others) became very supportive.” However at interview the 

IPCM explicitly denied that he behaved in the way suggested in this e-mail. 

30. Lack of constructive challenge. Whilst we found that efficient basic control, 

logging and progress chasing processes for FoIA/EIR requests were in place, we 

found a lack of independent working-level challenge in these systems. We found 

that the appeals mechanisms in place lacked the resources for effective challenge 

to basic assumptions. Similarly, the escalation processes failed to react sufficiently 

quickly to the dramatic change in the volume and character of requests and to 

provide timely high-level review and resources. 

31. Limited internal communication. We found a lack of understanding within 

University central functions of the presence of extensive, and long duration, 

backups of e-mail and other materials despite these being on a server housed 

within the central Information Technology (IT) facilities. Awareness of these 

might have led to much greater challenge of assertions regarding non-availability 

of material by CRU, notably in the case of a subject access request made under the 

DPA for material naming the requesting individual. 

32. The Review found an ethos of minimal compliance (and at times non-compliance) 

by the CRU with both the letter and the spirit of the FoIA and EIR. We believe 

that this must change and that leadership is required from the University‘s most 

senior staff in driving through a positive transformation of attitudes. Public trust 

in science depends on an inherent culture of honesty, rigour and transparency. 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

94

The requirements of FoIA and EIR must not be seen as impositions. They are a 

necessary part of the implicit contract between the scientist and broader society. 

Such an open culture will also lead to the best science. 

10.6 Recommendations

33. The Review offers the following recommendations for action within the UEA: 

Change fundamentally the perception that responsibility for FoIA/EIR 

compliance lies with administrative staff. University senior staff need to make 

clear their commitment to a culture of honesty, rigour and transparency, plus 

the supporting processes and resources. 

Review the resourcing and standing of the FoIA/EIR/DPA compliance and 

request handling processes. Our findings have highlighted significant 

problems in the areas of: imbalance of authority; lack of effective challenge at 

appeal; over dependence on single individuals; inadequate escalation 

processes and limited strategic oversight. 

A concerted and sustained campaign to win hearts and minds. This should 

include: promotion of the University‘s formal publication policy; 

incorporating more information on FoIA/EIR/DPA responsibilities in the 

induction processes for new staff members; developing a rolling awareness 

campaign to focus the attention of established staff, particularly in the context 

of the changing landscape e.g. Queens University judgment (see paragraph 

34); and issuing annual reminders of the importance of transparency and of 

key FoIA/EIR/DPA responsibilities; 

Once the improved awareness measures and processes are in place, to run a 

programme of independent, external, tests with requests for information to 

verify the continuing effectiveness of these operations. This is a special case 

of the more general recommendation on  ̳Audit processes‘ given in the 

Governance Chapter. 

34. The Review offers the following more general recommendations: 

Definition of research data. There is extensive confusion and unease within the 

academic community as to exactly how FoIA/EIR should be applied in terms 

of the materials developed during a research process. The Review believes 

that all data, metadata and codes necessary to allow independent replication of 

results should be provident concurrent with peer-reviewed publication. 

However the situation regarding supporting materials such as early drafts, 

correspondence with research colleagues and working documents is widely 

regarded as unclear. The American experience is instructive here. The so 

called ―Shelby Amendment‖ in 1998 directed the US ―Office of Management 

& Budget (OMB)‖ to produce new standards requiring all data produced under 

Federally funded research to be made available under the US Freedom of 

Information Act. This resulted in great concern within the US Scientific 

community, expressed through Congressional testimony, that a very broad 

interpretation of this requirement could seriously impair scientific research 

and collaboration. In the final OMB guidelines10, recognising these concerns, 

 

10 Federal Register: March 16, 2000 Volume 65, Number 52 Page 14406. 

http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=2000_register&docid=00-5674-filed 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

95

―research data‖ is defined as: ―the recorded factual material commonly 

accepted in the scientific community as necessary to validate research 

findings, but not any of the following: preliminary analyses, drafts of scientific 

papers, plans for future research, peer reviews, or communications with 

colleagues‖. The Review recommends that the ICO should hold consultations 

on a similar distinction for the UK FoIA/EIR. 

Orchestrated campaigns. As detailed in paragraph 23, CRU was the subject of 

an orchestrated campaign of FoIA/EIR requests in late July and early August 

2009. The Review believes that CRU helped create the conditions for this 

campaign by being unhelpful in its earlier responses to individual requests for 

station identifiers and the locations from which specific, detailed station raw 

data could be downloaded. Similarly a clearer publication policy, reflecting 

the wishes of both the University and the research funders might have avoided 

these challenges. The Review team can however conceive of situations where 

such orchestrated campaigns might recur, with literally overwhelming impacts 

on small research units. We urge the ICO to give guidance on how best to 

respond to such organised campaigns, consistent with the underlying 

principles of openness. 

Greater clarity in an evolving landscape. Particularly in the light of the recent 

Queens University Belfast determination by the ICO in respect of the release 

of Irish Tree Ring data11, it would be helpful if the ICO could re-engage more 

generally with the Higher Education sector about their understanding of FoIA 

and EIR obligations and also consider what further guidance could be 

provided for that sector. It would be particularly useful if guidance were 

available as to how long it is reasonable to retain data without release, pending 

full publication as part of a peer reviewed paper. It is however recognised that 

often such determinations have to be made on a case-by-case basis against a 

―public interest‖ test. 

35. As a final comment we find that a fundamental lack of engagement by the CRU 

team with their obligations under FoIA/EIR, both prior to 2005 and subsequently, 

led to an overly defensive approach that set the stage for the subsequent mass of 

FoIA/EIR requests in July and August 2009. We recognise that there was deep 

suspicion within CRU, as to the motives of those making detailed requests. 

Nonetheless, the requirements of the legislation for release of information are 

clear and early action would likely have prevented much subsequent grief. 

 

11 ICO Case Ref: FS50163282 

Date: 29/03/2010 

Public Authority: Queen‘s University Belfast

Summary: The complainant requested electronic data relating to tree ring research (dendrochronology). 

The public authority confirmed that it held the requested information but refused to provide it citing 

section 12 of the Act. The Commissioner indicated to the public authority that the withheld information 

fell within the definition of environmental information under the EIR. The public authority 

subsequently cited the exceptions at regulations 12(4)(d), 12(4)(b), 12(5)(c) and 12(5)(e) to refuse the 

13 cont information. The Commissioner finds that none of the exceptions is engaged and the withheld 

information should therefore be disclosed. The Commissioner also recorded a number of procedural 

breaches in the public authority‘s handling of the request.
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

96

i EIR definition of environmental information (see paragraph 10):

Any information in written, visual, aural, electronic or any other material form on: 

(a) the state of the elements of the environment, such as air and atmosphere, 

water, soil, land, landscape and natural sites including wetlands, coastal and 

marine areas, biological diversity and its components, including genetically 

modified organisms, and the interaction among these elements; 

(b) factors, such as substances, energy, noise, radiation or waste, including 

radioactive waste, emissions, discharges and other releases into the 

environment, affecting or likely to affect the elements of the environment 

referred to in (a); 

(c) measures (including administrative measures), such as policies, legislation, 

plans, programmes, environmental agreements, and activities affecting or 

likely to affect the elements and factors referred to in (a) and (b) as well as 

measures or activities designed to protect those elements;(d) reports on the 

implementation of environmental legislation; 

(d) reports on the implementation of environmental legislation; 

(e) cost-benefit and other economic analyses and assumptions used within the 

framework of the measures and activities referred to in (c); 

(f) the state of human health and safety, including the contamination of the food 

chain, where relevant, conditions of human life, cultural sites and built 

structures inasmuch as they are or may be affected by the state of the elements 

of the environment referred to in (a) or, through those elements, by any of the 

matters referred to in (b) and (c). 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

97
CHAPTER 11: GOVERNANCE 

98

CHAPTER 11: GOVERNANCE

11.1 Introduction and Method of Enquiry 

1. This Chapter addresses the fourth part of the Review‘s remit, ―Review and make 

recommendations as to the appropriate management, governance and security 

measures for CRU and the security, integrity and release of data it holds‖, to the 

extent that these issues are not addressed in previous Chapters. The previous 

Chapter deals extensively with the major governance issue of FoIA and EIR 

compliance. This Chapter considers other relevant aspects of the governance 

framework, first as it relates to Research Management Systems in Section 11.2 

and then secondly to the specific issue of the Management of Software, Data 

Management and Security in Section 11.3. The Chapter concludes with a set of 

recommendations in Section 11.4 on governance matters relating to both issues. 

2. The Review Team decided to focus on the efficacy of current control systems in 

UEA relevant to CRU‘s research, while developing a broad understanding of the 

evolution of UEA‘s policies and practices. 

3. Interviews were carried out with the: 

Vice-Chancellor 

PVC, Research Enterprise and Engagement 

Registrar and Secretary 

Associate Dean for Research, Faculty of Science 

Director of Information Services; 

Information Communications Technology (ICT) Systems Director; and 

CRU (part-time) IT Manager 

Director of Research, Enterprise and Engagement and Manager of Research 

Services 

Research Finance Management Accountant and Faculty of Science Finance 

Manager 

Director of Human Resources 

4. Notes of the relevant meetings and supporting documentation are accessible on 

the website. 

11.2 Research Management Systems 

11.2.1 Background

5. CRU is located within the School of Environmental Sciences (ENV) of UEA. 

Historically CRU had a high degree of autonomy, with separate reporting lines 

and distinct budgets. Apart from the Director, its staff were dependent on ―soft‖ 

money, that is on grants and other sources of funding mainly from sources such as 

UK Research Councils, EU programmes and the US Department of Energy. The 

position changed with the University‘s decision to invest strongly in 

environmental science. More recently up to 5 of the CRU staff have held 
CHAPTER 11: GOVERNANCE 

99

established positions and the current full-time equivalent is 3.5. 

6. The research policies and strategy of the University are set by a hierarchy of 

committees, with CRU being represented on the ENV Committee. The Director of 

CRU reports to the Head of ENV. 

11.2.2 Funding Management 

7. Responsibility for the University‘s funding management process rests with 

Research, Enterprise and Engagement (REE), through which major bids for 

funding are channelled. More routine funding bids are handled by Faculty 

Research Officers. The documentation supporting bids requires sign-off by the 

relevant principal investigator (PI) and Head of School, confirming their 

collective and individual responsibilities in delivering the research project in 

accordance with the funder‘s terms and conditions and the University‘s and 

School‘s policies and procedures.

11.2.3 Funders‟ Requirements 

8. Funders‘ requirements vary considerably and have developed over time. All seek 

publication of results from the work they support. The traditional expectation has 

been that this will be in peer reviewed journals. There is now some pressure for 

open access publication, which also involves peer review. The emphasis on 

protecting intellectual property varies. Regular progress reports are generally 

required, often as a condition of drawing down funds; and reports with specific 

formats and deadlines may be required on completion. There are rarely detailed 

requirements for the release of data or code. 

11.2.4 Good Research Practice

9. The university has a well developed set of policies on good research practice, 

research ethics, misconduct in research (including whistle blowing and grievance 

procedures), which apply to all those working in research, as well as regular 

performance reviews. The system is continuously developing; the good practice 

guidelines were first introduced in 2003 (we saw the 2009 version). Details of 

these structures, policies and procedures are referenced with the records of our 

interviews with the relevant senior managers, on our website. 

10. Prospective students and staff are sent weblinks before arrival. There is training 

for undergraduates carrying out research as part of their coursework, and for 

postgraduate students and for new members of staff there is probationary training. 

It is the responsibility of Principal Investigators to ensure that their people are 

fully trained 

11. The University‘s statement of terms and conditions of appointment of academic, 

teaching and research staff cross-refers to the requirements of a wide range of 

university policies. There is an induction process that includes coverage of what 

the requirements are and includes the allocation of a ―mentor‖, to act as an 

advisory colleague to a new member of staff. There is a standard annual training 

programme for both supervisors and new recruits. Compliance with the 
CHAPTER 11: GOVERNANCE 

100

requirements under the terms and conditions is the responsibility of Heads of 

School, with support from Human Resources as necessary. 

12. Outputs such as publication and citations are monitored on a yearly basis, research 

plans are prepared at institutional level and have to be signed off each year, and 

individuals receive appraisal with feedback every 2 years. UEA attaches 

importance to peer review as a check on people‘s work and publications, and 

encourages academics to seek outside involvement. 

13. Whistle blowing has never been used for any research related issue (it has been 

used a couple of times in all, in relation to other matters). The research 

misconduct procedure has been used on only a few occasions, mainly in relation 

to individuals‘ doctorate or master‘s theses. 

11.2.5 Financial Controls

14. Lead universities are normally responsible for administration of grants; payments 

are controlled by funders against a profile of expenditure and pass through central 

accounts only. 

15. Where cash advances are required, there are specific, senior approval procedures. 

In exceptional circumstances advances could be made to an employee‘s bank 

account. 

16. It was reported by UEA Finance that audit procedures are robust, both in general 

and in relation to certain EU projects where these are mandatory. UK Research 

Councils are increasingly asking for specific audit checks. 

11.2.6 Risk Management

17. The University maintains a risk register, regularly reviewed by Senior 

Management and annually by the Council and the Audit Committee. In relation to 

Research, the main concern has been the risk to reputation from slippage in league 

tables, with a range of mechanisms to drive performance in the desired direction. 

The register did not address the pressures on CRU as a result of the significance of 

its work for the climate change debate and the severity of the challenge to that 

work, and therefore did not consider the support CRU might need in handling its 

data and in making it available transparently. The upsurge of FoIA requests in 

2009 was not brought to the attention of senior management soon enough; and, 

when it was, there was no prepared framework for addressing the implications. 

Action is now being taken to raise awareness of the implications of FoIA and EIR, 

with leadership from senior management. 

11.2.7 Findings on Research Management Systems 

18. The Review team‘s reading of the improperly released e-mails together with 

Sherrington1

 suggested that cash advances to researchers in Russia might not have 

been adequately controlled, see also Chapter 4, paragraph 13. Whereas nearly all 

records from the mid-1990s have now been destroyed, in keeping with UEA‘s 

policy, we have confirmed that CRU did not have its own bank account and any 

 

1

 Sherrington submission (no. 107), 4 March 
CHAPTER 11: GOVERNANCE 

101

such payment would have required authorisation in writing by the Director of 

Finance at that time. While we have been unable to deal exhaustively with the 

issue, we are advised that there is a payment of $5000 in November 1998 properly 

recorded in the UEA ledger which is consistent with e-mails 911405082.txt and 

914256033.txt at that time. 

19. Apart from those issues addressed in Chapters 6 to 10, UEA‘s Research 

Management Systems appear adequate. Requirements have developed 

significantly over the period relevant to the Review, from fairly general 

expectations that terms and conditions set by funders would be complied with, to 

the highly developed system now in operation. 

20. To summarise, the focus of risk management in research was on quality and 

standards. The critical pressure on CRU, with its attendant risk to the University‘s 

reputation, was not on the radar screen. Senior management were not sufficiently 

aware of the issue, were not alerted in time to its emergence, and had no 

preparation in place to respond adequately. 

11.3 Software, Data Management and Data Security 

11.3.1 General Context 

21. In submissions, concerns were expressed to the Review over the quality of 

software development, testing and operational processes within CRU, though it 

was also asserted that much of such software is transient, and, though poorly 

structured and documented, may still be fit for the limited purposes envisaged. 

Submissions from Bratby2 and Barnes3 are relevant. We noted that a number of 

requests received by UEA under FoIA/EIR also sought access to code. Concerns 

have also been expressed about potential loss of data by CRU and the level of 

information security for CRU systems, which were vulnerable to unauthorised 

disclosure of the personal e-mails and other material subsequently published 

widely on the Internet. 

22. IT Organisation. In common with other areas of the Science Faculty, CRU 

operates largely independently of the central IS functions of the UEA. Central IS 

has, in recent years, made significant efforts to better support the Science Faculty 

and some use of central facilities (such as the Storage Area Network) has been 

achieved. The University IS team does not provide desktop, remote access, 

hosting, database or software support to CRU, nor any quality control or 

assessment. CRU has its own local architecture based on a mix of individual PC 

based and server based processing. In common with many other research groups 

across the university, this is distinct from the UEA preferred model of client –

server operation. Internet communications for CRU is however routed over the 

university network and through the university firewall. CRU has its own IT 

Manager for whom CRU is 40% of his workload. CRU originally had no central 

backup arrangements for the individual researchers‘ PCs however its IT Manager 

introduced automated backup (using open source software) to a simple server held 

securely within the Central IS machine room. 

 

2

 Bratby submission (no. 3), 22 February 

3

 Barnes submission (no. 1), 6 March 
CHAPTER 11: GOVERNANCE 

102

 

23. Policy. A high level  ̳Information Systems Policy‘ and a related  ̳Information 

Security Policy‘4 were agreed and put in place in 2005 under the aegis of UEA‘s 

Information Systems Strategy Committee (ISSC), which includes representatives 

of all four Faculties. Low level, detailed, security policies had been developed 

and put in place by 20075

.

24. Standards. The Review is aware of a number of industry standards relevant to this 

Chapter dealing with: software development, data stewardship and security. For 

further details see Appendix 8. 

11.3.2 Issues and the Investigation 

25. The Director of Information Services indicated that, whilst the central IT function 

were aware of the existence of the CRU Backup Server, they had no knowledge of 

the nature of the information held on the server as it was managed from CRU. 

26. The CRU IT Manager indicated that researchers within CRU worked individually 

or in small groups. There was no master index of resources, be these data, 

algorithms or software. No systematic approach to the creation of metadata 

existed. There was no central database of underlying climate data; rather 

individual researchers assembled sets of data drawn from different primary 

sources outside CRU (for example the Met. Office Hadley Centre for Climate 

Change). These might arrive by network (The United Kingdom‘s Educational and 

Research Network, or JANET), or on portable hard disks. 

27. With the exception of a small amount of tree ring data, CRU does not generate 

new raw data. It relies instead on accessing existing primary data sources. 

Nonetheless the importance of maintaining clear records of what data has been 

accessed for what purpose – for example in terms of station records processed – is 

clear. The benefits of such a central data catalogue (or ―data dictionary‖) had long 

been recognised within CRU and past attempts had been made to create such a 

resource. These attempts had foundered on the lack of resources – research grants 

made no provision for this and central UEA funding had not been available. 

Whilst there was no policy for the systematic archiving of data, many of CRU‘s 

processed results datasets are available on the Web6

. 

28. Individual researchers were responsible for acquiring or developing their own 

software applications (usually written in Fortran or Interactive Data Language, 

IDL). There was no formal quality control policy or review policy. However 

individual projects would comply with whatever quality control processes were 

specified as part of their funding arrangements. 

29. At interview, the ICT Systems Director indicated that ―lessons had been learnt‖ 

and he expected (subject to the results of a security audit report) to bring forward 

proposals within the University for: 

 

4

 These can be downloaded from: http://www.uea.ac.uk/is/itregs/ictpolicies 

5 A draft ―Security Manual‖ (not available for public download) was received by the Review on 8th 

February. 

6

 See: http://www.cru.uea.ac.uk/cru/data/hrg/ 
CHAPTER 11: GOVERNANCE 

103

Greater compliance with centrally defined IS policies and architecture; 

An audit of research data held in digital storage across the University; and 

Clear data retention (and destruction) policies. 

11.3.3 Findings on Software, Data Management and Data 

Security 

30. The software development process. We found that, in common with many other 

small units across a range of universities and disciplines, CRU saw software 

development as a necessary part of a researcher‘s role, but not resourced in any 

professional sense. Small pieces of software were written as required, with 

whatever level of skill the specific researcher happened to possess. No formal 

standards were in place for: 

Software specification and implementation; 

Code reviews; and 

Software testing 

31. Data management and archiving. We found that there were no formal processes 

in place within CRU with respect to the systematic retention and archiving of data 

sets, or more particularly of metadata (data that allows the data set to be correctly 

interpreted). Individual researchers took whatever actions they deemed 

appropriate within the context of specific agreements with research funding 

bodies. 

32. Information security. We found that the basic security processes had been 

appropriately specified and documented by the UEA‘s Information Systems 

Strategy Committee. We are constrained in our detailed findings by the fact that a 

police investigation into the unauthorised release of information is ongoing. 

11.4 Recommendations 

33. Risk management. UEA should be alert to the implications for their reputation of 

the sort of challenges we have seen in this case to the work of CRU and any other 

key groups. The risk register should reflect the range of external attitudes towards 

its key units and growing criticism or the attentions of pressure groups should be 

noted. Mitigation measures should be put in place, including increased security 

and a bias for openness and properly resourced policy on data management and 

availability. Reporting arrangements should ensure that key senior management 

are in touch with the issues and are informed quickly of problems; and response 

plans should be in place and rehearsed. These points are no doubt relevant to 

many other universities. 

34. Training for researchers. We believe that Universities should develop formal 

approaches to the training of researchers in basic software development 

methodologies and best practice, as well as best practice in the handling and 

sharing of research data. 

35. Provision of a formal metadata repository. Whilst we recognize and accept that 

CRU relies on other bodies both nationally and internationally to provide and to 
CHAPTER 11: GOVERNANCE 

104

archive basic weather station data, we believe that a formal approach to the 

storage and archiving of metadata is required. Such a repository would, for 

example, have made it far easier to respond quickly to requests for the list of 

station identifiers associated with particular CRUTEM datasets. Where a 

University is hosting a unit of such international significance, we believe that it 

should ensure funding is available for such a repository either through the research 

grant process or from central resources. 

36. Role of research sponsors. We note the recent statement by the US National 

Science Foundation (NSF)7 that, from October 2010, NSF plan to make inclusion 

of a ―Data Management Plan‖ a requirement for all research proposals. It will be 

important for such plans to recognize that in some areas of science huge volumes 

of data are created and a degree of processing and compression is inevitable 

before data suitable for storage is created. We agree that the way in which 

important research data (and the associated meta data to make that data useful) 

should be preserved, should be specified by those funding such research. Explicit 

budgetary and resource provision must be made. Sponsorship arrangements 

should include a clear statement of requirements on the extent to which such data 

should be placed in the public domain and any constraints on the timing of such 

release. The guidance from the UK Research Integrity Office (UKRIO) is helpful 

in this respect.8

 

37. Making source code publicly available. We believe that, at the point of 

publication, enough information should be available to reconstruct the process of 

analysis. This may be a full description of algorithms and/or software programs 

where appropriate. We note the action of NASA‘s Goddard Institute for Space 

Science in making the source code used to generate the GISTEMP gridded dataset 

publically available. We also note the recommendation of the US National 

Academy of Sciences in its report ―Ensuring the Integrity, Accessibility and 

Stewardship of Research Data in the Digital Age‖ that: “...the default assumption 

should be that research data, methods (including the techniques, procedures and 

tools that have been used to collect, generate or analyze data, such as models, 

computer code and input data) and other information integral to a publically 

reported result will be publically accessible when results are reported...‖. We 

commend this approach to CRU. 

38. Audit processes. It is entirely acceptable that the central functions of a University 

should set, document and disseminate the standards expected across all 

governance areas, but without necessarily mandating the precise means by which 

these will be achieved. These standards will reflect the University‘s interpretation 

of applicable law (Data Protection, Computer Misuse, Health & Safety, 

Environmental Information Regulations) and best practice. In areas such as 

Information Systems, it may well be appropriate to allow a degree of local 

autonomy. However it is then essential that robust audit procedures are in place to 

ensure that, where local solutions are implemented, these do meet fully the 

standards specified. 

 

7

 NSF Press Release 10-077 

http://www.nsf.gov/news/news_summ.jsp?cntn_id=116928&org=NSF&from=news 

8

See UKRIO  ̳Code of Practice for Research‘ Section 3.12 ―Collection and Retention of Data‖ at 

http://www.ukrio.org 
CHAPTER 11: GOVERNANCE 

105
106

ACKNOWLEDGEMENTS 

The Review Team would like to acknowledge the help and support they received 

during the conduct of the Review and in the preparation of this report. Our thanks go 

to: 

William Hardie who provided Secretariat support. 

David Walker who acted as our Project Manager. 

Kate Moffat, Adrian Dias, Helen Davison and Mike Granatt who provided 

communications support and advice, set up and handled the Review website, and 

managed the production of the report. 
107
APPENDIX 1

108

APPENDIX 1: REVIEW TEAM MEMBERS 

TEAM CV‘s

Sir Muir Russell, KCB, FRSE (Chair) 

Sir Muir Russell was Principal and Vice-Chancellor of the University of Glasgow 

from 2003 to 2009. During that period he was Convener of Universities Scotland, a 

member of the Universities UK Main Board, a Trustee of the Universities 

Superannuation Scheme, and a member of the UCAS Board. He graduated from the 

University of Glasgow in 1970 with a First in Natural Philosophy and took up a career 

in the civil service. He was appointed Permanent Secretary to the Scottish Office in 

1998, and was the first Permanent Secretary to the Scottish Executive following the 

establishment of the Scottish Parliament in 1999. He has honorary degrees from the 

Universities of Strathclyde, Glasgow and Edinburgh. He currently chairs the Judicial 

Appointments Board for Scotland. He is also a Trustee of the Glasgow School of Art, 

a Member of the Board of the Moredun Research Institute, and the Chairman of the 

Dunedin Concert Trust. 

Professor Geoffrey Boulton, OBE, FRS, FRSE 

Professor Geoffrey Boulton is Regius Professor Emeritus of Geology and former Vice 

Principal of the University of Edinburgh. His research is in the fields of glaciology, 

glacial geology and Quaternary science, and has been awarded several international 

awards and honorary degrees for his scientific work. He currently has research 

projects in Antarctica and Iceland. He has been the UK representative to the 

International Union of Geosciences and to the International Union of Quaternary 

Sciences. He is a member of the UK Prime Minister‘s Council for Science and

Technology, chairs the Advisory Board of the University of Heidelberg and is 

General Secretary of the Royal Society of Edinburgh, Scotland‘s national academy. 

He has been a member of the Councils of the Natural Environment Research Council 

and the Royal Society, a member of the Royal Commission on Environmental 

Pollution, the Scottish Science Advisory Committee and the Scottish Higher 

Education Funding Council. He was formerly Head of the Department of Geology 

and Geophysics and Provost of Science and Engineering in the University of 

Edinburgh. 

Professor Peter Clarke, FInstP, CPhys, FIET, CEng 

Peter Clarke is Professor of Physics at the University of Edinburgh. He has a B.Sc in 

Electronics Engineering (Southampton University, 1980) and a D.Phil in Particle 

Physics (Oxford 1985). He was a CERN Fellow before being appointed as a lecturer 

first at Brunel University in 1987 and then University College London in 1993. He 

was promoted to Reader and then Professor in 2001 and was Head of the Particle 

Physics Research Group between 2001-04. He moved to the University of Edinburgh 

in 2004 to take up the Chair of eScience and later become Director of the National 

eScience Centre 2006-09. 

David Eyton MA, MIoM3, CEng. 

David Eyton is Group Head of Research & Technology at BP, and was appointed in 

April 2008. He is accountable for technology strategy and its implementation across 

BP and conducting research and development (R&D) in areas of corporate renewal. 
APPENDIX 1

109

In this role, David oversees the R&D capability of the company and also sits on the 

UK Energy Technologies Institute and Science¦Business Innovation Boards. During 

his career he has held a number of Petroleum Engineering, Commercial and Business 

Management positions in the UK, Australia, Trinidad and USA. 

Professor James Norton, CDir, CEng, CITP, FIoD, FIET, 

FBCS, FRSA 

Aged fifty-seven, Jim Norton is an independent director and policy adviser. He is an 

external member of the Board of the UK Parliament's Office of Science & 

Technology (POST) and council member of the Parliamentary IT Committee 

(PITCOM). Jim is a Non-Executive Director of F&C Capital & Income Investment 

Trust plc, where he chairs the Audit & Management Engagement Committee. He is a 

Board Member and Trustee of the Foundation for Information Policy Research 

(FIPR), as well as a member of the 'Electronic Communications Expert Advisory 

Panel' for the Irish Commission for Communications Regulation (ComReg). Jim is a 

Vice-President and Trustee of the BCS – Chartered Institute for IT, an External 

Examiner for the Institute of Directors, and Chair of the IT Policy Panel for the 

Institution of Engineering & Technology. He also chairs the Steering Group for the 

Secure Software Development Partnership (SSDP) of the Technology Strategy Board. 
APPENDIX 2

110

APPENDIX 2: APPROACH AND WORK PLAN

1. This document sets out how the Review Team has approached its terms of 

reference, and the way in which it will fulfil its remit. 

2. The terms of reference are: 

―The Independent Review will investigate the key allegations that arose from 

a series of hacked e-mails from the University of East Anglia‘s Climatic 

Research Unit (CRU). The review will: 

2.1. Examine the hacked e-mail exchanges, other relevant e-mail exchanges 

and any other information held at CRU to determine whether there is any 

evidence of the manipulation or suppression of data which is at odds with 

acceptable scientific practice and may therefore call into question any of the 

research outcomes.

2.2. Review CRU‘s policies and practices for acquiring, assembling, 

subjecting to peer review and disseminating data and research findings, and 

their compliance or otherwise with best scientific practice.

2.3. Review CRU‘s compliance or otherwise with the University‘s policies 

and practices regarding requests under the Freedom of Information Act ( ̳the 

FoIA‘) and the Environmental Information Regulations ( ̳the EIR‘) for the 

release of data.

2.4. Review and make recommendations as to the appropriate management, 

governance and security structures for CRU and the security, integrity and 

release of the data it holds.‖

3. The remit requires the Review to address the specific allegations about the way in 

which CRU has handled its data, reflecting comments in the e-mail exchanges that 

have been made public. In a separate paper – Issues for examination - the Team 

has set out its initial view of the questions that need to be addressed. It will seek 

written submissions from CRU and other appropriate parts of UEA. It will also 

invite interested parties to comment on what the Issues paper covers, and to 

propose any further matters that clearly fall within the Remit and should also be 

examined. 

4. The Review‘s remit does not invite it to re-appraise the scientific work of CRU. 

That re-appraisal is being separately commissioned by UEA, with the assistance 

of the Royal Society. The Review‘s conclusions will complement that re-appraisal 

by pointing to any steps that need to be taken in relation to data, its availability 

and its handling. 

5. The Team wishes to focus on the honesty, rigour and openness with which CRU 

handled its data. It wishes to gain a proper understanding of: 

The range of data involved, and how it has been indexed and archived. 
APPENDIX 2

111

The procedures, processes and relevant protocols used to handle the data, 

recognizing that these may have changed over time as data-handling 

capacity has developed. 

The associated metadata, algorithms and codes used for analysis. 

The extent to which other independent analysis produces similar 

conclusions. 

The peer review process, examining how much was in common between 

the work of the reviewers and the reviewed. 

6. In making its analysis and conclusions, the Team will test the relevant work 

against pertinent standards at the time it was done, recognizing that such standards 

will have changed. It will also test them against current best practice, particularly 

statements of the ethics and norms such as those produced by the UK Government 

Office for Science and by the US National Academy of Sciences. These identify 

principles relating to rigour, respect and responsibility in scientific ethics and to 

integrity, accessibility and stewardship in relation to research data. This overall 

approach will allow the Team to establish a conceptual framework within which it 

can make judgements and comment about key issues such as the level of 

uncertainty inherent in all science, and the particular confidence limits associated 

with the CRU work. 

7. The police and the Information Commissioner are also considering issues in 

connection with the leaked e-mails, and the Team has established appropriate, 

continuing liaison with them. 

8. The Team‘s analysis and conclusions will include not only a view of what has 

happened in the past, but also comments and recommendations on best practice 

for the future. This will be done both at the level of CRU and the University as a 

whole, and may have broader implications for institutions undertaking scientific 

work. 

9. The Team will operate as openly and transparently as possible. It is establishing a 

website which will eventually display all of the submissions received, 

correspondence, analyses and conclusions. The aim will be to publish all received 

submissions quickly, unless there are wholly exceptional reasons to delay, for 

example legal issues. 
APPENDIX 3

112

APPENDIX 3 

The questions, criticisms and allegations raised in the „Issues for Examination‟ paper 

below are not intended to be read in isolation. Readers are referred to the Review‟s 

findings on these points as set out in the body of the report. 

ISSUES FOR EXAMINATION

1. The Review Team’s remit is set out in its terms of reference as follows: 

―The Independent Review will investigate the key allegations that arose from a 

series of hacked e-mails from the University of East Anglia‘s Climatic Research 

Unit (CRU). The review will: 

Examine the hacked e-mail exchanges, other relevant e-mail exchanges and any 

other information held at CRU to determine whether there is any evidence of the 

manipulation or suppression of data which is at odds with acceptable scientific 

practice and may therefore call into question any of the research outcomes.

Review CRU‘s policies and practices for acquiring, assembling, subjecting to peer 

review and disseminating data and research findings, and their compliance or 

otherwise with best scientific practice.

Review CRU‘s compliance or otherwise with the University‘s policies and 

practices regarding requests under the Freedom of Information Act ( ̳the FoIA‘) 

and the Environmental Information Regulations ( ̳the EIR‘) for the release of data.

Review and make recommendations as to the appropriate management, 

governance and security structures for CRU and the security, integrity and release 

of the data it holds.‖

1. The remit reflects the reaction to the e-mails that became public, much of it 

questioning and critical. In following up those questions and criticisms the Team 

stresses that it has formed no view on whether they are fair or justified. In 

formulating in its own words an expression of the issues for examination the 

Team is not adopting those issues as its own criticisms. 

2. The Team‘s approach is to distill the questions and criticisms into the broad 

questions set out below. Using its own enquiries and experience, it has added 

questions about the handling and dissemination of data, including the response to 

FOI requests. The issues addressed in the first three paragraphs of the terms of 

reference will inform recommendations for paragraph four, as to the appropriate 

management, governance and security structures for CRU and the release of data.

 

3. The Team will invite CRU and other parts of UEA to respond in writing to these 

questions, and will follow up those responses as required. The Team expect the 

CRU to provide original documentary evidence to support its responses. 

4. The Team invites those with an interest in the matter to comment on this Issues 

paper. The Team stresses that its remit does not involve re-evaluation of the 
APPENDIX 3

113

scientific conclusions of the CRU work, still less a reappraisal of the scientific 

debate about the existence and suggested causes of global warming. Please 

confine any comments on this paper to matters within the remit at paragraph 1 of 

this paper. 

5. Written responses from CRU and others are requested by 1 March 2010 to the 

address below:

Email: correspondence@cce-review.org 

Or post to, 

Climate Change E-Mails Review 

Box 18 

196 Rose Street 

Edinburgh 

EH2 4AT

ISSUES ARISING ON Para 1.1 OF THE TERMS OF 

REFERENCE 

6. The allegation of ignoring potential problems in deducing 

palaeotemperatures from tree ring data that might undermine the validity of 

the so-called “hockey-stick” curve.

7. In the late 20th century, the correlation between the tree ring record and 

instrumental record of temperature change diverges from that for the earlier 

period. The cause of this divergence does not appear to be understood. If the 

method used to deduce temperatures from tree ring proxy metrics for the earlier 

tree ring record is applied to the late 20th century tree ring series, then declining 

temperatures would be deduced for the late 20th century. It is alleged that if the 

cause of divergence between the tree ring and instrumental temperature record is 

unknown, it may have existed in earlier periods. Therefore if tree rings had 

similarly failed to reflect the warming of the early Middle Ages, they may 

significantly under-estimate the warming during the Medieval Warm Period, thus 

falsely enhancing the contrast between the recent warming and that earlier period. 

(It is this contrast that has led to statements that the late 20th century warming is 

unprecedented during at least the last 1000 years.) 

QUESTIONS TO ADDRESS: 

What method do you use to deduce palaeotemperatures from tree ring 

data? 

Does not the problem of divergence for the late 20th century record 

invalidate the deduction of tree ring palaeotemperatures for the period 

prior to the instrumental record? 

How open have you been about this issue? 

What attempts have you made to resolve it? 
APPENDIX 3

114

What is the evidence that the amplitude of warming during the Medieval 

Warm Period (MWP) is not underestimated by tree ring evidence? 

How does the tree ring evidence of the MWP compare with other proxy 

data? Have you showed how data from different sources compare or have 

you conflated them? If the latter, what is the justification? 

If tree ring proxies are removed from reconstructions, what evidence 

remains of the MWP? 

Have you been selective in utilizing tree ring evidence from Yamal in 

Siberia; and if so, what is the justification for selectivity and does the 

selection influence the deduced pattern of hemispheric climate change 

during the last millennium? 

8. The allegation that CRU has colluded in attempting to diminish the 

significance of data that might appear to conflict with the 20th century global 

warming hypothesis 

9. The CRU group, in consultation with Professor Michael Mann, is alleged to have 

systematically attempted to diminish the significance of the Medieval Warm 

Period, evidenced by an email from Mann 4th June 2003: ―I think that trying to 

adopt a timeframe of 2K, rather than the usual 1K, addresses a good earlier point 

that Peck made w/ regard to the memo, that it would be nice to try to "contain" the 

putative "MWP", even if we don't yet have a hemispheric mean reconstruction 

available that far back [Phil and I have one in review--not sure it is kosher to show 

that yet though--I've put in an inquiry to Judy Jacobs at AGU about this].‖ The use 

of the words ―contain‖ and ―putative‖ are alleged to imply an improper intention 

to diminish the magnitude and significance of the MWP so as to emphasise the 

late 20th century warming. 

QUESTIONS TO ADDRESS

What does the word “contain” mean in this context? 

What involvement have you had in “containing” the MWP?

How important is the assertion of “unprecedented late 20th century 

warming” in the argument for anthropogenic forcing of climate?

10. It is alleged that proxy temperature deductions and instrumental 

temperature data have been improperly combined to conceal mismatch 

between the two data series 

11. An attempt to hide the difficulty of combining these two data series and to mislead 

is alleged to be revealed in the following sentence in a November 1999 email from 

Professor Phillip Jones which is alleged to imply a conscious attempt to mislead: 

"I've just completed Mike's Nature trick of adding in the real temps to each series 

for the last 20 years (i.e. from 1981 onwards) and from 1961 for Keith's to hide 

the decline‖. 

QUESTIONS TO ADDRESS

What is the meaning of the quotation from the 1999 email? 
APPENDIX 3

115

How do you justify combining proxy and instrumental data in a single 

plotted line? 

What method do you use? 

12. It is alleged that there has been an improper bias in selecting and adjusting 

data so as to favour the anthropogenic global warming hypothesis and details 

of sites and the data adjustments have not been made adequately available 

13. It is alleged that instrumental data has been selected preferentially to include data 

from warmer, urban in contrast to rural sites; that the rationale for the choice of 

high/low latitude sites is poor; and that the processes by which data has been 

corrected, accepted and rejected are complex and unclear. 

QUESTIONS TO ADDRESS

What is the rationale for the choice of data stations worldwide? 

How has this choice been tested as appropriate in generating a global or 

hemispheric mean temperature (both instrumental and proxy data)? 

Describe as clearly as possible the protocols you have followed in 

selecting, correcting and rejecting data and stations. 

Has this been an orderly and objective process applied to all datasets? 

To what extent have different procedures for data of different vintages and 

different sources been unified? 

What means do you use to test the coherence of the datasets? 

ISSUES ARISING ON Para 1.2 OF THE TERMS OF 

REFERENCE 

14. It is alleged that there have been improper attempts to influence the peer 

review system and a violation of IPCC procedures in attempting to prevent 

the publication of opposing ideas.

15. It is alleged that there has been an attempt to subvert the peer review process and 

exclude publication of scientific articles that do not support the Jones-Mann 

position on global climate change. A paper by Soon & Balunias was published in 

the Journal Climate Research arguing that the 20th century was abnormally warm. 

An email from Professor Michael Mann on 11th March 2003 contained the 

following: "I think we have to stop considering Climate Research as a legitimate 

peer-reviewed journal. Perhaps we should encourage our colleagues in the climate 

research community to no longer submit to, or cite papers in, this journal." The 

allegation is that journals might be pressured to reject submitted articles that do 

not support a particular view of climate change. 

16. In an email to a fellow researcher in June 2003, Briffa wrote: ―Confidentially I 

now need a hard and if required extensive case for rejecting (an unnamed paper) 

to support Dave Stahle‘s and really as soon as you can.‖

 

17. In an email to Mann on 8th July 2004, Jones wrote: "The other paper by MM is 

just garbage. [...] I can't see either of these papers being in the next IPCC report. 
APPENDIX 3

116

Kevin and I will keep them out somehow — even if we have to redefine what the 

peer-review literature is!" The allegation is of an attempt to prevent ideas being 

published and the author being prepared to subvert the peer review process for a 

journal and to undermine the IPCC principle of accounting properly for 

contradictory views. 

QUESTIONS TO ADDRESS 

Give full accounts of the issue in relation to the journal Climate Research, 

the June 2003 email, and the March 2004 email to Mann (“recently 

rejected two papers (one for Journal of Geophysical Research & one for 

Geophysical Research Letters) from people saying CRU has it wrong over 

Siberia. Went to town over both reviews, hopefully successfully. If either 

appears I will be very surprised”.

Are the first two instances evidence of attempts to subvert the peer review 

process? 

In relation to the third, where do you draw the line between rejecting a 

paper on grounds of bad science etc, and attempting to suppress contrary 

views? 

To what extent is your attitude to reviewing conditioned by the extent that 

a paper will set back the case for anthropogenic global warming and the 

political action that may be needed to mitigate it? 

What is the justification for an apparent attempt to exclude contrary views 

from the IPCC process? 

18. The scrutiny and re-analysis of data by other scientists is a vital process if 

hypotheses are to rigorously tested and improved. It is alleged that there has 

been a failure to make important data available or the procedures used to 

adjust and analyse that data, thereby subverting a crucial scientific process. 

19. It is alleged that there has been a systematic policy of denying access to data that 

has been used in publications, referring to an email from Jones to Mann on 2nd

February 2005 which contains the following: 

"And don't leave stuff lying around on ftp sites - you never know who is trawling 

them. The two MMs have been after the CRU station data for years. If they ever 

hear there is a Freedom of Information Act now in the UK, I think I'll delete the 

file rather than send to anyone. Does your similar act in the US force you to 

respond to enquiries within 20 days?—our does! The UK works on precedents, so 

the first request will test it. We also have a data protection act, which I will hide 

behind‖. 

QUESTIONS TO ADDRESS

Do you agree that releasing data for others to use and to test hypotheses 

is an important principle? 

If so, do you agree that this principle has been abused? 

If so, should not data be released for use by those with the intention to 

undermine your case, or is there a distinction you would wish to make 

between legitimate and illegitimate use? 
APPENDIX 3

117

If not, do others have reasonable access to the data at all levels and to the 

description of processing steps, in order to be able to carry out such a reanalysis? 

Can you describe clearly the data-sets and relevant meta-data that have 

been released; what has not been released and to what extent is it in 

useable form? Where has it been released? 

Where access is limited, or not possible, or not meaningful, for legitimate 

reasons please explain why? 

20. The keeping of accurate records of datasets, algorithms and software used in 

the analysis of climate data. 

21. A key concern expressed by a number of correspondents and commentators has 

been as to whether datasets, and analyses based thereon, were deleted. 

QUESTIONS TO ADDRESS 

Were formal „data dictionaries‟ kept of the data sets acquired by the CRU 

at various times from other bodies such as the UK Meteorological Office 

Hadley Centre and its equivalents around the World? 

Were comprehensive records kept of the way these various data sets were 

used, the statistical and other algorithms used in processing them, and the 

various software programmes and modules used to carry out that 

processing? 

Does a formal library of algorithms and software used by the CRU exist? 

What quality control measures were used to test the various algorithms 

and software modules developed by the CRU? 

What techniques did members of the CRU employ to ensure the integrity 

of the various applications used to process climate data? 

What policies are in place to ensure the formal archiving of data sets and 

resultant analyses for future use and review. 

ISSUES ARISING ON Para 1.3 OF THE TERMS OF 

REFERENCE 

22. Response to Freedom of Information requests. 

23. A number correspondents and commentators assert that requests under the 

Freedom of Information Act (FoIA) and the Environmental Information 

Regulations (EIR) were incorrectly denied by the University of East Anglia on 

advice from the CRU. This is the subject of a separate inquiry by the Information 

Commissioner, but does fall within the terms of reference of the Review Team. 

QUESTIONS TO ADDRESS 

What formal processes were in place both centrally and within the CRU to 

ensure fair and impartial assessment of FoIA requests? 
APPENDIX 3

118

Were there any processes in place centrally to review recommendations 

from the CRU that information should not be released? 

Over the five years to November 2009: 

o how many requests were received? 

o how many were rejected, and on what grounds? 

o how many received full release of information? 

o how many received partial release of information? 

Independent Climate Change E-Mails Review, February 2010 
APPENDIX 3

119
APPENDIX 4

120

APPENDIX 4: INDEX OF MEETINGS, INTERVIEWS, 

SUBMISSIONS, FOLLOW UP ENQUIRIES AND 

RESPONSES 

Team Meetings were held on the following dates in 2010. Confirmed notes of Team 

Meetings can be viewed at: http://www.cce-review.org/Meetings.php 

12 January 

4 February 

25 February 

20 March 

1 April 

13 April 

22 April 

28 April 

11 May 

26 May 

7 June 

11 June 

15 June 

Other Meetings and Interviews conducted. Notes of the meetings and supporting 

documentation can be accessed from: http://www.cce-review.org/Evidence.php 

18 December 2009, meetings conducted by Sir Muir Russell at the UEA. Notes were 

taken by Lisa Williams (Senior Assistant Registrar), UEA: 

Brian Summers (Registrar and Secretary), Professor Trevor Davies (PVC 

Research, Enterprise and Engagement), Professor David Richardson (Dean, 

Faculty of Science), UEA 

Stuart Holmes, (Chair, UEA Council), UEA 

Jonathan Colam-French (Director of Information Services), Iain Reeman (ICT 

Systems Director), Steve Mosley (ICT Policy Manager), UEA 

Brian Summers (Registrar and Secretary), David Palmer (Information Policy 

& Compliance Manager) and Jonathan Colam-French (Director of Information 

Services), UEA 

Alan Preece (Director of Marketing and Communications) and Annie Ogden 

(Head of Communications), UEA 

Professor Trevor Davies (PVC Research, Enterprise and Engagement) and 

Professor Philip Jones (Climatic Research Unit), UEA 

Professor Trevor Davies (PVC Research, Enterprise and Engagement) and 

Professor Keith Briffa (Climatic Research Unit), UEA 

Superintendent Julian Gregory, Norfolk Constabulary (meeting note withheld 

to avoid prejudicing ongoing police investigation). 
APPENDIX 4

121

27 January 2010, meetings conducted by Sir Muir Russell and Professor Jim Norton 

at the UEA: 

Mike Gorrill (Head of Enforcement) and David Clancy (Investigations 

Manager) of the Information Commissioner‘s Officer 

Jonathan Colam-French (Director of Information Services) and Mike Salmon 

(IT Manager to the CRU - 40% time), UEA 

Professor Philip Jones and Professor Keith Briffa, Climatic Research Unit, 

UEA 

Superintendent Julian Gregory and Andy Guy from Norfolk Constabulary 

(meeting note withheld to avoid prejudicing ongoing police investigation) 

4 March 2010, interviews conducted by Professor Peter Clarke and Professor Jim 

Norton at the UEA: 

Professor Philip Jones, Dr Tim Osborn and Ian Harris, Climatic Research 

Unit, UEA 

24 March 2010, meeting conducted by Professor Jim Norton at the Information 

Commissioner‘s Office:

Mike Gorrill (Head of Enforcement) and Steve Wood (Assistant 

Commissioner FOI) 

26 March 2010, meetings conducted by Sir Muir Russell and David Eyton at the 

UEA: 

Ian McCormick (Director of Research, Enterprise and Engagement) and Alan 

Walker (Manager of Research Services), UEA 

Rob Bell (Research Finance Management Accountant) and Laura McGonagle 

(Faculty of Science Finance Manager), UEA 

Professor Trevor Davies (PVC Research, Enterprise and Engagement), UEA 

Professor Trevor Davies (PVC Research, Enterprise and Engagement) and 

Professor David Russell (Associate Dean for Research, Faculty of Science), 

UEA 

Cecile Piper (Director of Human Resources), UEA 

Brian Summers (Registrar and Secretary), UEA 

Professor Edward Acton (Vice-Chancellor), UEA 

30 March 2010, meetings conducted by Sir Muir Russell and Professor Jim Norton at 

the UEA: 

Michael McGarvie (Science Faculty FoIA contact), UEA 

David Palmer (Information Policy & Compliance Manager) and Jonathan 

Colam-French (Director of Information Services), UEA 

9 April 2010, meetings conducted by Professor Geoffrey Boulton and Professor Peter 

Clarke at the UEA: 

Professor Keith Briffa, Professor Philip Jones, Dr Tim Osborn and Dr Tom 

Melvin (Climatic Research Unit), David Palmer (Information Policy & 
APPENDIX 4

122

Compliance Manager) and Jonathan Colam-French (Director of Information 

Services), UEA 

15 June 2010, telephone interview conducted by Sir Muir Russell and David Eyton 

with Professor Trevor Davies, PVC Research, Enterprise and Engagement 

Telephone interviews conducted by Professor Geoffrey Boulton: 

1 May, Professor Sir Brian Hoskins, IPCC AR4 Review Editor Ch 3 (Surface 

and Atmospheric Climate Changes) 

1 June, Professor John Mitchell, IPCC AR4 Review Editor Ch 6 

(Palaeoclimate). 

Follow- up enquiries, responses and related correspondence. This material can be 

accessed from: http://www.cce-review.org/Evidence.php 

Presentations from 20 March Review Team Meeting 

Independent forensic analyst report and commentary on e-mail examination 

Redacted set of FOI and EIR requests received relating to the Climatic 

Research Unit since 2005 

Correspondence with Lord Oxburgh in respect of the separate reviews 

Follow-up request to Professor Jones and response on issues relating to the 

IPCC 

Clarification request to Professor Jones and response on issues relating to the 

IPCC 

Follow-up request to Professor Jones and response in relation to raw 

instrument station availability for each CRUTEM data set 

Follow-up request to Professor Briffa and response in relation to best scientific 

practice in dissemination scientific evidence into the public domain 

Follow-up request and response on the scope of material that was backed up 

on the server that was compromised 

Follow-up request and response in relation to the UEA‘s main university email server domain 

Follow-up request and response in relation to governance and risk 

management issues 

Follow-up request and response in relation to financial controls 

Follow-up request to ICO and response in relation to the holding of 

information or data 

Response to the Science and Technology Committee‘s Eight Report on The 

Disclosure of Climate Data from the Climatic Research Unit (Note: at the time 

of writing, the Science and Technology Commons Select Committee has not 

given permission to publish this response)

Submissions

The Review was launched on 11 February and submissions were invited from this 

date and accepted up to 16 April. The Review accepted a small number of 

submissions that it deemed to have significant relevance to its remit after this close 

date. 
APPENDIX 4

123

Submissions to the Review can be viewed at: http://www.ccereview.org/Evidence.php 

Submissions Published

Submission Name Date received 

Number 

1 Nicholas Barnes 06-Mar

2 David Archer 25-Feb

3 Dr Phillip Bratby 22-Feb

4 Philip Brohan 24-Feb

5 Climatic Research Unit 01-Mar

6 David Cockroft 28-Feb

7 Gill Chant 13-Feb

8 Tor Berge S. Gjersvik 12-Feb

9 Professor Gabriele Hegerl 02-Mar

10 Michael Hughes 17-Feb

11 Professor Mike Hulme 26-Feb

12 Professor Dr Fortunat Joos 01-Mar

13 Dr D.R. Keiller 14-Mar

14 Andrew MacIntyre 12-Feb

15 Dr Ross McKitrick 26-Feb

16 Dr P.C. Matthews 01-Mar

17 Forrest M.Mims III 18-Feb

18 Andrew Montford 11-Feb

19 Professor Raymond Bradley 01-Mar

20 Jim Stathos 26-Feb

21 Professor Simon Tett 01-Mar

22 The Global Warming Policy Foundation 28-Feb

23 Stephen McIntyre 02-Mar

24 Professor Rob Wilby 26-Feb

25 Dr Austin Woods 11-Feb

26 Professor Malcolm Hughes 01-Mar

27 W.F. Lenihan 15-Feb

28 Richard Calhoun 14-Feb

29 Dr Henry Barnard 27-Feb

30 Ron Cram 15-Feb

31 Charlie Kilgore 13-Feb

32 Vincent Moran 14-Feb

33 Dr Richard North 11-Feb

34 Stephen Richards 15-Feb

35 Cameron Rose 16-Feb

36 Dr C.W. Schoneveld 15-Feb

37 David Shepherd 13-Feb

38 Geoffrey Sherrington 17-Feb

39 Martin Vermeer 27-Feb

40 Robert Wright 15-Feb
APPENDIX 4

124

41 Brent Hargreaves 16-Feb

42 Professor Michael Mann 28-Feb

43 Dr Sonja Boehmer-Christiansen 23-Feb

44 Dr Benjamin Santer 28-Feb

45 Trevor Jones 28-Feb

46 D R G Andrews 08-Apr

47 Professor Joshua Halpern 08-Apr

48 Mike Haseler 22-Feb

49 John R. Smith 17-Feb

50 Dr D.R. Keiller 17-Feb

51 Dr D.R. Keiller 17-Feb

52 Dr David Lehmiller 15-Feb

53 Dr D.R. Keiller 24-Mar

54 Bob Smith 19-Feb

55 Robert Wright 19-Feb

56 Ian MacDonald 28-Feb

57 Dr David Lehmiller 15-Feb

58 Stephen Richards 11-Feb

59 Ray Soper 17-Feb

60 David Sandeman 18-Feb

61 Dr Josep Verges 15-Feb

62 Dominic O'Kane 15-Feb

63 Vid Stimac 17-Feb

64 Chris Allen 12-Feb

65 Ian MacDonald 20-Feb

66 Tony Brown 13-Feb

67 Gerry Morrow 12-Feb

68 Stephen Graves 12-Feb

69 Will Hawkes 12-Feb

70 C. Barling 12-Feb

71 Susan Ewens 13-Feb

72 Malcolm McClure 13-Feb

73 Robert Owen 13-Feb

74 Bruce Garrick 13-Feb

75 Stephen Johnson 13-Feb

76 Jim Stathos 13-Feb

77 Dr Barry Napier 14-Feb

78 Joe Olson 16-Feb

79 Roderick Campbell 16-Feb

80 John R. Smith 16-Feb

81 Ray Soper 18-Feb

82 Julie Grace 17-Feb

83 Doug Vickers 15-Feb

84 David Shepherd 15-Feb

87 Mike Haseler 12-Feb

88 Dr D.R. Keiller 06-Mar
APPENDIX 4

125

89 Patrick Keane 17-Feb

90 Patrick Keane 18-Feb

91 Dr D.R. Keiller 20-Feb

92 Richard Calhoun 11-Feb

93 Robert Denton 18-Feb

94 Robert Denton 27-Feb

95 John Graham-Cumming 17-Feb

96 D.G. McBeath 22-Feb

97 George Mihailides 28-Feb

98 David Shepherd 11-Feb

99 Patrick Keane 15-Feb

100 Dr D.R. Keiller 23-Feb

101 Dr D.R. Keiller 26-Feb

102 Dr D.R. Keiller 28-Feb

103 Climate Scientists Letter 28-May

104 McIntyre Submission 9-Jun

105 Dr McKitrick supplementary sub 1 13-Apr

106 Dr McKitrick supplementary sub 2 13-Apr

107 Geoffrey Sherrington 4-Mar

108 Professor Tom Wigley 28-Feb

109 Patrick Keane 12-Feb

Submissions Not Published

David Holland 

Conor McMenemie (submission to be treated as confidential) 

The Review reserved the right to withhold publication of submissions if they were 

abusive, potentially defamatory, anonymous, or there were other legal difficulties 

which prevented publication. 
APPENDIX 5

126

APPENDIX 5: PEER REVIEW 

UNDERSTANDING UNCERTAINTY: A BRIEF HISTORY 

OF PEER REVIEW

By Richard Horton, Editor of The Lancet 

Amid the public and scientific furore over alleged events at the University of East 

Anglia‘s Climatic Research Unit (CRU), peer review has emerged as a central issue in 

the dispute. In the Times Higher Education, for example, Andrew Montford, author of 

The Hockey Stick Illusion: Climategate and the Corruption of Science (1), argued that 

events at the CRU had far-reaching implications for the world of scientific publishing 

(2). His charge sheet was extensive – undermining the peer-review process, 

threatening editors who published work contrary to orthodox scientific opinion, 

organising mass resignations from editorial boards, and persuading colleagues to stop 

submitting papers to allegedly offending journals. Montford suggests that ―as many as 

four different journals may have had their normal procedures interfered with‖. He 

continues, 

 “What is an ethical way to deal with a journal?...

 At what point does valid protest elide into something 

 more sinister?...If, as the [CRU] emails suggest, some 

scientists are in fact putting illegitimate pressure on 

journals, either to influence the peer-review process or 

to prevent the release of data, it is easy to see how editors 

may find it difficult to respond.”

Implicit in Montford‘s argument is that peer review is critical to the process of – and 

thereby public trust in – science. Writing in The Guardian, George Monbiot put it this 

way: ―science happens to be [a] closed world with one of the most effective forms of 

self-regulation: the peer review process.‖(3). 

The importance of peer review has been invoked by climate sceptics in other domains 

of the climate debate. Christopher Booker has challenged Dr Rajendra Pachauri, for 

example, for claiming that his Intergovernmental Panel on Climate Change (IPCC) 

included only peer-reviewed research (4). By contrast, Booker reports that a third of 

IPCC sources were newspaper articles, student dissertations, even press releases. 

Again, the suggestion is that the peer-reviewed literature is something special and 

sacred to science. One can make strong and reliable assertions if those statements are 

underpinned by peer-reviewed science. If evidence has not been peer-reviewed, it is 

next to worthless. 

In the context of the CRU, there have been claims that peer reviewers censored 
APPENDIX 5

127

evidence deliberately to exclude those findings from scientific journals. Questions 

have been raised about the line between rejecting a paper on grounds of bad science 

and rejecting it as part of an attempt to suppress contrary opinions. Peer reviewers 

have been accused of letting their politics get in the way of science. Did reviewers‘ 

concerns about the potential adverse policy and political implications of supporting 

perfectly good science at peer review override their scientific judgment? The 

suggestion is that they did. 

The House of Commons Science and Technology Committee report on the CRU (5) 

has emphasised that ―the question of the operation of peer review is going to be a 

critical issue‖ in any evaluation of the CRU‘s work. In their report, in a section 

headed ―Perverting the peer review process‖, MPs set out several specific allegations 

against CRU scientists. First, that they colluded to subvert peer review, slowing or 

blocking the publication of research which disagreed with their own views. Second, 

that climate scientists planned to ―redefine what the peer-review literature is‖ in their 

recommendations to the IPCC (6). Third, that they tried to suppress a paper on 

research fraud. And finally, that CRU staff exerted improper pressure on the editor of 

one journal, an editor who had been open to scientists challenging climate change 

orthodoxy. The House of Commons Committee concluded that there was no evidence 

to suggest that the CRU had tried to subvert the peer review process. They wrote that, 

―Academics should not be criticised for making informal comments on academic 

papers‖.

Much of the concern – and, indeed, confusion – about what took place at the CRU in 

relation to peer review may stem from misunderstandings about what peer review is 

and what it can be expected to do. 

Peer review: firewall or the weakest link? 

For scientific journals, peer review is the confidential evaluation of a submitted 

manuscript by one or more individuals who are experts in an aspect of the work under 

scrutiny. 

Who invented peer review? It‘s hard to be sure, but possibly the prize goes to Ishaq 

bin Ali Al Rahwi (AD 854-931) (7). In his book, Ethics of the Physician, Al Rahwi 

apparently encouraged doctors to keep contemporaneous notes on their patients, later 

to be reviewed by a jury of fellow physicians. But the serious business of journal peer 

review had to wait another 800 years. Henry Oldenburg, editor of Philosophical 

Transactions of the Royal Society, was the first modern editor to adopt peer review in 

the seventeenth century. He used it to famous effect, provoking often fractious, but 

illuminating, debates between scientists across Europe. 

Technology – first the typewriter, then the photocopier, and now the Internet – has 

greatly facilitated peer review. Any scientific journal that lays claim to respectability 
APPENDIX 5

128

must have a robust peer review process. At The Lancet, the process goes like this. A 

research paper is submitted electronically to a secure database and allocated by an 

editor to a colleague. The first or second editor can reject the manuscript at that early 

stage if the paper is judged to be scientifically poor, unsuitable for the journal‘s 

readership, unoriginal, or insufficiently topical. Journals differ here. For The Lancet

around three-quarters of manuscripts are rejected at this point. 

If a paper survives preliminary editorial review, it is discussed at a pre-review 

meeting to assess its suitability for external peer review. If judged a potential 

candidate for publication, the manuscript is sent to three expert advisors, commonly 

international and representing different methodological dimensions of the research, as 

well as a statistician. There is always the risk of group-think among experts. That is, 

there is an orthodox belief about a particular subject, strongly held, which resists 

alternative perspectives. Editors try to reduce the risk of group-think by sending 

papers to different and widely dispersed reviewers, deliberately seeking or even 

provoking critical reviews (just like Henry Oldenburg). Reviewers are not referees in 

the sense that they can blow a whistle and call time on the paper. We ask reviewers to 

provide written comments for the authors, confidential comments to the editors, and a 

detailed rating for each section of the paper. Those comments are collected, presented, 

and discussed at a once-weekly manuscript meeting attended by all the journal‘s 

editors. 

At this stage, a paper can be rejected or we can open negotiations with authors. If we 

proceed, reviewers‘ questions and concerns are put to the authors, with appropriate 

guidance from editors. The authors will reply by answering each question from 

reviewers, submitting a revised manuscript that attempts to respond to all points 

raised by editors and reviewers alike. The authors may also disagree with or challenge 

reviewers with varying degrees of force. The revised paper is discussed again at a 

manuscript meeting. The options at this stage are to reject, accept, go back to the 

authors with further requests for clarification, or return to reviewers (old or new) for 

additional opinions. We proceed with further revisions of the paper until a final 

reject/accept decision is made. We know that with such a high rejection rate we may 

get it wrong. To limit errors of omission, we have a formal appeals process where 

editors promise to look again at a paper, weigh up the authors‘ arguments, and 

reconsider our decision. 

Once the paper is provisionally accepted, the peer review process is not over. The 

paper is then passed to a scientifically qualified assistant editor who edits the paper‘s 

technical content. Mistakes may still be found at this stage, leading to further editorial 

or expert review, even (though rarely) rejection. A lesson learned from sometimes 

bitter experience is that a paper is not fully accepted until it is published. 

Here are some of the commonest questions asked about the peer review process (8). 
APPENDIX 5

129

Do reviewers make mistakes in their judgments? Of course, and so do editors. 

Sadly, the scientific literature is littered with retractions of papers that once passed the 

test of peer review. Reviewers and editors are disappointingly human. 

Are reviewers objective in their judgments? Pure objectivity is impossible. For some 

subjects, an editor can predict the judgment of the reviewer based on past experience 

with that reviewer. But this misses the point of what an editor is seeking. It is not 

simply the judgment of reject/accept that an editor wants from a reviewer. That 

decision is the responsibility of the editor and the editor alone. What an editor really 

seeks is a powerful critique of the manuscript – testing each assumption, probing 

every method, questioning all results, and sceptically challenging interpretations and 

conclusions. Armed with that critique, the editors decide – and take full responsibility 

for deciding. 

Are reviewers willing to accept new ideas? Certainly, they are, although they might 

question those ideas to destruction. The vast majority of reviewers take their 

responsibility as advisors very seriously indeed. They themselves are often on the 

receiving end of peer review. Most try to be as open as possible to new findings, 

although we encourage them to ask difficult and awkward questions. There are 

occasional exceptions. For example, the world of complementary and alternative 

medicine (CAM) divides the medical community. Orthodox medicine mostly rejects 

papers about reflexology, iridology, and acupuncture treatment that invokes invisible 

pathways (meridians) of qi. CAM is served by a separate class of journals that have 

little overlap with the more mainstream medical literature. In this instance, ideas are 

incommensurable. 

Despite peer review, are authors able to get away with dishonest or dubious 

research? Yes, they are. Peer review does not replicate and so validate research. Peer 

review does not prove that a piece of research is true. The best it can do is say that, on 

the basis of a written account of what was done and some interrogation of the authors, 

the research seems on the face of it to be acceptable for publication. This claim for 

peer review is much softer than often portrayed to the general public. Experience 

shows, for example, that peer review is an extremely unreliable way to detect research 

misconduct. 

Are peer reviewers accountable for what they do? Yes, to the editor. But in a broader 

sense, to the scientific community and to the public as well. To a large extent, the 

trust society places in science depends on the scientific process, including peer review 

and publication, getting it right most of the time. 

Does peer review improve the quality of published research? In our everyday 

practice, we see that it does. And research suggests that it does too (9). Peer review 

improves discussion of the limitations of research. It emphasises uncertainty. It invites 

justification of generalisability. As one study of peer review concluded, ―peer review 
APPENDIX 5

130

is a negotiation between authors and journal about the scope of the knowledge claims 

that will ultimately appear in print‖ (9).

Is there still a need for peer review, given the extraordinary ability of the Internet to 

enable continuous open criticism of research once published (that is, surely a 

thousand readers as reviewers after publication are better than 4 reviewers selected 

by editors before publication)? There is no right answer to this question. Different 

views have been expressed. Certainly, post-publication peer review adds greatly to the 

understanding of a piece of research. But watching pre-publication peer review in 

action - both at the macro level of external expert review and the micro level of 

technical editing - and seeing the extent to which research papers change (mostly for 

the better) after peer review, I think that pre-publication review still has an important 

part to play in science. At its best, pre-publication peer review clarifies, introduces 

uncertainty, insists on placing new work in the context of the totality of available 

evidence, demands a careful explanation of limitations, and prevents flights of 

fanciful over-interpretation. 

Peer review has changed considerably during the past two decades (since I became an 

editor). First, the stakes are higher. Individual and institutional success depends on 

getting papers published in high-impact journals. Citation data are now a standard 

metric for measuring research performance. This trend has increased competition and 

rivalry for places in the best journals. Second, the globalisation of science has 

expanded the geographic range of papers submitted to journals. Research originating 

from China, for example, is now far more common than even five years ago. The 

internationalisation of science has further intensified competition for publication. 

Third, research papers are increasingly multi-disciplinary, requiring a much broader 

range of expertise during peer review. Fourth, science is a stronger part of our public 

culture now than it once was. What scientists used to write only for other scientists is 

today available to – and sometimes read by – non-scientists, policy makers, and the 

media. Fifth, the importance of statistics has grown substantially. Whereas twenty 

years ago The Lancet had no separate statistical peer review process, every paper we 

now publish has been carefully scrutinised by an independent statistical advisor. 

Editors are now far more aware of analytic errors in research. Sixth, to address the 

often conflicting results of individual research studies that are trying to answer the 

same (or a similar) question, a new type of research method has been devised – the 

systematic, as opposed to the narrative, review. Systematic reviews aim to search for 

particular types of study (eg, the randomised trial), then select only the best according 

to pre-specified criteria, and, if possible, to combine those findings in a statistically 

meaningful way (which is called meta-analysis). Examples include the risk of cervical 

cancer among women taking hormonal contraceptives (10) and the effects of a class 

of medicines on heart disease (11). In biomedicine, the Cochrane Collaboration is the 

most mature example of an effort to create a database of systematic reviews on 

treatments. 
APPENDIX 5

131

Finally, editors have had to face an upsurge in the discovery of episodes of research 

misconduct (fabrication, falsification, and plagiarism). The increasing awareness of 

research fraud had led not only to greater vigilance (hopefully not suspicion) among 

editors but also to the birth of institutional mechanisms to set standards and advise on 

research practice (eg, the Committee on Publication Ethics). 

 

Because of the faith journal editors have in peer review, together with the empirical 

evidence they believe exists to support peer review, they take it very seriously indeed 

(12). That said, editors are well aware that peer review is anything but 

uncontroversial. Scientific discoveries that later turn out to be flagrant episodes of 

dishonesty – from Woo-Suk Hwang‘s fabricated claims in Science about cloning 

embryonic stem cells, to Andrew Wakefield‘s falsifications in The Lancet – are not 

uncommon. They raise troubling questions about the robustness of peer review. 

Editors are only too well aware of the limitations of the peer-review system. Authors, 

for example, can be deeply resistant to responding to questions from anonymous 

critics (this fact at least partly drives the argument for fully transparent peer review, 

where reviewers have to disclose their names to authors). The reluctance of some 

authors – and some very famous authors, at that – to take the comments of their peers 

seriously stems from the fact that they believe they have no peers. As one historian of 

peer review put it, somewhat poetically, ―anyone who possessed the MD degree had 

no reason to defer to any colleague as an expert greater than he or she‖ (13).

So what is peer review in today‘s scientific culture? Various views have been more or 

less vividly expressed. Peer review is a ―sacred academic cow‖, according to one 

editor (14). She put it rather well: 

 “the „sacred‟ cow of peer review wanders the meadows 

of scientific publishing because together, scholars and 

editors, believe that it is the best mechanism we have to 

improve the quality of published papers...The one 

component that we cannot control is that of competition 

in the academic world, an issue that continues to circle 

in all disciplines. We all have stories to tell about the 

viciousness of academic politics...although we might 

not be able to eradicate the politics, we can at least 

understand them for what they are...”

Everyone – scientists, the public, policymakers, politicians – would like to believe 

that peer review is a firewall between truth and error (or dishonesty) (15). But as the 

editor of one leading specialist medical journal has rightly pointed out, ―There is no 

question that, when it comes to peer review, the reviewers themselves are the weakest 

(or strongest) links‖ (16). This frustration among editors and scientists that peer 

review cannot always live up to the claims sometimes made for it produces frequent 
APPENDIX 5

132

expressions of dismay. Is peer review a castle built on sand or the bedrock of 

scientific publishing (17)? Is peer review a landmark, landmine, or landfill (18)? Or, 

put bluntly, is peer review simply in crisis? (19). Is it ―a flawed process at the heart of 

science and journals‖ (20)?

Unfortunately, there is evidence of a lack of evidence for peer review‘s efficacy. In 

2002, Tom Jefferson and colleagues published a startling systematic review of all the 

evidence about editorial peer review in biomedical journals. Their exhaustive search 

yielded only a handful of studies. The conclusion? ―Editorial peer review, although 

widely used, is largely untested and its effects are uncertain‖ (21). They went on, 

―Given the widespread use of peer review and its importance, it is surprising that so 

little is known of its effects.‖ Jefferson and his colleagues have confirmed their 

observations more recently (22). Their findings have been replicated by others (23). 

To be fair, there is some evidence that micro peer review – technical editing – can 

improve papers in biomedical journals (24). But, once again, this evidence is not as 

robust as one would either like or have expected. 

Jefferson extended his investigation of peer review by arguing that the objectives of 

the review process were also unclear (25). Without clear objectives, proving the value 

of peer review (or not) would be impossible. After almost 350 years of journal peer 

review, our zeal for and confidence in the peer review process seem inversely 

proportional to our knowledge about what it actually does for science. Those who 

make big claims for peer review need to face up to this disturbing absence of 

evidence. 

Worse still, what evidence is slowly accumulating should perhaps make scientists, 

policymakers, and the public pause. Many who place great weight on the reliability of 

the peer-reviewed scientific literature believe that it reflects the judgment of the 

scientific community about the quality of research. But evidence suggests that 

acceptance of research for publication may well depend on factors other than 

scientific quality alone (26). Furthermore, peer reviewers will disagree greatly in their 

recommendations to editors about a particular research paper. Yet editors seem to be 

significantly influenced by reviewers who, when the quality of their advice is 

measured independently, turn out to be extremely unreliable in their overall 

judgments (27). Editors, some critics could reasonably argue, need to pay less, not 

more, attention to the recommendations of their peer reviewers. 

Scepticism about peer review is healthy. But every editor knows that peer review can 

be an indispensible aid to his or her work. Peer review can rescue science from

embarrassment and error. An extreme example goes some way to showing why. Peter 

Duesberg is a well-known molecular virologist who believes that HIV is not the cause 

of AIDS. In 2009, the journal Medical Hypotheses published a paper by Duesberg 

arguing that the deaths attributed to AIDS in South Africa were false. The editor of 

Medical Hypotheses operated an editorial policy of no external peer review. The 
APPENDIX 5

133

justification was that peer review might suppress creative thinking. In the case of the 

Duesberg paper, the idea that HIV does not cause AIDS was not new. More 

importantly, South Africa is only now reversing its disastrous denialist policies on 

HIV-AIDS. To consider Duesberg‘s old (and discredited) idea at a critical moment for 

the country he was writing about would, most reasonable editors would conclude, 

require some kind of external peer review to assist decision-making. The editor did 

not seek expert reviews. He accepted the paper within a few days of its submission. 

Many scientists in the AIDS community were appalled. They wrote to the publishers 

(Elsevier, also the publishers of The Lancet) to complain. Elsevier removed the paper 

from its online database pending the results of an independent investigation. The 

Lancet was asked to review the paper. We did so and the reviews were uniformly and 

deeply critical. No journal could have conceivably published the Duesberg paper 

based on these reviews. The Duesberg paper remains retracted, excised from the 

scientific literature. Here is an example of what can happen when peer review is 

excluded from a journal‘s processes, and why peer review can bring important 

information to bear on judgments about the suitability of research for publication. 

Thanks to these events, the journal will now implement peer review. The publishers 

are seeking a new editor (28). 

Climate science and peer review

The events surrounding the peer review of certain climate science papers have raised 

important questions about peer review. But some of these questions may be based on 

a misinformed view of the peer review process. Here, I touch on general themes in 

peer review that have emerged during the debate about the role of climate scientists in 

research publication. 

It is common for editors to have multiple, intense, and sometimes sharp interactions 

with authors and reviewers. Publication matters. Authors and reviewers are frequently 

passionate in their intellectual combat over a piece of research. The tone of their 

exchanges and communications with editors can be attacking, accusatory, aggressive, 

and even personal. If a research paper is especially controversial and word of it is 

circulating in a particular scientific community, third-party scientists or critics with an 

interest in the work may get to hear of it and decide to contact the journal. They might 

wish to warn or encourage editors. This kind of intervention is entirely normal. It is 

the task of editors to weigh up the passionate opinions of authors and reviewers, and 

to reflect on the comments (and motivations) of third parties. To an onlooker, these 

debates may appear as if improper pressure is being exerted on an editor. In fact, this 

is the ordinary to and fro of scientific debate going on behind the public screen of 

science. Occasionally, a line might be crossed. We experienced such a border crossing 

recently, where several reviewers and third parties encouraged us to delay publication 

of a paper for non-scientific reasons (29). Defining that line is the crucial task when 

judging the role of CRU scientists. 
APPENDIX 5

134

One issue that is important to solve for the peer review process to work effectively is 

the full disclosure of all financial and relevant non-financial conflicts of interest. If a 

research paper about drug A for disease Y is sent to a reviewer who has shares in a 

company that makes drug B, also for disease Y, there is a potential for the 

introduction of bias into that reviewer‘s advice to the journal – favouring drug B over 

drug A. The editor may still want and value that reviewer‘s advice, but s/he needs to 

know about the reviewer‘s financial conflict to judge the weight s/he gives to the 

review. Non-financial conflicts may be even more important. If a scientist has devoted 

a life‘s work to theory A about disease Y, then clearly s/he might be biased if s/he is 

sent a manuscript that criticises theory A and proposes an alternative and compelling 

theory B for that same disease. Again, the editor would expect the reviewer to declare 

any non-financial academic or intellectual conflicts that might have the potential to 

influence that reviewer‘s critique. In the field of climate research, conclusions about 

the meaning of the science may have been strongly held. The fact that these 

conclusions were strongly held is entirely to be expected and should simply have been 

fully disclosed to editors during the peer review process. 

It would be wrong for editors not to listen to advice about publication even after 

acceptance of a paper. A paper is only fully accepted when it is published. New 

information that informs the decision to publish a provisionally accepted paper before 

publication can be very valuable. The Lancet has rejected papers in this twilight zone 

of peer review. After publication, criticism is common and welcome, even lethal 

criticism. This is the much vaunted self-regulation of science – except that sometimes 

editors and authors are reluctant to act when things go wrong after publication. In 

climate science, disputatious research is unsurprising. It would be extremely 

surprising if that research did not provoke heated exchanges both before and after 

publication. 

Much has been made of whether scientists should or should not take public positions 

on the meaning of their data, especially if those data relate directly to policy or 

practice. The reality is that they do, all the time. Science does not exist in a political 

vacuum. The idea that scientists, including climate scientists, are neutral observers, 

bereft of opinions, is naïve. In biomedical and public health research, scientists are 

often quick to make statements applying their data to the real world. They will often 

do so passionately and be well known for those passionate views. Indeed, the current 

climate of science is such that scientists are encouraged at every stage of their 

research to consider the impact – economic or human – of what they do, and to 

trumpet that impact. Research assessments in the future are likely to include a 

measure of impact when judging the quality of a scientist‘s work. In relation to peer 

review, the scientific, policy, or political positions an author, reviewer, or editor may 

hold could intervene to bias a review in one particular direction. There have been 

many examples of such conflicts in other scientific disciplines – eg, psychology (30) 

and genetic epidemiology (31). These episodes are troubling, but an almost inevitable 

consequence of the way peer review is ordinarily done. 
APPENDIX 5

135

The intersection of politics and science in climate change has arisen at least once 

before. The Skeptical Environmentalist, written by Bjørn Lomborg and published by 

Cambridge University Press, led to huge pressure on the publishers to withdraw the 

book (32). Although the manuscript was reviewed by four experts who all 

recommended publication, the scientific backlash was acute. Letters of protest were 

written to newspapers. One scientist refused to work with Cambridge University Press 

ever again. Lomborg was attacked physically. 

Chris Harrison, in his thoughtful reflections as the editor at Cambridge University 

Press who dealt with Lomborg‘s book, points out that peer review ―offers no 

guarantees of always ensuring the  ̳truth‘‖ (32). But in the case of The Skeptical 

Environmentalist, the concerns were as much political as scientific. The publication of 

this book by a respected scholarly press might play ―to a particular political agenda 

and can be used and abused by vested corporate and political interests.‖ Harrison 

rejected the idea that he should have applied these kinds of value judgment in the 

editorial process. But he notes, ruefully, 

 “While it may not be the responsibility of an editor to

second-guess how a publication will be received in political 

circles, it is clear that politicians and advocacy groups have 

always maintained a keen interest in what is published and 

that they have sought to influence editorial decisions.”

Harrison defended the scholarly publishing industry‘s commitment to pluralism. He 

wrote, 

“Given the scale of interest in the environment it is 

perhaps peculiarly incumbent on the academy and 

general interest intermediaries [Cambridge University 

Press] to host as full and as open a debate as possible...the 

public and the academy can surely only be better served 

by an opportunity to review and debate a wide range 

of perspectives.”

This commitment to pluralism would be the likely view of many scientific editors, 

even when controversy follows. There are interesting parallels between the response 

of the scientific community to Lomborg‘s sceptical book and the reaction of some 

scientists to journal articles on climate change which expressed an opinion contrary to 

their own. One might conclude that these kinds of extreme debate, although difficult, 

are part of the normal fabric of scientific discourse. The question to be answered is: 

where is the line to be drawn between vigorous scientific exchange and improper 

attempts to close down debate (these two positions can be remarkably close to one 

another)? But one should also be conscious of what some observers have described as 
APPENDIX 5

136

the ―chilling‖ effect of political controversy on science. A survey of US National 

Institutes of Health scientists revealed that many engaged in self-censorship after they 

found themselves the subject of political criticism for their work (33). Political 

disagreement over science can shape not only the behaviour of scientists but also the 

future of science itself. As Joanna Kempner has noted, 

 “There is a role for democratic public engagement in 

science. The policy challenge will be to encourage this 

public voice in scientific decision-making, while enabling 

scientists to submit and conduct innovative studies, even 

when they may provoke controversy.” 

Increasingly, commercial, as well as political, interests are also intervening to threaten 

the integrity of peer review (34). 

Peer review and publication can provoke important questions about access to data. 

During the review process, reviewers may seek more information. Except in 

allegations of fraud, it would be highly unusual to provide or request raw data (even 

then, journals expect institutions to take responsibility for investigating the 

authenticity and reliability of original data). But access to data may be sought after 

publication. This is a highly contentious and unresolved issue. For example, Andrew 

Montford argues that, 

 “the more important story in terms of the conduct of 

science in this country concerns the repeated refusals of 

CRU staff to release the data and code underlying their 

global-temperature index...sceptics are universally of 

the opinion that the scientific method requires all research 

material to be released to friend and foe alike...” (2)

In the field of medicine, these issues are currently the subject of much disagreement. 

While many parties might like to see greater sharing of data, this practice remains 

unusual. The Wellcome Trust is taking an especially strong interest in data access. It 

proposes a code of conduct calling for ―maximum public access to data of public 

health importance.‖ The very fact that this proposal is being made illustrates the point 

that routine access to data is not a settled issue or a universal norm in science, as some 

claim. 

The issue of retention of records and exclusion of data in climate research is also a 

matter relevant to the peer review process and the ordinary working of journals. 

Journals do expect records to be kept for limited periods (say, 5 years, although 

journal practices vary). And they are comfortable with the exclusion of data provided 

that those exclusions – and the reasons for exclusion – are fully described, with 

appropriate sensitivity analyses being completed where necessary. 
APPENDIX 5

137

Two additional dimensions of peer review have been raised in the debate over CRU‘s 

work. One relates to confidentiality, the other to uncertainty. Editors send manuscripts 

to reviewers based on a principle of confidentiality. The author expects the editor to 

maintain a covenant of trust between the two parties. The editor will not misuse the 

author‘s work by circulating it outside of the confidential peer review process. The 

editor expects that covenant of trust to be honoured by the peer reviewer. No 

manuscript should be passed to a third party by a reviewer without the permission of 

the editor, usually on the grounds of improving the quality of the critique of the 

manuscript by involving a colleague in the review process. A disclosure to a third 

party without the prior permission of the editor would be a serious violation of the 

peer review process – a breach of confidentiality. It is also of paramount importance 

to report fully in all published scientific papers both quantitative and qualitative 

measures of uncertainty. One of the main benefits of peer review is to focus on areas 

of potential uncertainty and to ensure that those uncertainties are fully acknowledged, 

measured, and reported. 

The future of peer review

Peer review is a human process and so will always contain flaws, produce errors, and 

occasionally mislead. Given that journals are the gatekeepers of scientific publication, 

they have enormous – probably too much – influence over the reputations of 

scientists, research units, and universities. Many measures of academic success 

depend upon journal publication – promotion, tenure, grants, fame, and personal 

wealth. It is not surprising that journals, and the main decision aid used by journals 

(peer review), are the subject of constant tension and occasionally explosive 

controversy. At such moments, it is not only essential to be clear (and modest) about 

what peer review can do, but also to look for opportunities to do better. 

Journal articles are highly stylised reports of research. The linear and logical style of 

the research they report rarely presents a true or accurate picture of how a piece of 

research was done. As the Nobel laureate, Peter Medawar, put it (35) in his essay ―Is 

the scientific paper a fraud?‖ (to which he answered that it was),

 “[the scientific paper] misrepresents the processes

 of thought that accompanied or give rise to the work 

that is described in the paper...The scientific paper in 

its orthodox form does embody a totally mistaken 

conception, even a travesty, of the nature of scientific 

thought.”

Medawar‘s point was that, ―There is no such thing as unprejudiced observations.‖ To 

add insult to injury, research papers may not even fully represent the views of the 
APPENDIX 5

138

authors who completed the work (36), and when faults are found after publication 

those faults may be completely ignored in the subsequent use of that research (37). 

There are actions that the scientific community could take to improve this far from 

happy state of affairs surrounding one its foundational processes. First, there are new 

opportunities and techniques available to search out, identify, and eliminate (or at 

least reduce) unwanted bias in the peer review process (38, 39). Second, all young 

scientists should receive formal training – which they currently do not – in the 

standards and ethics expected in the peer review process (40). It is scandalous that 

peer review is simply not taken as seriously as it should be in the training of scientists. 

The result is that peer review is often idiosyncratic and sometimes unreliable, fuelling 

scientific controversies, such as that over climate science, rather than defusing those 

controversies. Strengthening the training, standards, and expectations around peer 

review would do much to make the quality of peer reviewing part of the formal 

appraisal of a scientist‘s contribution to his or her subject. There is a demand for 

training in peer review (41). And the ethical dimensions of the review process are 

now sufficiently concerning to scientists that they merit training as much as the more 

formal methodological aspects of reviewing (42). Disappointingly, existing training 

packages in peer review deliver little benefit to the quality of the peer review process 

(43-45). 

Third, the peer review process is enormously inefficient. Individual journals will 

undertake peer review and reject manuscripts that will then cycle around other 

journals until either the paper is accepted or the authors are sufficiently exhausted that 

they abandon attempts at publication. In the face of such gross inefficiencies, some 

scientific communities have tried to bring journals together to cooperate and make the 

review process not only more efficient, but also less costly on the time and energy of 

reviewers, authors, and editors (46). Alternatively, there may be intra-journal 

procedures that can be introduced to deliver more efficient peer review (47). 

Fourth, journal editors should adopt more effective methods to resolve disputes 

between authors, reviewers, and readers. Within the journal, an ombudsperson 

operating independently of the editors can be one useful way to resolve intractable 

disagreements about journal processes (48). If a dispute remains impossible to 

resolve, journal editors can take their concerns to the Committee on Publication 

Ethics, a charity that aims to set standards for journal practices, including peer review. 

Journal editors should consider using this facility more often than they currently do –

in some ways, it represents the collective wisdom of a wide range of journal editors, a 

collective wisdom that any scientific editor can draw upon in times of crisis. 

Lastly, peer review should be a subject for research in its own right. Although there is 

a small group of scientists who study peer review (a biomedical peer review congress 

is held every 4 years), that community is extraordinarily fragile when measured 

against the size and importance of the contribution peer review makes to science (49). 
APPENDIX 5

139

Historically, science funding bodies have been reluctant to invest in research on peer 

review. This reluctance is partly responsible for the present vacuum in our knowledge 

about the way scientific knowledge is constructed, reported, and discussed. One 

positive result of the debate over the role of CRU scientists in peer review might be to 

encourage funding bodies – such as the Medical Research Council and the National 

Institute for Health Research – to take the science of peer review far more seriously. 

Journals have inevitable limitations. When a paper with important policy implications 

is considered, editors can ask authors to balance their conclusions by putting the work 

in the context of existing evidence. Or we can commission an editorial that does the 

same. But a journal cannot adjudicate a public debate, and neither can conventional 

peer review. For those occasions when science meets (or clashes with) policy, there 

may be a case for referring the area of controversy to an independent body for a 

public inquiry. In the US, the model used is the Institute of Medicine, which tackles 

controversial aspects of health through its thorough and wide-ranging investigations 

and reports. There may be a case for such a body in the UK. 

The best one might hope for the future of peer review is to be able to foster an 

environment of continuous critique of research papers before and after publication. 

Many writers on peer review have made such a proposal, yet no journal has been able 

to create the motivation or incentives among scientists to engage in permanent peer 

review (50-52). Some observers might worry that extending opportunities for 

criticism will only sustain maverick points-of-view. However, experience suggests 

that the best science would survive such intensified peer review, while the worst 

would find its deserved place at the margins of knowledge. 

This process of weeding out weak research from the scientific literature can be 

accelerated through more formal mechanisms, such as the systematic review. A 

systematic approach to selecting evidence focuses on the quality of scientific methods 

rather than the reputations of scientists and their institutions. This more rigorous 

approach to gathering, appraising, and summing up the totality of available evidence 

has been profoundly valuable to clinical medicine. There may be useful lessons here 

for the IPCC. Climate sceptics and climate scientists, along with their colleagues in 

other scientific disciplines, would likely welcome this greater rigour and scrutiny. It 

would certainly promote quality and strengthen accountability to a more critical 

public (and media) with higher expectations of science. More importantly, intensified 

post as well as pre publication review would put uncertainty – its extent and 

boundaries – at the centre of the peer review and publication process. This new 

emphasis on uncertainty would limit the rhetorical power of the scientific paper (53), 

and offer an opportunity to make continuous but constructive public criticism of 

research a new norm of science. 

Finally, there are more general institutional lessons to be learned from the events that 

took place at the CRU. Peer review cannot eliminate controversy. Leaders of research 
APPENDIX 5

140

organisations – from University Vice-Chancellors to Unit Directors – might consider 

a risk assessment of the research being done under their stewardship. Mapping 

research in this way – who is doing what, where, and how? – would provide the 

institution with signals of potential controversy if the work addressed an issue of 

substantive public, policy, or political interest. This kind of awareness might enable 

scientists – and, indeed, the institution itself – to prepare better for any likely public 

disputes. Vigilance over the research being undertaken at a university could usefully 

anticipate difficulties, promote best practices, protect reputations, and build wider 

public trust and confidence in the research process. These outcomes, together with 

more realistic expectations about peer review and the processes of science, would be 

valuable consequences of the experiences at the University of East Anglia. 

1. Montford A. The Hockey Stick Illusion: Climategate and the corruption of 

science (Stacey International, 2010) 

2 . Montford A. Heated discussions. Times Higher Education March 25, 2010: 

 43-44.

3. Monbiot G. Our narrow, antiquated school system is at the root of the climate 

 email fiasco. The Guardian April 6, 2010: 25. 

4. Booker C. Climategate: a scandal that won‘t go away. The Sunday Telegraph 

 April 18, 2010: 29. 

5. House of Commons Science and Technology Committee. The disclosure of 

 climate data from the Climate Research Unit at the University of East Anglia 

 (Stationery Office, 2010). 

6. This quotation is taken from an email from Phil Jones to Michael Mann. 

7. Spier R. The history of the peer review process. Trends in Biotechnology

 2002; 20: 357-58. 

8. Hernon P, Schwartz C. Peer review revisited. Library and Information Science

Research 2006; 28: 1-3. 

9. Goodman SN, Berlin J, Fletcher SW, Fletcher RH. Manuscript quality before 

and after peer review and editing at Annals of Internal Medicine. Ann Intern 

Med 1994; 121: 11-21. 

10. Smith JS, Green J, de Gonzalez AB, et al. Cervical cancer and use of 

hormonal contraceptives: a systematic review. Lancet 2003; 361: 1159-67. 

11. Jun M, Foote C, Lv J, et al. Effects of fibrates on cardiovascular outcomes: a 

systematic review and meta-analysis. Lancet 2010 

12. Green SM, Callaham ML. Current status of peer review at Annals of 

Emergency Medicine. Ann Emerg Med 2006; 48: 304-08.
APPENDIX 5

141

13. Burnham JC. The evolution of editorial peer review. JAMA 1990; 263: 1323-

9. 

14. Fitzpatrick JJ. Peer review: a 2008 report on the sacred academic cow. 

 Applied Nursing Research 2008; 21: 53. 

15. Harms M. Peer review: the firewall of science. Physiotherapy 2006; 92: 193-

94. 

16. DeMaria AN. Peer review: the weakest link. JACC 2010; 55: 1161-62. 

17. Berger E. Peer review: a castle built on sand or the bedrock of scientific 

 publishing? Ann Emerg Med 2006; 47: 157-59. 

18. Balistreri WF. Landmark, landmine, or landfill? The role of peer review in 

assessing manuscripts. J Pediatr 2007; 151: 107-08.

19. Mulligan A. Is peer review in crisis? Oral Oncology 2005; 41: 135-41. 

20. Smith R. Peer review: a flawed process at the heart of science and journals. J 

R Soc Med 2006; 99: 178-82.

21. Jefferson T, Alderson P, Wager E, Davidoff F. The effects of editorial peer 

 review. JAMA 2002; 287: 2784-86.

22. Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for 

improving the quality of reports of biomedical studies. Cochrane Database 

Syst Rev 2007; 2: MR16. 

23. Richards D. Little evidence to support the use of editorial peer review to 

ensure quality of published research. Evid Based Dent 2007; 8: 88-89. 

24. Wager E, Middleton P. Technical editing of research reports in biomedical 

journals. Cochrane Database Syst Rev 2008; 4: MR2. 

25. Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer 

 review. JAMA 2002; 287: 2786-90.

26. Aarssen LW, Lortie CJ, Budden AE, et al. Does publication in top-tier 

journals affect reviewer behaviour? PLoS ONE 2009; 4: e6283. 

27. Kravitz RL, Franks P, Feldman MD, et al. Editorial peer reviewers‘ 

recommendations at a general medical journal: are they reliable and do editors 

care? PLoS ONE 2010; 5: e10072. 

28. Enserink M. Elsevier to Editor: change controversial journal or resign. Science

2010; 327: 1316. 

29. Horton R. Maternal mortality: surprise, hope, and urgent action. Lancet 2010; 
APPENDIX 5

142

30. McCarty R. Science, politics, and peer review: an editor‘s dilemma. American 

 Psychologist 2002; 57: 198-201.

31. Calnan M, Smith GD, Sterne JA. The publication process itself was the major 

cause of publication bias in genetic epidemiology. J Clin Epidemiol 2006; 59:

1312-18.

32. Harrison C. Peer review, politics, and pluralism. Environmental Science and 

Policy 2004; 7: 357-68. 

33. Kempner J. The chilling effect: how do researchers react to controversy. PLoS 

Medicine 2008; 5: e222. 

34. Curfman GD, Morrissey S, Annas GJ, Drazen JM. Peer review in the balance. 

N Engl J Med 2008; 358: 2276-77.

35. Medawar PB. The Strange Case of the Spotted Mice (Oxford, 1996). 

36. Horton R. The hidden research paper. JAMA 2002; 287: 2775-78.

37. Horton R. Postpublication criticism and the shaping of knowledge. JAMA 

2002; 287: 2843-47. 

38. Bornmann L, Mutz R, Daniel H-D. How to detect indications of potential 

sources of bias in peer review. J Informatics 2008; 2: 280-87.

39. Ross JS, Gross CP, Desai MM, et al. Effect of blinded peer review on abstract 

acceptance. JAMA 2006; 295: 1675-80.

40. Walbot V. Are we training pit bulls to review our manuscripts? J Biol 2009; 8: 

24.

41. Snell L, Spencer J. Reviewers‘ perceptions of the peer review process for a 

medical education journal. Med Educ 2005; 39: 90-97.

42. Resnik DB, Gutierrez-Ford C, Peddada S. Perceptions of ethical problems 

with scientific journal peer review. Sci Eng Ethics 2008; 14: 305-10. 

43. Schroter S, Black N, Evans S, et al. Effects of training on quality of peer 

review. BMJ 2004 doi: 10.11136/bmj.38023.700775.AE. 

44. Callaham ML, Tercier J. The relationship of previous training and experience 

of journal peer reviewers to subsequent review quality. PLoS Medicine 2007; 

4: e40. 

45. Schroter S, Black N, Evans S, et al. What errors do peer reviewers detect, and 

does training improve their ability to detect them? J R Soc Med 2008: 101: 

507-14.
APPENDIX 5

143

46. Saper CB, Maunsell JHR. The Neuroscience Peer Review Consortium. Brain

Res 2009; 1272: 1-2.

47. Johnston SC, Lowenstein DH, Ferriero DM, et al. Early editorial manuscript 

screening versus obligate peer review. Ann Neurol 2007; 61: A10-12. 

48. Horton R. The journal ombudsperson: a step toward scientific press oversight. 

JAMA 1998; 280: 298-99. 

49. Linkov F, Lovalekar M, LaPorte R. Scientific journals are ―faith based‖: is 

there science behind peer review? J R Soc Med 2006; 99: 596-98.

50. von Segesser LK. Peer review versus public review – new possibilities of online publishing! Interactive Cardiovascular and Thoracic Surgery 2002; 1:

61-62.

51. Mandviwalla M, Patnayakuni R, Schuff D. Improving the peer review process 

with information technology. Decision Support Systems 2008; 46: 29-40. 

52. Liesegang TJ. Peer review should continue after publication. Am J Ophthalmol

2010; March; 149: 359-60. 

53. Horton R. The rhetoric of research. BMJ 1995; 310: 985-88.

Commentary on „Understanding uncertainty:a brief history of 

peer review’

Elizabeth Wager, Publications Consultant 

Chair of the Committee on Publication Ethics (COPE) 

Richard Horton suggests that  ̳If evidence has not been peer-reviewed, it is next to 

worthless‘. Earlier commentators have gone even further. The evolutionary biologist 

Edward Wilson commented that  ̳a discovery does not exist until it is safely reviewed 

and in print‘.1 

Peer review has been used to select articles for publication since the 17th century. It is 

likely that when Oldenburg was using peer review at London‘s Royal Society, his 

contemporaries at the Royal Society in Edinburgh and across the channel at the 

Académie Royale de Médicine were using similar systems.2 

Interestingly, the inability of peer review to guarantee scientific integrity was noted as 

early as 1785 when the Committee of Papers of the Literary and Philosophical Society 

of Manchester noted that  ̳Responsibility concerning the truth of facts, the soundness 

of reasoning ... [and] the accuracy of calculations is wholly disclaimed; and must rest 

alone, on the knowledge, judgement, or ability of the authors‘.3 

Horton describes the peer review system used at The Lancet. This is typical of major 

medical journals that employ several full-time editors. Such journals reject a 
APPENDIX 5

144

considerable proportion of submissions on the basis of in-house review alone. For 

example, at JAMA (the journal of the American Medical Association) only 36% of 

submissions are sent for external review and the remaining 64% are rejected after 

review by the journal editors alone.4 At journals such as The Lancet and JAMA, 

everything accepted for publication will have been reviewed by experts who are 

independent of the journal, but many articles will be rejected without review by 

external experts. In contrast, journals that do not have large editorial offices or fulltime editors, but whose editors are academics (generally fulfilling this role in addition 

to their regular job), generally send virtually all submissions for external review. 

The Lancet generally sends papers to three reviewers. Other journals may use fewer, 

or occasionally more, reviewers. A survey of 200 journals from a range of disciplines 

found that 73% used two reviewers, 18% used three, 6% used one and 3% used more 

than three.5 

Horton notes that  ̳the scientific literature is littered with retractions of papers that 

once passed the test of peer review‘. The biomedical database Medline (which 

includes over 19 million citations) currently contains nearly 1500 retractions. There 

have also been well-documented cases of journals failing to recognise important 

work. There is even a website devoted to accounts of journals that have rejected work 

that later led to their authors winning the Nobel prize. 

[ttp://www2.uah.es/jmc/nobel/nobel.html#reje]. 

Churchill‘s famous observation about democracy has often been applied to peer 

review, namely that it is the worst system for selecting papers except for all the other 

forms that have been tried. Horton mentions the possibility of alternative systems 

such as post-publication or public review. A few journals have experimented with 

these systems but found them unworkable. When Nature ran a trial of open peer 

review in 2006 (by posting submitted papers on its website), almost half the papers 

received no comments.6 

While formal evidence for the effectiveness of peer review is lacking, there is 

extensive anecdotal evidence from both editors and authors of its benefits. Editors and 

publishers should strive to optimise their peer review systems and to reduce potential 

harm or bias. Guidance is available from several organizations including the 

Committee on Publication Ethics (www.publicationethics.org). 

While the internet has not produced dramatic new ways of assessing articles 

submitted to journals, information technology does offer possibilities which could 

increase the efficiency of disseminating and verifying scientific data. Gene sequence 

data is now routinely entered onto public databases, and such data posting is a 

requirement for publication in traditional journals in some cases. However, before 

such posting could be practical, even for such homogeneous data as gene sequences, 

agreements had to be developed (e.g. MIAME).7

Such standardization may be 

possible for other types of data yet, as Horton notes, in many disciplines, data sharing 

remains either controversial, or limited by technological, practical or ethical issues.8

Even with increased research and investment it seems unlikely that, despite its 

shortcomings, conventional peer review will be replaced or transformed in the next 

few years. In the meantime, researchers will probably continue to criticise the peer 
APPENDIX 5

145

review system, sometimes with justification (because the system is flawed) and 

sometimes without (because researchers are human, feel strongly about their work and 

do not accept that some rejections are justified). The research community should 

listen to the criticisms of peer review and work towards fixing the remediable faults 

and correcting the inevitable errors. Editors and publishers should continue to work 

hard to prevent and detect misconduct by authors, reviewers and editors and should 

have systems in place for responding appropriately when misconduct occurs. 

However, judgement must be exercised to ensure that resources are wisely spent and 

care should be taken to avoid overzealous responses to individual cases or 

disproportionate reactions to rare problems. 

References 

1. Wilson EO. Consilience: The Unity of Knowledge. Knopf, New York, 1998 

2. Wager E. Peer review in science journals: past, present and future. In Holliman R et 

al (eds) Practising Science Communication in the Information Age. Oxford University 

Press, Oxford, 2009 

3. Kronick DA. Peer review in 18th-century scientific journalism. Journal of the 

American Medical Association 1990;263:1321-2 

4. Fontanarosa P, DeAngelis C. To JAMA peer reviewers and authors – thank you. 

Journal of the American Medical Association 2010;303:780 

5. Current Practice in Peer Review. ALPSP, EASE, ALSSS. www.alpsp.org 

6. Anon Overview: Nature‘s peer revew trial, 2006 

www.nature.com/nature/peerreview/debate/nature05535.html 

7. Brazma A. Minimum information about a microarray experiment (MIAME) --

toward standards for microarray data. Nature Genetics 2001;29:365-71

8. Wager E. Publishing clinical trial results: the future beckons. 

PLoS Clinical Trials 2006;1:e31 

Competing interests statement 

Elizabeth Wager is a freelance publications consultant who provides consulting, 

training, and editorial services mainly to pharmaceutical companies, publishers and 

academic institutions. She chairs the Committee on Publication Ethics (but has written 

this commentary in her individual capacity). She was part of the team that produced 

systematic reviews on the effects of peer review and technical editing cited by 

Richard Horton. She received a fee for producing this commentary. 
APPENDIX 6

146

APPENDIX 6: DATA MINING - ACCESS TO THE 

ORIGINAL CRU E-MAIL ARCHIVE

1. Recognising that the e-mails improperly released into the public domain represent 

only a tiny fraction (less than 0.3%) of the e-mails archived by the key individuals 

in the CRU, the Review team sought to set these in context. The backup server 

(CRUBACK3) had been taken as evidence by the police as part of their own 

investigation and was held by police contracted forensic investigators. A full 

context could only be established by some form of access to the information held 

on this server. In seeking to gain this access a number of legal issues arose, 

notably that: 

the server and its contents were evidence in the continuing police 

investigation; and 

in the opinion of UEA‘s legal advisers, unconstrained access to the contents of 

e-mails on the server by the Review would raise potential privacy and data 

protection issues. 

2. The compromise eventually reached with both the police investigative team and 

the UEA Registrar was for: 

the University to contract an independent computer forensic analyst; 

the police forensic consultants to extract from CRUBACK3 all the e-mails 

from the various archived mailboxes of key UEA staff and to provide these 

under strict security conditions to the independent forensic analyst; 

the independent analyst, respecting the high evidential security requirements 

set out by the police team, to work within secure premises authorized by the 

police; 

the independent analyst to seek to determine the search or selection parameters 

that had extracted the improperly released e-mails from the multiple archives 

and to determine whether this process highlighted any additional material of 

relevance to the Review; and 

any material identified by the the analyst to be redacted by the University, in 

terms of protecting the identity of non-UEA recipients or authors, prior to 

being made available to the Review. 

3. This whole process took an extended period to negotiate and implement. It 

became clear that a full analysis would require considerable further time and 

extensive manual intervention. It would introduce significant delay to the 

publication of the Review‘s report. A decision was reached not to pursue this 

further on grounds of both time and cost against likely results. The Review had 

always regarded the e-mails as pointers to areas for detailed investigation and this 

had been complemented by extensive public requests for submissions and any 

other information in the public domain. A summary report by the independent 

forensic analyst has been placed on the Review website. 
APPENDIX 6

147

4. The Review‘s own analysis of those e-mails already in the public domain is 

highlighted in the following charts, including a  ̳heat map‘ of correspondence 

intensity by month. 

5. The selected e-mails relate largely to controversial issues, although they do not 

appear to have been selected on the basis of a simple word search, as indicated in 

the table below which shows the number of e-mails in which interesting words (or 

a component thereof) occur. Some of the more infamous e-mails contain words 

such as trick or hide and these occur comparatively infrequently:- 

1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

Jan 2 5 2 1 1 1 3 9 32 9 16 26 10

Feb 5 5 3 7 1 2 9 11 48 4 5 4

Mar 1 2 4 4 7 5 11 5 5 21 6 3 8

Apr 1 1 10 2 4 8 6 11 4 9 5 3

May 1 1 12 2 4 1 6 4 14 9 6 24 7

Jun 2 2 2 6 1 5 1 19 6 10 4 6 10 4

Jul 2 2 6 4 6 11 4 30 29 3 5 18

Aug 4 2 1 3 8 4 2 8 7 16 12 4 5 8

Sep 3 6 13 14 6 2 7 4 3 11 4 4 22

Oct 4 4 12 8 8 3 20 8 6 1 10 35

Nov 3 4 2 4 3 1 7 3 3 3 4 6 8

Dec 3 1 5 3 2 2 14 9 16 9

Totals 22 19 46 69 53 47 33 96 73 144 157 75 112 127

Key Word Component Number Key Word Component Number

Briffa 580 error 178

Jones 563 wrong 173

Mann 489 Santer 173

issue 470 fund 150

differ 460 difficult 136

temperature 453 bad 134

good 450 bias 127

review 376 (sk)eptic 124

agree 368 contain 112

IPCC 357 station 106

problem 354 doubt 105

Osborn 347 anomal(y) 101

correct 308 Hulme 101

tree 307 peer 100

miss(ing) 305 avoid 100

author 290 Yamal 100

understand 251 reject 75

concern 248 trick 65

Jansen 212 hockey 60

proxy 207 decline 49

Overpeck 196 hid(e) 48

critic 188 delete 44

argu(ment) 188 mislead 37

Wigley 181 diverge 32
APPENDIX 6

148

6. The periods of greatest intensity generally coincide with repeated selection 

of embedded e-mails having the same core title – with 4 or more core title 

repetitions being viewed as significant in the table below. 

Date Core Title Repeats Subject

Oct-98 climate of the last millennia... 4 Meeting to discuss multi-proxy data 

prompted by Jones 1998 1 

May-99 Straight to the Point 4 Perceived criticism of Briffa 19992

Sep-99 IPCC revisions 7 Making the case for tree ring proxies in 

IPCC TAR Chapter 2

Apr-02 Your letter to Science 5 Criticism of draft letter Mann 20023

Mar-03 Soon & Baliunas 9 Criticism of Soon 20034

Jun-03 Review- confidential 4 Validity of proxy-based temperature 

reconstruction

Oct-03 draft 4 of proposed EOS response to Soon 20034

Feb-04 Stephen McIntyre 4 Requesting Russian tree ring data

Jan-05 [Wg1-ar4-ch06] IPCC last 2000 years 

data

5 Drafting of IPCC chapter

Apr-05 WG1 LA2 meeting - Overlap cluster 

A

5 Coordination between IPCC Lead Authors

Jul-05 MWP figure 5 for IPCC report

Dec-05 HadCRUT2v 4 Missing Southern Pole data

Feb-06 some figures at last! 4 Proxy reconstruction for IPCC AR4

Mar-06 latest draft of 2000-year section text 7 IPCC Chapter 6 in the SOD for the AR4

May-06 Wahl & Ammann paper 4 Use of Wahl 20065

in the SOD for the AR4

Jul-06 Special instructions/timing adjustment 4 Proxy reconstruction for IPCC

Sep-06 No Subject 4 Significance of recent temperatures in IPCC 

report

Jan-07 not so fast 4 J. Eddy figure in IPCC report

Apr-07 FYI 4 Climate Audit attacks on Wang and Jones

Dec-07 sorry to take your time up, but really 

do need a scrub of this 

singer/christy/etc effort

11 Response to Douglas 2007 6

Jan-08 Update on response to Douglass et al. 5 Publication options to rebut Douglas 20076

May-08 EA 21389 - Probabilistic information 

to inform EA decision making on 

climate change impacts - PCC(08)01

4 Tender pack for Environment Agency bid

May-08 JOC-08-0098 - International Journal 

of Climatology

4 Review of Santer 20087

May-08 Our d3* test 4 Validation of climate models

Jan-09 Temperatures in 2009 5 Prognosis for 2009 and UHI effect

Jun-09 2009JD012960 (Editor - Steve 

Ghan):Decision Letter

4 Review of McLean 2009 8

Jul-09 ENSO blamed over warming - paper 

in JGR

14 Response to McLean 2009 8

Sep-09 attacks against Keith 5 Selection of tree-ring data

Oct-09 BBC U-turn on climate 10 Recent 'lack' of warming
APPENDIX 6

149

References 

1. P.D. Jones, K.R. Briffa, T.P. Barnett and S.F.B. Tett, ―High-resolution 

palaeoclimatic records for the last millennium: interpretation, integration and 

comparison with General Circulation Model control-run temperatures‖, The 

Holocene 8, 4 (1998), pp. 455–471

2. KR. Briffa and T.J. Osborn, ―Seeing the wood for the trees‖, Science Vol. 284, 7 

May 1999, pp. 926-927 

3. M.E. Mann and M.K. Hughes, ―Tree-ring chronologies and climate variability‖, 

Science, Volume 296, Issue 5569, 3 May 2002, pp. 848-849, 

4. W. Soon, S. Baliunas, ―Proxy climatic and environmental changes of the past 

1000 years‖, Climate Research Volume 23: 89-110, January 2003 

5. E.R. Wahl, D.M. Ritson, C.M. Ammann, ―Reconstructing Past Climate from 

Noisy Data‖, Science 312, 529 (2006) 

6. D.H. Douglass, J.R. Christy, B.D. Pearson and S.F. Singer, ―A comparison of 

tropical temperature trends with model predictions‖, International Journal of 

Climatology, (2007) 

7. B.D. Santer, P.W. Thorne, L. Haimberger, K.E. Taylor, T.M.L. Wigley, J.R. 

Lanzante, S. Solomon, M. Free, P.J. Gleckler, P.D. Jones, T.R. Karl, S.A. Klein, 

C. Mears, D. Nychka, G.A. Schmidt, S.C. Sherwood, and F.J. Wentz, 

―Consistency of modelled and observed temperature trends in the tropical 

troposphere‖, International Journal of Climatology, 28: 1703–1722 (2008) 

8. J.D. McLean, C.R. de Freitas, and R.M. Carter, ―Influence of the Southern 

Oscillation on tropospheric temperature‖, Journal of Geophysical Research, Vol 

14, D14104 (2009) 
APPENDIX 7

150

APPENDIX 7: LAND STATION TEMPERATURE DATA 

1. Contents 

This Appendix supports Chapter 6 of the report addressing land temperature data. It 

contains further information on two issues: 

the Trial Analysis, referred to in Sections 6.3 and 6.4 of Chapter 6, and 

the particular issue of matching station identifiers, referred to in Section 6.5.1 

of Chapter 6. 

2. The Trial Analysis 

2.1 Introduction 

The Review has conducted a trial analysis to demonstrate what an independent 

researcher is able to do, using publicly available land station temperature information, 

should they wish to replicate the CRUTEM analysis. 

We have only carried out each step as far as necessary to demonstrate proof of 

principle and to compare results at a level sufficient to inform our findings in respect 

of allegations made against CRU. It is important to note that we have not carried out 

all of the steps with the necessary degree of scientific rigour needed to form scientific 

conclusions, and we emphasise that no such conclusions should be drawn from this 

work. 

We have performed the following steps: 

1. Obtained raw station temperature data from two repositories as ASCII files. 

2. Written computer code in C++ to read this data into memory in a suitable 

format. 

3. Written computer code to process this data and create gridded global average 

temperature trends. 

4. Compared the results to those of CRU and others in order to inform our 

findings in respect of the allegations made against CRU. 

We have not carried out the full process of homogenising all data. There is no 

fundamental barrier to doing so. It is a time consuming step and requires the 

experience and expertise of a climate research scientist to process each of the raw 

station data entries before averaging. We comment upon the significance of 

homogenisation below. 

2.2 Primary Station Data 

Primary station data was obtained from: 

NCAR:http://www.ncar.ucar.edu/tools/datasets/ 

WWR: http://dss.ucar.edu/datasets/ds570.0/ 

GHCN: http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php. 
APPENDIX 7

151

The GHCN data was freely available. GHCN provided several files including a list of 

stations and locations, a data set of unadjusted data, and a dataset of adjusted data. 

This contained 7280 stations in the unadjusted file and 4771 in the adjusted file. This 

data set contained in some cases several duplicates from the same station. 

For NCAR it was necessary to register (free) first. The NCAR data was a single file 

containing all necessary information and contained 4646 stations. 

The formats of each file were easy to understand. The formats were different from 

each other. 

2.3 Code to Process the Data 

Code was written in C++ to: 

Parse the source files and assemble the information into computer memory. 

Pass over each station and calculate the monthly ―normals‖ defined as the 

average over a set period. In this case we required good data from at least 10 

years in the period 1955-1995. This is larger than was used by CRUTEM in 

order to obtain more valid normals without having to resort to external 

records. 

In the case of GHCN, choose the duplicate with the most valid monthly 

normals. 

Calculate the anomaly for each monthly measurement for each station. The 

anomaly is defined as the monthly temperature minus the normal for that 

month. 

Either choose to use all stations, or only those matched to the CRUTEM3 

station list published in 2009. 

Pass over the data set and assemble average annual anomalies in each cell of a 

5x5 degree grid of latitude and longitude. 

Calculate the overall annual average anomaly by averaging over grid cells. 

Do this for each of GHCN (unadjusted), GHCN (adjusted) and NCAR, and 

plot the annual average anomaly as a function of time. 

Separately read the published CRUTEM3 product and form the same average 

over grid cells and plot this on the same figure for comparison. 

This code was straightforward to write from scratch. The minimal code needed to read 

and process the data sets amounts to only a few hundred executable lines and took 

about 2 days to develop. 

2.4 Results of Global Temperature Trends 

Figure A1 - A4 show the comparison of the time series obtained from the three data 

sets GHCN (unadjusted), GHCN (adjusted) and NCAR. A simple 5 year smoothing is 

applied. Also shown is the result obtained from the CRUTEM3 5x5 gridded product. 
APPENDIX 7

152

The set of input stations used for each line is not identical - the numbers are GHCN 

(unadjusted) [7280], GHCN (adjusted)[4771], NCAR [4646]. Of these, all stations 

which have a valid monthly normal and valid monthly data are included in Fig A1. 

In Fig A2 we have approximately matched the stations in each dataset to those used in 

CRUTEM3. This is described in more detail below. The numbers matched are GHCN 

(unadjusted) [3722], GHCN (adjusted) [2962], NCAR [1957]. CRUTEM itself used 

4138 stations. 

Figure A3 is the same as A2 but showing only the two lines which correspond to 

unadjusted data (GHCN and NCAR). 

Figure A4 is the same as A2 but showing only the two lines which correspond to 

adjusted data (GHCN (adjusted) and CRUTEM). 

Figure A5 reproduces a comparison of different studies taken from the IPCC 4th

Report Chapter 3. 

Figure A1: Temperature anomaly time series created by the Inquiry Team‘s own trial analysis using a 

5x5 grid with 5 year smoothing. Shown are results obtained from GHCN (blue), GHCN-adjusted 

(yellow) and NCAR (green). Also shown is the CRUTEM3 line (black). The Y-axis is degrees x 10. 

The X-axis is year. 
APPENDIX 7

153

Figure A2: Same as Fig A1 but with approximate match of stations to those used in CRUTEM3. Axes 

same as Fig A1. 

Figure A3: Same as Fig A2 but showing only the unadjusted lines (GHCN and NCAR) Axes same as 

Fig A1. 

Figure A4: Same as Fig A2 but showing only the adjusted lines (GHCN (adjusted) and CRUTEM). 

Axes same as Fig A1. 
APPENDIX 7

154

Figure A5: Taken from IPCC 4th Report Chapter 3.

The exercise and comparison of all figures demonstrates that: 

1. Any independent researcher may freely obtain the primary station data. It is 

impossible for any group to withhold data. 

2. It is impossible for any group to tamper improperly with data unless they have 

done so to the GHCN and NCAR (and presumably the NMO) sources 

themselves. 

3. The steps needed to create a temperature trend are straightforward to 

implement. 

4. The computer code necessary is straightforward to write from scratch and 

could easily be done by any competent programmer. 

5. The shape obtained in all cases is very similar: in other words if one does the 

same thing with the same data one gets very similar results. 

6. The result does not depend significantly on the exact list of stations. 

7. Adjustments make little difference. 

By performing this simple test one determines easily that the results of the CRUTEM 

analysis follow directly from the published description of the method, and that the 

resultant temperature trend is not significantly different from the other results 

regardless of stations used or adjustments made. The test is therefore sufficient to 

demonstrate that, with respect to the declared method, the CRUTEM analysis does not 

contain either error or adjustments which are responsible for the shape of the resultant 

temperature trend. 
APPENDIX 7

155

A researcher can evidently produce a study which would test the CRUTEM analysis 

quite precisely, without let or hindrance from CRU. 

We do not make any statement regarding the correctness of any of these results in 

representing global temperate trends. We do not address any alleged deficiencies 

such as allowance for non climatic effects or the significant drop in station number 

post 1991. We do not address any possible deficiencies of the gridding method. These 

are entirely matters for proper scientific study and debate and lie outside the scope of 

this Review. 

2.5 Discussion of Adjustments (Homogenisation) 

In our trial analysis the GHCN and NCAR lines are without homogenisation 

adjustment. The CRUTEM3 and GHCN (adjusted) lines are both after 

homogenisation adjustments. The adjustments to CRUTEM and GHCN are made in 

different ways by different people. Even so the results are extremely similar. It is 

evident from this comparison alone that neither CRU nor those assembling the GHCN 

(adjusted) dataset have made inappropriate adjustments in order to bias or falsify the 

global temperature trend simply because those adjustments have no significant effect 

on the global average. Nevertheless we consider additional evidence that the 

adjustments made have no significant effect on the global temperature trend. 

In the CRUTEM3 product the homogenisation adjustments are (i) made to only 298 

(10%) of the stations and (ii) are known to be approximately evenly distributed about 

zero. Thus they would not be expected to have a large effect in the global average. 

CRU have submitted detailed evidence of this (p49 and p54 of their submission) 

reproduced here as figure A6 and A7. Figure A6 shows a comparison of CRUTEM3 

with the results obtained by (i) not applying adjustments to the 298 stations and (ii) 

leaving out the 298 adjusted stations. There is no significant difference as expected. 

Figure A7 shows the CRUTEM3 analysis compared to unadjusted results from other 

groups. Again no significant difference is seen. This is fully in accord with the results 

of the trial analysis. 

Comparison of all of these plots demonstrates that adjustments which have been made 

are largely immaterial to the overall shape of the temperature series and are not on 

their own the cause of the rising temperature trend in recent years. 

We have not addressed the opposite question of whether the adjustments made are 

adequate to allow for non climatic effects. This is entirely a matter for proper 

scientific study and debate and outside the scope of this review. 
APPENDIX 7

156

FIGURE A6 CRUTEM3: Removing Adjustments 
APPENDIX 7

157

FIGURE A7 CRUTEM3: Comparison with unadjusted results from other groups 

3. Matching Station Identifiers 

We have used the information supplied by CRU from its website 

http://www.cru.uea.ac.uk/cru/data/landstations/. This lists the 4138 stations used in 

CRUTEM3. We have attempted to match these stations to those appearing the GHCN 

and NCAR data sets. 

This process is not straightforward (at least was not straightforward to us as non 
APPENDIX 7

158

climate scientists). Problems arose through 

Lack of uniqueness of station identification numbers; 

Different identifiers appearing for the same station in some cases; 

Different station names in a few cases (spelling, order of words); 

Locations differing by a few degrees; 

Different conventions for location; 

Ambiguity due to duplicates in GHCN 

We found it impossible to match stations purely from identifiers. Instead we used a 

combination of conjunctions of matching latitude and longitude, the first letters of 

station names, and station identifiers. We were in the end able to match the numbers 

shown in the table. 

Number in CRUTEM3 Matched in GHCN Matched in NCAR

4138 3722 (90%) 1957 (47%)

The number matched in GHCN (90%) is in accord with that claimed by CRU in their 

written responses. We have presumed that the remaining data is available from 

National Meteorological Offices or as part of more recent update messages. 

The Review feels that it has pursued this enough to understand that (i) the matching of 

meta data (lists of stations) across sources is not straightforward and (ii) obtaining the 

final 10% of the primary data would require some additional effort (although anyone 

familiar with the field would presumably have a better knowledge of how to do this). 

This is a secondary issue to this Review, but we make the following observations 

(i) It would benefit the global climate research community if a standardised 

way of defining station metadata and station data could be agreed, 

preferably through a standards body, or perhaps the WMO. As example an 

xml based format which would make the interpretation, use, comparison, 

and exchange of data much more straightforward. 

(ii) Without such standardisation there will remain residual problems in 

issuing unambiguous lists, and assembling primary data from them. We 

feel it would be in the public interest if CRU and other such groups 

developed a process to capture and publish a snapshot of the data used for 

each important publication. 
APPENDIX 7

159
APPENDIX 8

160

APPENDIX 8: SOFTWARE AND DATA STANDARDS 

Software Standards - The available guidance on software development methodologies 

is very extensive ranging from  ̳formal methods‘, where the desired properties of the 

system are expressed in a mathematically formal language, through to the  ̳Unified 

Process‘ developments of this decade. The UK Royal Academy of Engineering has 

published extensively in this area1

. 

Data Management and Archiving Guidance is also available on Records Management 

and archiving from: 

the Joint Information Systems Committee (JISC) of the Higher Education 

Funding Council2

; 

the UK Research Integrity Office (UKRIO)3; and 

the US National Academy of Sciences report ―Ensuring the Integrity, 

Accessibility and Stewardship of Research Data in the Digital Age‖4. We note 

in particular Recommendation 2: “Research institutions should ensure that 

every researcher receives appropriate training in the responsible conduct of 

research, including the proper management of research data in general and 

within the researcher‟s field of specialization. Some research sponsors 

provide support for this training and for the development of training 

programs”. We also note Recommendation 5: “All researchers should make 

research data, methods, and other information integral to their publicly 

reported results publicly accessible in a timely manner to allow verification of 

published findings and to enable other researchers to build on published 

results, except in unusual cases in which there are compelling reasons for not 

releasing data. In these cases, researchers should explain in a publicly 

accessible manner why the data are being withheld from release”.

Information Security - There is again very extensive guidance available of 

information security issues including: 

from a standards perspective ISO/IEC 27002; 

for thought leadership the ―Jericho Forum‖5and the ―Information Security 

Forum‖6

; and 

from a professional perspective the ―Institute of Information Security 

Professionals‖7

 

1

 http://www.raeng.org.uk/news/publications/list/reports/Engineering_values_in_IT.pdf 

 http://www raeng.org.uk/news/publications/list/reports/Complex_IT_Projects.pdf 

2

 http://www.jiscinfonet.ac.uk/partnerships/records-retention-he/managing-research-records 

3

See UKRIO  ̳Code of Practice for Research‘ Section 3.12 ―Collection and Retention of Data‖ at 

http://www.ukrio.org 

4

 http://www.nap.edu/catalog/12615 html 

5

 http://www.opengroup.org/jericho/ 

6

 https://www.securityforum.org/ 

7

 https://www.instisp.org/SSLPage.aspx