The Independent 

Climate Change 

E-mails Review 

July 2010

Chair: Sir Muir Russell

Review team:

Professor Geoffrey Boulton

Professor Peter Clarke

David Eyton

Professor James Norton
2

INDEX OF CONTENTS 

 

GLOSSARY 8

Chapter 1 EXECUTIVE SUMMARY 10

1.1 Introduction 10

1.2 The Review Process 11

1.3 Findings 11

1.3.1 Land Station Temperatures 12

1.3.2 Temperature Reconstructions from Tree Ring Analysis 12

1.3.3 Peer Review and Editorial Policy 13

1.3.4 Misuse of IPCC Process 13

1.3.5 Compliance with the Freedom of Information Act (FoIA) and the 

Environmental Information Regulations (EIR)

14

1.3.6 Other Findings on Governance 14

1.4 Recommendations 14

1.5 Broader Issues 14

Chapter 2 INTRODUCTION 18

2.1 Background 18

2.2 The Review 19

Chapter 3 TERMS OF REFERENCE AND METHOD OF ENQUIRY 22

3.1 Terms of Reference 22

3.2 Method of Enquiry 22

Chapter 4 CONTEXT OF THE E-MAILS 26

4.1 Characterising the E-mails 26

4.2 The Timeline 27

4.3 What the E-mails Tell Us 32

Chapter 5 THE CHANGING CONTEXT OF MODERN SCIENCE 36

5.1 The Scientific Process 36

5.2 The Nature of Climate Science 36

5.3 Data Accessibility in the Digital Age 37

5.4 Handling Uncertainty 38

5.5 Scientific Journals and the Peer Review Process 39

5.6 The Responsibilities of Scientists in Communicating in the Public Domain 40

5.7 Communicating to Policymakers 41

5.8 The Changing Forum for Debate and the Blogosphere 41

Chapter 6 LAND STATION INSTRUMENTAL TEMPERATURE DATA 44

6.1 Background 44

6.2 The Allegations 44

6.3 The Approach Adopted by the Review Team 45

6.4 The Results of the Analysis 46

6.5 Checking Specific Details in the CRUTEM Analysis 49

6.5.1 Identification of Data Sources 49

6.5.2 The Availability of Computer Codes 49

6.6 Use of Local temperature Data from China

6.7 Conclusions and Recommendations

51

53
3
4

Chapter 7 TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 54

7.1 Background 54

7.2 The Allegations 54

7.3 Findings 56

7.3.1 IPPC Reports 56

7.3.2 Divergence 59

7.3.3 Withholding Data 60

7.3.4 Mishandling Data 61

7.4 Conclusions and Recommendations 62

Chapter 8 PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC 

JOURNALS

64

8.1 Background: Peer Review, Testing and Verification 64

8.2 The Allegations 64

8.3 The Soon and Baliunas Affair & Climate Research 64

8.4 The Conflict with Dr Boehmer-Christiansen 66

8.5 Peer Review and Professor Briffa‘s Editorship of Holocene 67

8.6 Conclusions 68

Chapter 9 COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 70

9.1 Background 70

9.2 The Allegations 70

9.3 The CRUTEM Temperature Series 71

9.3.1 The Scientific Challenge 71

9.3.2 The Allegations 71

9.3.3 Evidence in Support of the Allegations 71

9.3.4 Jones‘ Response 73

9.3.5 Evidence from IPCC Review Editor for Chapter 3 (Professor Sir Brian 

Hoskins)

74

9.3.6 Findings 75

9.4 The Tree Ring Proxy Temperature Series 77

9.4.1 The Scientific Challenge 77

9.4.2 The Allegations 77

9.4.3 Evidence in Support of the Allegations 78

9.4.4 Responses from Briffa 80

9.4.5 Evidence from IPCC Review Editor for Chapter 6 (Professor John 

Mitchell)

82

9.4.6 Findings 83

9.5 Conclusions 84

Chapter 10 COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION

REGULATIONS

86

10.1 Introduction and Method of Enquiry 86

10.2 The Allegations 86

10.3 General Context 86

10.4 Investigation 89

10.5 Findings 91

10.6 Recommendations 94
5
6

Chapter 11 GOVERNANCE 98

11.1 Introduction and Method of Enquiry 98

11.2 Research Management Systems 98

11.2.1 Background 98

11.2.2 Funding Management 99

11.2.3 Funders‘ Requirements 99

11.2.4 Good Research Practice 99

11.2.5 Financial Controls 100

11.2.6 Risk Management 100

11.2.7 Findings on Research Management Systems 100

11.3 Software, Data Management and Data Security 101

11.3.1 General Context 101

11.3.2 Issues and the Investigation 102

11.3.3 Findings on Software, Data Management and Data Security 103

11.4 Recommendations 103

ACKNOWLEDGEMENTS 106

APPENDIX 1: REVIEW TEAM MEMBERS 108

APPENDIX 2: APPROACH AND WORK PLAN 110

APPENDIX 3: ISSUES FOR EXAMINATION 112

APPENDIX 4: INDEX OF MEETINGS, INTERVIEWS, SUBMISSIONS, 120

FOLLOW UP ENQUIRIES AND RESPONSES

APPENDIX 5: PEER REVIEW 126

APPENDIX 6: DATA MINING – ACCESS TO THE ORIGINAL CRU E-MAIL 146

ARCHIVE

APPENDIX 7: LAND STATION TEMPERATURE DATA 150

APPENDIX 8: SOFTWARE AND DATA STANDARDS 160
7
8

GLOSSARY 

AR4 Fourth Assessment Report of the Intergovernmental Panel on 

Climate Change in 2007

CLA Coordinating Lead Author

COP Conference of the Parties

COPE Committee on Publication Ethics

CRU Climatic Research Unit

CRUTEMX Land air temperature anomalies on a 5° by 5° grid-box basis, 

version X

DEFRA UK Government Department for Environment, Food and Rural 

Affairs

DPA Data Protection Act 1998

E&E Energy and Environment

EC European Community

EIR Environmental Information Regulations

ENSO El Niño-Southern Oscillation

ENV University of East Anglia School of Environmental Sciences

FOI Freedom of Information

FoIA Freedom of Information Act 2000

GHCN Global Historical Climatology Network

GISS Goddard Institute for Space Studies

GISTEMP Goddard Institute for Space Studies Surface Temperature Analysis

GWPF Global Warming Policy Foundation

HadCRUTX Combined land and marine temperature anomalies on a 5° by 5° 

grid-box basis, version X

IAC Inter Academy Council

ICCER Independent Climate Change E-Mails Review

ICO Information Commissioner‘s Office

ICT Information Communications Technology

IDL Interactive Data Language

IEC International Electrotechnical Commission

IPCC Intergovernmental Panel on Climate Change

IPCM Information Policy and Compliance Manager

IS Information Systems

ISSC Information Systems Strategy Committee

IT Information Technology

JANET The United Kingdom‘s Education and Research Network

JISC Joint Information Systems Committee

LA Lead Authors

LIA Little Ice Age

MBH Mann, Bradley and Hughes

MM McKitrick and Michaels
9

M&M McIntyre and McKitrick

MWP Medieval Warm Period

NAO North Atlantic Oscillation

NASA National Aeronautics and Space Administration

NCAR National Center for Atmospheric Research

NCDC National Climatic Data Center

NMO National Meteorological Office

NOAA National Oceanic and Atmospheric Administration

PERL Practical Extraction and Reporting Language

PI Principal Investigator

REE Research, Enterprise and Engagement

S&B Soon and Baliunas

SAP Scientific Assessment Panel

SPM Summary for Policy Makers

TAR Third Assessment Report of the IPCC in 2001

UEA University of East Anglia

UHI Urban Heat Island

UKRIO United Kingdom Research Integrity Office

UN United Nations

UNEP United Nations Environment Programme

WMO World Meteorological Organization

WWR World Weather Records
CHAPTER 1: EXECUTIVE SUMMARY 

10

CHAPTER 1: EXECUTIVE SUMMARY 

1. The main findings of the Independent Climate Change E-mails Review (―the 

Review‖) are set out in Section 1.3 below, and the main recommendations in 

Section 1.4. We comment in Section 1.5 on some of the more general issues 

raised by the Review that we think are important about the context in which 

scientists operate and in which science contributes to public policy. 

1.1 Introduction 

2. In November 2009, approximately 1000 e-mails from the Climatic Research Unit 

(CRU) of the University of East Anglia (UEA) were made public without 

authorisation. 

3. CRU is a small research unit which over the last 30 years has played an important 

role in the development of climate science, in particular in their work on 

developing global temperature trends. 

4. The e-mails fuelled challenges to the work of CRU, to the reliability of climate 

science generally, and to the conclusions of the Intergovernmental Panel on 

Climate Change (IPCC). All this happened shortly before the Copenhagen 

Summit, and was extensively referred to there. 

5. In response, the UEA commissioned two inquiries. The first led by Lord Oxburgh, 

into the science being undertaken at CRU, has already reported. This document is 

the report of the second inquiry – The Independent Climate Change E-mails 

Review – which examines the conduct of the scientists involved and makes 

recommendations to the University of East Anglia. Our inquiry addresses a 

number of important allegations that were made following the e-mail release. 

6. The allegations relate to aspects of the behaviour of the CRU scientists, such as 

their handling and release of data, their approach to peer review, and their role in 

the public presentation of results. 

7. The allegations also include the assertion that actions were taken to promote a 

particular view of climate change by improperly influencing the process of 

advising policy makers. Therefore we have sought to understand the significance 

of the roles played by those involved from CRU and of the influence they had on 

the relevant outcomes. 

8. The Review examines the honesty, rigour and openness with which the CRU 

scientists have acted. It is important to note that we offer no opinion on the 

validity of their scientific work. Such an outcome could only come through the 

normal processes of scientific debate and not from the examination of e-mails or 

from a series of interviews about conduct.
CHAPTER 1: EXECUTIVE SUMMARY 

11

1.2 The Review Process

9. The approach taken by the Review was to identify and investigate the allegations 

to which the e-mails gave rise. This reflected our reading of the emails and the 

comments made on them. An online consultation was undertaken to ensure that 

the Team‘s initial analysis of the allegations and concerns was sound. The method 

of investigation is explained in the relevant Chapters and Appendices to the 

report. The Review‘s evidence base is published on the website, which it intends 

to archive. 

10. In addressing the allegations about CRU‘s impact on climate science, we sought 

evidence to place these into perspective: 

On handling global temperature data, we went to global primary sources and 

tested how data was handled.

On tree-ring temperature reconstructions, we looked at the overall picture 

painted in Chapter 6 of the Fourth Assessment Report of the IPCC in 2007 

(AR4) and examined the influence of CRU.

On peer review, we sought independent input (from the Editor of The Lancet) 

on how the system works, to provide a context for our judgement.

On influencing the IPCC process, we sought advice from the Review editors 

on the role individual contributors can play.

11. This work provided a context in which we considered the evidence about the 

specific allegations made in the submissions and identified in our interviews with 

CRU and others. 

12. Reflecting this approach, the report and conclusions are set out as follows. The 

heart of the report lies in Chapters 6 through 10 where the important allegations 

arising from the e-mail release are examined. Chapters 2 and 3 contain 

introductory material, 4 deals with the body of e-mails and 5 presents important 

contextual material. The report concludes with Chapter 11 on other governance 

issues. 

1.3 Findings 

13. Climate science is a matter of such global importance, that the highest standards 

of honesty, rigour and openness are needed in its conduct. On the specific 

allegations made against the behaviour of CRU scientists, we find that their 

rigour and honesty as scientists are not in doubt. 

14. In addition, we do not find that their behaviour has prejudiced the balance of 

advice given to policy makers. In particular, we did not find any evidence of 

behaviour that might undermine the conclusions of the IPCC assessments.

15. But we do find that there has been a consistent pattern of failing to display 

the proper degree of openness, both on the part of the CRU scientists and on the 

part of the UEA, who failed to recognise not only the significance of statutory 
CHAPTER 1: EXECUTIVE SUMMARY 

12

requirements but also the risk to the reputation of the University and, indeed, to 

the credibility of UK climate science. 

1.3.1 Land Station Temperatures 

16. On the allegation of withholding temperature data, we find that CRU was not 

in a position to withhold access to such data or tamper with it. We 

demonstrated that any independent researcher can download station data directly 

from primary sources and undertake their own temperature trend analysis. 

17. On the allegation of biased station selection and analysis, we find no evidence 

of bias. Our work indicates that analysis of global land temperature trends is 

robust to a range of station selections and to the use of adjusted or unadjusted 

data. The level of agreement between independent analyses is such that it is highly 

unlikely that CRU could have acted improperly to reach a predetermined outcome. 

Such action would have required collusion with multiple scientists in various 

independent organisations which we consider highly improbable. 

18. On the allegation of withholding station identifiers we find that CRU should 

have made available an unambiguous list of the stations used in each of the 

versions of the Climatic Research Unit Land Temperature Record 

(CRUTEM) at the time of publication. We find that CRU‟s responses to 

reasonable requests for information were unhelpful and defensive.

19. The overall implication of the allegations was to cast doubt on the extent to 

which CRU‟s work in this area could be trusted and should be relied upon 

and we find no evidence to support that implication. 

1.3.2 Temperature Reconstructions from Tree Ring Analysis 

20. The central implication of the allegations here is that in carrying out their work, 

both in the choices they made of data and the way in which it was handled, CRU 

scientists intended to bias the scientific conclusions towards a specific result and 

to set aside inconvenient evidence. More specifically, it was implied in the 

allegations that this should reduce the confidence ascribed to the conclusions in 

Chapter 6 of the IPCC 4th Report, Working Group 1 (WG1).

21. We do not find that the way that data derived from tree rings is described 

and presented in IPCC AR4 and shown in its Figure 6.10 is misleading. In 

particular, on the question of the composition of temperature reconstructions, we 

found no evidence of exclusion of other published temperature reconstructions 

that would show a very different picture. The general discussion of sources of 

uncertainty in the text is extensive, including reference to divergence. In this 

respect it represented a significant advance on the IPCC Third Assessment 

Report (TAR). 

22. On the allegation that the phenomenon of “divergence” may not have been 

properly taken into account when expressing the uncertainty associated 

with reconstructions, we are satisfied that it is not hidden and that the 
CHAPTER 1: EXECUTIVE SUMMARY 

13

subject is openly and extensively discussed in the literature, including CRU 

papers. 

23. On the allegation that the references in a specific e-mail to a „trick‟ and to 

„hide the decline‟ in respect of a 1999 WMO report figure show evidence of 

intent to paint a misleading picture, we find that, given its subsequent iconic 

significance (not least the use of a similar figure in the IPCC Third 

Assessment Report), the figure supplied for the WMO Report was 

misleading. We do not find that it is misleading to curtail reconstructions at 

some point per se, or to splice data, but we believe that both of these procedures 

should have been made plain – ideally in the figure but certainly clearly 

described in either the caption or the text. 

24. On the allegations in relation to withholding data, in particular concerning 

the small sample size of the tree ring data from the Yamal peninsula, CRU 

did not withhold the underlying raw data (having correctly directed the 

single request to the owners). But it is evidently true that access to the raw 

data was not simple until it was archived in 2009 and that this delay can rightly 

be criticized on general principles. In the interests of transparency, we believe 

that CRU should have ensured that the data they did not own, but on which their 

publications relied, was archived in a more timely way. 

1.3.3 Peer Review and Editorial Policy

25. On the allegations that there was subversion of the peer review or editorial 

process we find no evidence to substantiate this in the three instances 

examined in detail. On the basis of the independent work we commissioned 

(see Appendix 5) on the nature of peer review, we conclude that it is not 

uncommon for strongly opposed and robustly expressed positions to be taken up 

in heavily contested areas of science. We take the view that such behaviour does 

not in general threaten the integrity of peer review or publication. 

1.3.4 Misuse of IPCC Process 

26. On the allegations that in two specific cases there had been a misuse by 

CRU scientists of the IPCC process, in presenting AR4 to the public and 

policy makers, we find that the allegations cannot be upheld. In addition to 

taking evidence from them and checking the relevant records of the IPCC 

process, we have consulted the relevant IPCC review Editors. Both the CRU 

scientists were part of large groups of scientists taking joint responsibility for the 

relevant IPCC Working Group texts, and were not in a position to determine 

individually the final wording and content. 
CHAPTER 1: EXECUTIVE SUMMARY 

14

1.3.5 Compliance with the Freedom of Information Act 

(FoIA) and the Environmental Information Regulations 

(EIR) 

27. On the allegation that CRU does not appear to have acted in a way 

consistent with the spirit and intent of the FoIA or EIR, we find that there 

was unhelpfulness in responding to requests and evidence that e-mails 

might have been deleted in order to make them unavailable should a 

subsequent request be made for them. University senior management should 

have accepted more responsibility for implementing the required processes for 

FoIA and EIR compliance. 

1.3.6 Other Findings on Governance

28. Given the significance of the work of CRU, UEA management failed to 

recognise in their risk management the potential for damage to the 

University‟s reputation fuelled by the controversy over data access. 

1.4 Recommendations 

29. Our main recommendations for UEA are as follows: 

Risk management processes should be directed to ensuring top management 

engagement in areas which have the potential to impact the reputation of the 

university. 

Compliance with FoIA/EIR is the responsibility of UEA faculty leadership 

and ultimately the Vice-Chancellor. Where there is an organisation and 

documented system in place to handle information requests, this needs to be 

owned, supported and reinforced by University leadership.

CRU should make available sufficient information, concurrent with any 

publications, to enable others to replicate their results. 

1.5 Broader Issues

30. Our work in conducting the Review has led us to identify a number of issues 

relevant not only to the climate science debate but also possibly more widely, on 

which we wish to comment briefly. 

31. The nature of scientific challenge. We note that much of the challenge to 

CRU‘s work has not always followed the conventional scientific method of 

checking and seeking to falsify conclusions or offering alternative hypotheses 

for peer review and publication. We believe this is necessary if science is to 

move on, and we hope that all those involved on all sides of the climate science 

debate will adopt this approach. 

32. Handling Uncertainty – where policy meets science. Climate science is an 

area that exemplifies the importance of ensuring that policy makers –
CHAPTER 1: EXECUTIVE SUMMARY 

15

particularly Governments and their advisers, Non-Governmental Organisations 

and other lobbyists – understand the limits on what scientists can say and with 

what degree of confidence. Statistical and other techniques for explaining 

uncertainty have developed greatly in recent years, and it is essential that they 

are properly deployed. But equally important is the need for alternative 

viewpoints to be recognized in policy presentations, with a robust assessment of 

their validity, and for the challenges to be rooted in science rather than rhetoric. 

33. Peer review - what it can/cannot deliver. We believe that peer review is an 

essential part of the process of judging scientific work, but it should not be overrated as a guarantee of the validity of individual pieces of research, and the 

significance of challenge to individual publication decisions should be not 

exaggerated. 

34. Openness and FoIA. We support the spirit of openness enshrined in the FoIA 

and the EIR. It is unfortunate that this was not embraced by UEA, and we make 

recommendations about that. A well thought through publication scheme would 

remove much potential for disruption by the submission of multiple requests for 

information. But at the level of public policy there is need for further thinking 

about the competing arguments for the timing of full disclosure of research data 

and associated computer codes etc, as against considerations of confidentiality 

during the conduct of research. There is much scope for unintended 

consequences that could hamper research: US experience is instructive. We 

recommend that the ICO should initiate a debate on these wider issues. 

35. Handling the blogosphere and non traditional scientific dialogue. One of the 

most obvious features of the climate change debate is the influence of the 

blogosphere. This provides an opportunity for unmoderated comment to stand 

alongside peer reviewed publications; for presentations or lectures at learned 

conferences to be challenged without inhibition; and for highly personalized 

critiques of individuals and their work to be promulgated without hindrance. 

This is a fact of life, and it would be foolish to challenge its existence. The 

Review team would simply urge all scientists to learn to communicate their work 

in ways that the public can access and understand. That said, a key issue is how 

scientists should be supported to explain their position, and how a public space 

can be created where these debates can be conducted on appropriate terms, 

where what is and is not uncertain can be recognised. 

36. Openness and Reputation. An important feature of the blogosphere is the 

extent to which it demands openness and access to data. A failure to recognise 

this and to act appropriately, can lead to immense reputational damage by 

feeding allegations of cover up. Being part of a like minded group may provide 

no defence. Like it or not, this indicates a transformation in the way science has 

to be conducted in this century. 

37. Role of Research Sponsors. One of the issues facing the Review was the 

release of data. At various points in the report we have commented on the 

formal requirements for this. We consider that it would make for clarity for 

researchers if funders were to be completely clear upfront in their requirements 

for the release of data (as well as its archiving, curation etc). 
CHAPTER 1: EXECUTIVE SUMMARY 

16

38. The IPCC. We welcome the IPCC‘s decision to review its processes, and can 

only stress the importance of capturing the range of viewpoints and reflecting 

appropriately the statistical uncertainties surrounding the data it assesses. Our 

conclusions do not make a judgement on the work of IPCC, though we 

acknowledge the importance of its advice to policy makers. 
CHAPTER 1: EXECUTIVE SUMMARY 

17
CHAPTER 2: INTRODUCTION 

18

CHAPTER 2: INTRODUCTION 

2.1 Background 

1. Prior to the 1960‘s there had been little investigation of past climatic changes and 

variability, except by geologists and botanists and on geological timescales. The 

Climatic Research Unit (CRU) in the UEA, established in 1972, was one of the 

first institutes in the world to address the science of climate change on more recent 

timescales. Its objective was ―to establish the past record of climate over as much 

of the world as possible, as far back in time as was feasible, and in enough detail 

to recognise the basic processes, interactions, and evolutions in the earth‘s fluid 

envelopes and those involving the earth‘s crust and its vegetation cover‖. A useful 

history of its work is available on its website: 

http://www.cru.uea.ac.uk/cru/about/history. 

2. CRU is a small organisation: at present there are around 16 staff, with 3.5 

established posts and the rest postgraduate students and post-doctoral researchers. 

Two of the main areas of its work are the focus of this review: the development of 

methodologies for calculating the extent to which the average temperature of the 

earth‘s land masses is changing, using instrumental temperature measurements 

made over the past 160 years (the period for which reliable measurements have 

been available); and the estimation of the earth‘s temperature over the last 

millennium, using tree ring data as a proxy for temperature. 

3. One of CRU‘s most important contributions to climate science is the production of 

a land based, gridded temperature data set showing how the temperature has 

varied year by year since 1850 relative to the 1961 to 1990 average. This work 

was started in 1978 and continues today. CRU is now one of a number of 

organisations working in this area. Others carrying out similar development of 

temperature records include the Goddard Institute for Space Studies (GISS) and 

the National Climatic Data Center (NCDC). 

4. Over the period that CRU has been in existence and especially over the last 20 

years, there has been a transformation in the importance attached to climate 

science. One clear indication of this was the creation of the Intergovernmental 

Panel on Climate Change (IPCC) in 1988, a body established by the United 

Nations Environment Programme (UNEP) and the World Meteorological 

Organization (WMO) ―to provide the world with a clear scientific view on the 

current state of climate change and its potential environmental and socioeconomic consequences‖. It assesses the most recent scientific, technical and 

socio-economic information produced worldwide that is relevant to the 

understanding of climate change. Members of CRU have played several 

significant roles on the IPCC. In its successive assessment reports the IPCC has 

sought to achieve a scientific consensus, but many continue to challenge the basis 

of its work and its conclusions. 

5. The pioneering work conducted by CRU therefore began to assume a great deal of 

significance in the international debate that surrounded climate science. Not 

surprisingly, given the enormous level of public interest in the subject, the debate 
CHAPTER 2: INTRODUCTION 

19

soon moved out of the confines of climate scientists and became highly polarized 

in websites, journals and conferences across the world. As a result, the work 

conducted by CRU became the focus of intense scrutiny and challenge with 

multiple demands from both fellow scientists and laymen for background 

information and data. 

6. In November 2009, the nature of the debate and challenge took on a whole new 

significance when approximately 1000 of CRU‘s e-mails were made public 

without authorization. This material fuelled more challenges to their work, to the 

reliability of climate science generally, and to the conclusions of the IPCC. All 

this happened shortly before the Copenhagen Summit, and was extensively 

referred to there. 

2.2 The Review 

7. The material in the e-mails led to a set of allegations against the leading members 

of CRU, focusing on whether CRU had operated in accordance with best scientific 

practice at the relevant time, and in particular: whether data had been manipulated 

or suppressed; whether peer review and the dissemination of findings had been 

properly handled; and whether CRU had complied with the Freedom of 

Information Act. 

8. In response to this, the Vice-Chancellor of the UEA established the Independent 

Climate Change E-mails Review, to be led by Sir Muir Russell. Sir Muir was 

given complete freedom to develop the terms of reference as necessary and to 

assemble an appropriate team and appropriate support; and was asked if possible 

to report in spring 2010. The terms of reference are in Chapter 3 and details of the 

team members are in Appendix 1. 

9. Two other formal inquiries have addressed the matter. First, the House of 

Commons Science and Technology Committee held a hearing on 1 March 2010, 

reporting on 31 March 20101. Secondly, the Vice-Chancellor commissioned 

Lord Oxburgh to review CRU‘s scientific output. The Oxburgh Scientific 

Assessment Panel and this Review proceeded entirely independently, though the 

latter took steps to ensure, following the House of Commons Science and 

Technology Committee‘s Report, that Lord Oxburgh was aware of the approach it 

was taking to issues that might bear on his work. Lord Oxburgh‘s report was 

published on 14 April 20102

.

10. In addition investigations have been initiated by the police, looking at the 

circumstances of the unauthorised disclosure of the e-mails; and by the ICO, 

looking at compliance with the Data Protection Act, the FoIA and the EIR. 

11. Responding to a joint request by the Chairman of the IPCC and the Secretary-

 

1

 Science and Technology Committee - Eighth Report: The disclosure of climate data from the Climatic 

Research Unit at the University of East Anglia 

2 Report by Lord Oxburgh‘s Science Assessment Panel 

http://www.uea.ac.uk/mac/comm/media/press/CRUstatements/SAP
CHAPTER 2: INTRODUCTION 

20

General of the United Nations, an independent review of the IPCC‘s processes 

and procedures is now under way by the InterAcademy Council (IAC), chaired by 

economist Harold T. Shapiro, former president of Princeton University. 

12. Although the focus of this Review is quite clearly on the behaviour of a number of 

scientists, it is also apparent that there are many wider issues raised by the events 

described above. These include issues such as the workings of the peer review 

system, the reporting of uncertainty in the translation of scientific findings into 

policy, the handling and release of data, and the role of research funders. As a 

contribution to the wider scientific debate about the conduct of science in the 21st

century, this Review also considers these questions. 
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

21
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

22

CHAPTER 3: TERMS OF REFERENCE AND METHOD 

OF ENQUIRY 

3.1 Terms of Reference 

1. The Review‘s terms of reference are as follows:

―The Independent Review will investigate the key allegations that arose from a 

series of hacked* e-mails from CRU. The Review will: 

Examine the hacked e-mail exchanges, other relevant e-mail exchanges and 

any other information held at CRU to determine whether there is any evidence 

of the manipulation or suppression of data which is at odds with acceptable 

scientific practice and may therefore call into question any of the research 

outcomes. 

Review CRU‘s policies and practices for acquiring, assembling, subjecting to 

peer review and disseminating data and research findings, and their 

compliance or otherwise with best scientific practice. 

Review CRU‘s compliance or otherwise with the University‘s policies and 

practices regarding requests under the Freedom of Information Act ( ̳the 

FoIA‘) and the Environmental Information Regulations ( ̳the EIR‘) for the 

release of data. 

Review and make recommendations as to the appropriate management, 

governance and security structures for CRU and the security, integrity and 

release of the data it holds.‖

*Note: The word  ̳hacked‘ as contained in the Review‘s terms of reference 

has been challenged in submissions to the Review, on the basis that the means 

by which the unauthorized disclosure of the e-mails was made has not been 

established. This matter is subject to police enquiries and the Review has 

made no judgment on the question. 

3.2 Method of Enquiry 

2. The Review sought to operate in an open and transparent way. This is described in 

the  ̳Approach and Work Plan‟ paper (Appendix 2) made public at the launch at 

the Science Media Centre in February 2010. The Review called for submissions, 

which it undertook to publish on its website: http://www.ccereview.org/Evidence.php 

3. To provide a focus for submissions the Review produced and placed on its 

website an  ̳Issues for Examination‟ paper (Appendix 3) which addresses the main 

allegations made against the members of CRU. This paper set out our initial 

understanding based on a preliminary reading of the e-mails. Approximately 100 

submissions were then received, including one from CRU itself. Some provided 

further background considered relevant by the authors, but there was no 

suggestion in the submissions that the original statement of the issues was 

fundamentally misdirected. As its work progressed, the team kept the issues paper 

constantly under review and gradually developed a sharper focus on the key 
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

23

allegations. 

4. Thus this report is structured to address the principal allegations. It is important to 

recognise that this is not a detailed inquiry into the precise meaning of every email. 

5. The team proceeded to investigate the allegations by interviewing members of 

CRU and others from the University. We considered that the nature of our inquiry 

was such that holding public hearings to gather evidence, as some had urged, 

would be unlikely to add significant value over and above the written record. Nor 

have we produced transcripts of the interviews. This is because our conclusions 

are founded on information given in submissions and at interviews relating to 

facts that can be checked and referenced, rather than on interview testimony as 

such. The team found that this process helped it follow up key points, leading to 

supplementary submissions and references. 

6. The team did not carry out interviews other than with CRU and other UEA staff 

(apart from preliminary discussions with ICO and the police and interviews with 

two relevant IPCC Review Editors). We recognise that natural justice requires 

that those in respect of whom findings will be made should have an opportunity to 

be heard: this does not apply to the authors of submissions and other parties, in 

respect of whom the Review has made no findings. 

7. Under our publication policy notes of the team‘s meetings and interviews have 

been placed on the website. Interview notes have been checked for factual 

accuracy with those involved. Only those that are potentially sensitive, such as 

notes of discussions with the police, have been withheld. The Team has sought to 

publish all the submissions received unless they were explicitly described as 

confidential, subject only to the redaction, on legal advice, of content the 

publication of which could have led to legal action. In addition, one submission 

has been withheld on legal advice. The Review has invited the author to make it 

available directly to enquirers. It is a matter for the author whether to proceed in 

this way. This submission has been given full consideration. 

8. A full list of submissions and details of meetings can be found in Appendix 4. 

9. It is important to note that the allegations relate to aspects of the behaviour of the 

CRU scientists and not to the content of their work and hence this Review 

addressed scientific questions only to the extent necessary to place this behaviour 

in context. 

10. On the subject of peer review, in addition to investigating the specific allegations 

the team commissioned a contextual paper from a distinguished Journal editor –

Dr Richard Horton of The Lancet – on which Elizabeth Wager, Chair of the 

Committee on Publication Ethics (COPE), was invited to comment in a personal 

capacity. This material forms Appendix 5 and is referred to Chapters 5 and 8. 

11. The Review maintained contact with the police and the ICO. It was important to 

ensure that the work of the team did not compromise their statutory 

responsibilities, and also to avoid duplication of effort. The team‘s findings and 
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

24

recommendations respect this division of responsibilities and the fact that 

investigations are continuing. 

12. The Review intends that the website and the submissions and evidence published 

on the website should be archived by the British Library Web Archive. 
CHAPTER 3: TERMS OF REFERENCE AND METHOD OF ENQUIRY 

25
CHAPTER 4: CONTEXT OF THE E-MAILS 

26

CHAPTER 4: CONTEXT OF THE E-MAILS 

4.1 Characterising the E-mails 

1. This Chapter gives an overview of the e-mails and seeks to relate their content to 

key events happening in the climate change world, thus giving context to the 

criticism of CRU.

2. The information released comprises a very small (less than 0.3%) subset of files 

which were held on the back-up server at CRU, which include e-mails and other 

documents – such as text files, Word documents, Excel spreadsheets, PDF 

documents, and computer code. 

3. The focus of this Chapter is e-mails which spanned the period 7th March 1996 to 

12th November 2009. The  ̳primary‘ e-mails number 1073 in total with 166 

authors. There are more e-mails and authors if the associated e-mail chains are 

included. When printed on A4 paper the e-mails run to 3,375 pages and contain 

many embedded duplicates. Self-evidently each of the primary e-mails was either 

sent by or received by CRU members, but this is not the case for many of the 

associated e-mail chains. Those who authored the largest numbers of primary emails are as follows. 

Author Number Role

Philip Jones 174 Director, CRU, UEA and Coordinating Lead Author 

IPCC 4th Assessment Report

Michael 

Mann

140 Director, Earth System Science Centre, Pennsylvania 

State University (from 2005), and Lead Author IPCC 

3

rd Assessment Report

Keith Briffa 117 Professor, CRU, UEA and Lead Author IPCC 4th

Assessment Report

Jonathan 

Overpeck

90 Institute Director, University of Arizona and 

Coordinating Lead Author IPCC 4th Assessment 

Report

Tim Osborn 59 Academic Fellow, CRU, UEA and Contributing 

Author IPCC 4th Assessment Report

Ben Santer 51 Researcher, Lawrence Livermore National Lab, US

and Contributing Author IPCC 4th Assessment Report

Tom 

Wigley

35 Scientist, University Corporation for Atmospheric 

Research, Contributing Author IPCC 4th Assessment 

Report and a former Director of CRU

4. The e-mails relate to a number of the major developments in recent climate 

science from the Kyoto Summit in December 1997, through the 4th Assessment 

Report of the IPCC in 2007, to the run-up to the 15th Conference of the Parties 

(COP) in Copenhagen. Since CRU played a significant role in providing scientific 

input to these events, the release of the emails resulted in reduced global 
CHAPTER 4: CONTEXT OF THE E-MAILS 

27

confidence in climate science and more specifically in the findings of the IPCC. 

4.2 The Timeline 

5. The  ̳story‘ which underlies the selected e-mails, and hence is reflected in them, is 

summarised in the table and text which follow. 

Year Event

1997 Kyoto Summit

1998 MBH981including what has become known as the ―Hockey Stick‖

1999 MBH992reconstruction extended to 1000 AD

WMO Statement on the Status of the Global Climate in 19993

2001 IPCC 3rd Assessment Report4

2002 Esper et al 20025

2003 Soon & Baliunas 20036, McIntyre &McKitrick 20037criticism of MBH

Mann et al 20038,9

2004 von Storch 200410 questioning the statistical methods used in MBH

Launch of RealClimate11 website

 

1 M.E. Mann, R.S. Bradley and M.K. Hughes, ―Global-scale temperature patterns and climate forcing 

over the past six centuries‖, Nature 392, 779-787 23 April (1998) 

2 M.E. Mann, R.S. Bradley and M.K. Hughes, ―Northern hemisphere temperatures during the past 

millennium: Inferences, uncertainties, and limitations‖, Geophysical Research Letters, Volume 26. No. 

6, pp. 759-762 (1999) 

3

 http://www.wmo.ch/pages/prog/wcp/wcdmp/statemnt/wmo913.pdf 

4

 http://www.grida.no/publications/other/ipcc_tar/ 

5

 J. Esper, E.R. Cook, F.H. Schweingruber, ―Low-Frequency Signals in Long Tree-Ring Chronologies 

for Reconstructing Past Temperature Variability‖, Science, Volume 295, 22 March (2002) pp. 2250-

2253

6

 W. Soon, S. Baliunas, ―Proxy climatic and environmental changes of the past 1000 years‖, Climate 

Research Volume 23: 89-110, January 2003 

7

 S. McIntyre, R. McKitrick, ―Corrections to the Mann et. al. (1998) proxy data base and Northern 

hemispheric average temperature series‖, Energy & Environment, Volume 14, No. 6 (2003) pp. 751-

771

8 M.E. Mann and P.D. Jones, ―Global surface temperatures over the past two millennia‖, Geophysical 

Research Letters Volume 30, No. 15, 1820, August 2003 

9

 M.E Mann, C.M Ammann, R.S. Bradley, K.R. Briffa, T.J. Crowley, M.K. Hughes, P.D. Jones, M. 

Oppenheimer, T.J. Osborn, J.T. Overpeck, S. Rutherford, K.E. Trenberth, T.M.L. Wigley, ―On past 

temperatures and anomalous late 20th century warmth‖, Eos, 84, 256-258, 2003. 

10 H. von Storch, E. Zorita, J.M. Jones, Y. Dimitriev, F. González-Rouco, S.F.B. Tett, ―Reconstructing 

Past Climate from Noisy Data‖, Science, 22 October 2004, Volume 306. no. 5696, pp. 679 – 682 

11 http://www.realclimate.org/ 
CHAPTER 4: CONTEXT OF THE E-MAILS 

28

Year Event (continued)

2005 UK Freedom of Information Act12

Launch of ClimateAudit13 website

McIntyre &McKitrick 200514 criticism of MBH

Rutherford et al 200515 defence of MBH

2006 US National Research Council review16

US Committee on Energy & Commerce (Wegman) statistical review17

2007 IPCC 4th Assessment Report18

Wahl & Ammann 200719defence of MBH

2009 15th COP in Copenhagen

CRU‘s work assumed progressively greater significance as it became 

apparent that both the global temperature record and tree ring data it was 

collecting and analysing were being cited prominently in the work of the 

IPCC. 

Building in part on the work of CRU, Mann, Bradley & Hughes (MBH) 

published a paper in Nature in 199820 which sought to reconstruct historic 

temperatures back to 1400 AD, well before significant numbers of 

instrumental climate records began in around 1850. They used 

measurements including those from tree rings, ice cores and corals, socalled proxy data, which can reflect local temperature changes. This 

paper was updated in 199921 with the reconstruction extended back to 

1000 AD in the Northern Hemisphere. The resulting temperature profile 

had a  ̳hockey stick‘ shape. In particular it reduced the significance of 

what is termed the Medieval Warm Period (MWP), that is believed to 

have reached its peak between about 950 and 1100 AD, and the Little Ice 

Age (LIA) (from about 1500 to 1850 AD, and centered around 1700 AD), 

relative to previous representations. Temperatures over the last 50 years 

appeared unprecedented in the past thousand years. These data were used 

to create an iconic representation of anthropogenic global warming, as 

supplied by Jones, in the 1999 World Meteorological Organisation for its 

 

12 http://www.ico.gov.uk/what_we_cover/freedom_of_information.aspx 

13 http://climateaudit.org/ 

14 S. McIntyre, R. McKitrick, ―Hockey sticks, principal components, and spurious significance‖, 

Geophysical Research Letters, Volume 32, No. 3 (2005) 

15 S. Rutherford, M.E. Mann, T.J. Osborn, R.S. Bradley, K.R. Briffa, M.K. Hughes, P.D. Jones, 

―Proxy-based Northern Hemisphere surface temperature reconstructions: sensitivity to methodology, 

predictor network, target season and target domain‖, Journal of Climate (2005); 18: 2308-2329

16 Surface Temperature Reconstructions for the Last 2,000 Years 

http://www.nap.edu/catalog.php?record_id=11676 

17 Ad Hoc Committee Report on the  ̳Hockey Stick‘ Global Climate Reconstruction (2006) for the US 

Committee on Energy and Commerce 

18 http://www.ipcc.ch/ 

19 E.R. Wahl, C.M. Ammann, ―Robustness of the Mann, Bradley, Hughes reconstruction of Northern 

Hemisphere surface temperatures: examination of criticisms based on the nature and processing of 

proxy climate evidence‖ Climatic Change, Volume 85, Numbers 1-2, pp.33-69, November 2007 

20 Ibid. 1 

21 Ibid. 2 
CHAPTER 4: CONTEXT OF THE E-MAILS 

29

Statement on the Status of the Global Climate in 199922. They were also 

referenced in the Third Assessment Report (TAR) of the IPCC in 200123. 

The IPCC process occurs over a number of years, with drafts being 

produced periodically, leading up to the final report and summary for 

policy makers (SPM). The process is clearly documented24 with extensive 

voluntary involvement of many eminent scientists in authoring and 

reviewing chapters of the report. Key players in CRU were involved in 

the development of both the TAR and AR4 in 2007 and many of the emails relate to their involvement in the process. 

Questions were increasingly asked about MBH. In 2002 Esper published 

a paper25 which suggested that MBH had underestimated the strength of 

the MWP, but it was Soon & Baliunas‘s (S&B) papers published in 

Climate Research26 and with minor amendments in Energy & 

Environment (E&E)27 which first challenged MBH‘s results directly. An 

e-mail by Osborn circulating S&B‘s 2003 paper28 is one of the most 

duplicated in the released e-mails triggering nine of the subsequently 

selected e-mail chains. S&B‘s papers were followed in 2003 by another 

paper by McIntyre and McKitrick (M&M)29 also critical of MBH. At 

issue in the whole exchange of e-mails were the following key scientific 

issues:- 

 (i) the selection of tree ring data (particularly those from Yamal 

and the Polar Urals); 

(ii) the statistical methods for extracting information; 

(iii) the record of local and hence global temperatures (especially 

the impact of the urban heat island (UHI) effect); 

(iv) relating proxy records to instrumental temperature records; and 

(v) uncertainties in comparing recent warmth with that over the last 

2000 years. 

Prior to the publication of S&B‘s papers in 2003 those critical of MBH 

had not had a paper published in a mainstream journal. The publications 

in Climate Research and Energy & Environment were significant not only 

because they challenged MBH but also because they had been peer 

reviewed. Not only were rebuttals published in 2003 by Mann, Jones, 

Briffa, Osborn et al30,31

, but also the process of peer review at Climate 

Research was questioned. The editor, de Freitas, sought initially to 

 

22 Ibid. 3 

23 Ibid. 4 

24 http://www.ipcc.ch/organization/organization_procedures htm 

25 Ibid. 5 

26 Ibid. 6 

27 W. Soon, S. Baliunas, C. Idso, S. Idso. D.R. Legates, ―Reconstructing climatic and environmental 

changes of the past 1000 years: a reappraisal‖, Energy & Environment, Volume 14: Nos. 2 & 3, 2003, 

pp. 233-296

28 Ibid. 6 

29 Ibid. 7 

30 Ibid. 8 

31 Ibid. 9 
CHAPTER 4: CONTEXT OF THE E-MAILS 

30

defend himself as the e-mail extract below shows but he ultimately 

resigned as did members of the editorial board of Climate Research. The 

matter is discussed in detail in Section 8.3 of Chapter 8. 

In an e-mail dated 18/6/03 (1057944829.txt) de Freitas wrote: 

“I have spent a considerable amount of my time on this matter and had 

my integrity attacked in the process. I want to emphasize that the people 

leading this attack are hardly impartial observers. Mike himself refers to 

"politics" and political incitement involved. Both Hulme and Goodess are 

from the Climate Research Unit of UEA that is not particularly well 

known for impartial views on the climate change debate. The CRU has a 

large stake in climate change research funding as I understand it pays the 

salaries of most of its staff.”

In 2004 von Storch32 questioned the statistical methods used in MBH and, 

at around the same time, the RealClimate33 and Climate Audit34 websites 

were launched, the former defending and the latter critical of majority 

climate science. This marked a new phase of more public climate science 

debate and criticism, and was seen as an unwelcome development by 

those in CRU, who wanted criticism to follow the normal channels of 

private peer review and publication. This view remained relatively 

unchanged for over a decade, as shown by the following: 

In an e-mail dated 6/5/99 (926031061.txt) Jones wrote: 

“I must admit to having little regard for the Web. Living over here makes 

that easier than in the US - but I would ignore the so-called skeptics until 

they get to the peer-review arena. I know this is harder for you in the US 

and it might become harder still at your new location. I guess it shows 

though that what we are doing in [sic] important. The skeptics are 

fighting a losing battle.”

In an e-mail dated 30/9/09 (1254323180.txt) Jones wrote: 

“Another issue is science by blog sites - and the then immediate response 

mode. Science ought to work through the peer-review system..... sure 

you've said all these things before. We're getting a handful of nasty emails 

coming and requests for comments on other blog sites.”

Critics of CRU claimed that they were unable to reproduce CRU‘s work 

due to a lack of access to data. The UK EIR and FoIA35 should have 

encouraged the release of information, but this does not seem to have 

been the initial result. The volume of requests grew over time, more 

formally under the FoIA starting in 2007 and spectacularly in 2009, 

particularly related to details of the CRUTEM land temperature datasets 

 

32 Ibid. 10 

33 Ibid. 11 

34 Ibid. 13 

35 Ibid. 12 
CHAPTER 4: CONTEXT OF THE E-MAILS 

31

and the role CRU played in the development of IPCC AR4. 

 

In an email dated 7/5/04 (1083962601.txt) Jones wrote: 

“Many of us in the paleo field get requests from skeptics (mainly a guy 

called Steve McIntyre in Canada) asking us for series. Mike and I are not 

sending anything, partly because we don't have some of the series he 

wants, also partly as we've got the data through contacts like you, but 

mostly because he'll distort and misuse them. Despite this, Mike and I 

would like to make as many of the series we've used in the RoG [Reviews 

of Geophysics] plots available from the CRU web page.”

We do not suggest that the allegations made against McIntyre are correct. 

In the run up to IPCC AR4 further papers were published criticising 

(McIntrye & McKitrick 200536) and defending MBH (Rutherford 200537

, 

Wahl and Ammann 200738). In parallel, political interest in AR4 grew 

and in the United States of America two reviews were commissioned, one 

by the National Research Council39 into surface temperature 

reconstructions for the last 2000 years and the other by the Committee on 

Energy and Commerce (Wegman)40 into the quality of the statistical 

analysis used in MBH. These reviews were critical of MBH; nevertheless 

the AR4 SPM41 contained the following statement. 

“Average Northern Hemisphere temperatures during the second half of 

the 20th century were very likely higher than during any other 50-year 

period in the last 500 years and likely the highest in at least the past 1300 

years.”

In the aftermath of AR4, attempts were made to demonstrate that the 

IPCC process had been abused by CRU members. Jones was accused of 

attempting to exclude reference to a 2004 paper by McKitrick and 

Michaels42, and Briffa of including reference to Wahl 200743 before it was 

in publication, contrary to IPCC rules. 

These accusations were accompanied by Freedom of Information FOI 

requests. In an email dated 10/12/08 (1228922050.txt) Jones wrote: 

“Anyway requests have been of three types - observational data, paleo 

data and who made IPCC changes and why. Keith has got all the latter –

 

36 Ibid. 14 

37 Ibid. 15 

38 Ibid. 19 

39 Ibid. 16 

40 Ibid. 17 

41 AR4 WG1 Summary for policymakers: 

http://www.ipcc.ch/publications_and_data/ar4/wg1/en/spmsspm-a-palaeoclimatic.html 

42 R. McKitrick and P.J Michaels, ―A test of corrections for extraneous signals in gridded surface 

temperature data‖, Climate Research 26, 159-173, 2004 

43 Ibid. 19 
CHAPTER 4: CONTEXT OF THE E-MAILS 

32

and there have been at least 4. We made Susan aware of these - all came 

from David Holland. According to the FOI Commissioner's Office, IPCC 

is an international organization, so is above any national FOI. Even if 

UEA holds anything about IPCC, we are not obliged to pass it on, unless 

it has anything to do with our core business - and it doesn't!”

4.3 What the E-mails Tell Us

6. Since the communication was assumed to be private, it was generally informal, 

using slang, jargon and acronyms. Now that the e-mails have become public, 

some are doubtless regretted by their authors. Indeed, some submissions have 

characterised them as  ̳unprofessional‘, or as evidence of CRU‘s contribution to a 

 ̳poisoned atmosphere‘ in climate science. The question is whether this was 

unusual, or simply characteristic of the normal mode of expression in e-mail 

communication. 

7. This is a well known issue in social psychology and communication. Early 

authoritative works44,45,46, reinforced by later authors, explore normative language 

in electronic communication in comparison with other written and oral forms. 

They demonstrate that e-mail communication is less inhibited than other written 

and spoken forms, and suggest reasons why. Extreme forms of language are 

frequently applied to quite normal situations by people who would never use it in 

other communication channels. 

8. It is also clear from the submissions that it is possible to place different 

interpretations on the same phrase. In such circumstances, only the original 

author can really know what their intentions were. For example, on 16/11/99 

(942777075.txt) Jones wrote his now infamous e-mail including the following: 

“I've just completed Mike's Nature trick of adding in the real temps to each series 

for the last 20 years (ie. from 1981 onwards) amd [sic] from 1961 for Keith's to 

hide the decline.”

McIntyre in his submission47 to the Review Team, states 

“The IPCC “trick” was not a “clever” mathematical method – it was merely the 

deletion of inconvenient data after 1960.”

This compares with Jones‘s own commentary in the UEA submission48 to the 

Review Team. 

“The email was written in haste and for a limited and “informed” audience (the 

people that had provided data). The word “trick” was not intended to imply any 

 

44 Sproull, L. and Kiesler, Sara. 1986. Reducing Social Context Cues: Electronic Mail in Organisational 

Communications. Management Science, 32, p.1492-1512

45 Sproull, L. and Kiesler, S. 1991. Connections. MIT Press, Cambridge MA. 

46 Shapiro, N and Anderson, R. 1985. Towards and Ethics and Etiquette for Electronic Mail. Rand 

Corporation. 

47 McIntyre Submission (No.23), 1 March 2010, paragraph 22 

48 Climatic Research Unit Submission (No.5), 1 March 2010, 3.1 on p.36 
CHAPTER 4: CONTEXT OF THE E-MAILS 

33

deception, simply the “best way of doing or dealing with something”. The 

reconstruction from the tree-ring density network was not shown after 1960, and 

thus in this sense it is “hidden” – but justifiably so: excluding the anomalous 

tree-ring density data is justified if the purpose is to illustrate the most likely 

course of temperature based on a combination of proxy and measured 

temperatures. Again, no deception was intended or practised.”

9. E-mails are rarely definitive evidence of what actually occurred. They are open to 

interpretation, but they are also indicative. Having identified specific allegations 

against CRU, based on the e-mails, the Review then sought to obtain evidence to 

substantiate or refute these allegations, as described in the subsequent Chapters of 

this report. 

10. The presumption is that e-mails were selected to support a particular viewpoint. 

Recognising that they were a tiny fraction of those archived, the Review Team 

sought to learn more about the full contents of the back-up server. This attempt, 

summarised in Appendix 6, was largely unsuccessful due to the sheer scale of the 

task and ongoing police investigation. 

11. In some instances the leaked e-mails contain statements which might be viewed as 

supporting the behavior of CRU and countering the allegations in paragraph 9 

above. The following extracts from e-mails by Briffa are examples. 

E-mail dated 22/9/99 (938018124.txt): 

“the early (pre-instrumental) data are much less reliable as indicators of global 

temperature than is apparent in modern calibrations that include them and when 

we don't know the precise role of particular proxies in the earlier portions of 

reconstruction it remains problematic to assign genuine confidence limits at 

multidecadal and longer timescales.”

E-mail dated 3/2/06 (1138995069.txt): 

“we are having trouble to express the real message of the reconstructions - being 

scientifically sound in representing uncertainty , while still getting the crux of the 

information across clearly...We have settled on this version (attached) of the 

Figure which we hoe [sic] you will agree gets the message over but with the rigor 

required for such an important document.”

12. On the other hand there are some e-mails which show that CRU members may 

have gone out of their way to make life difficult for their critics, just as they 

perceived the critics to be frustrating their work. Some examples follow: 

 

Jones e-mail dated 27/4/05 (1114607213.txt): 

“I got this email from McIntyre a few days ago. As far as I'm concerned he has 

the data - sent ages ago. I'll tell him this, but that's all - no code. If I can find it, it 

is likely to be hundreds of lines of uncommented fortran ! I recall the program 

did a lot more that just average the series. I know why he can't replicate the 

results early on - it is because there was a variance correction for fewer series.”
CHAPTER 4: CONTEXT OF THE E-MAILS 

34

Jones e-mail dated 3/12/08 (1228330629.txt):

“The inadvertent email I sent last month has led to a Data Protection Act request 

sent by a certain Canadian, saying that the email maligned his scientific 

credibility with his peers! If he pays 10 pounds (which he hasn't yet) I am 

supposed to go through my emails and he can get anything I've written about him. 

About 2 months ago I deleted loads of emails, so have very little - if anything at 

all.”

Osborn e-mail dated 23/6/08 (1214228874.txt):

“I've just had a quick look at CA. They seem to think that somehow it is an 

advantage to send material outside the formal review process. But *anybody* 

could have emailed us directly. It is in fact a disadvantage! If it is outside the 

formal process then we could simply ignore it, whereas formal comments had to 

be formally considered. Strange that they don't realise this and instead argue for 

some secret conspiracy that they are excluded from! I'm not even sure if you sent 

me or Keith anything, despite McIntyre's conviction! But I'd ignore this guy's 

request anyway. If we aren't consistent in keeping our discussions out of the 

public domain, then it might be argued that none of them can be kept private. 

Apparently, consistency of our actions is important.”

13. During the work of the Review an additional concern surfaced, namely that of 

financial controls, for example see extract below. This is addressed in Chapter 11 

(paragraph 18). In an e-mail dated 7/3/96 (826209667.txt) Shiyatov (A Russian 

tree ring researcher) wrote: 

“It is important for us if you can transfer the ADVANCE money on the personal 

accounts which we gave you earlier and the sum for one occasion transfer (for 

example, during one day) will not be more than 10,000 USD. Only in this case we 

can avoid big taxes and use money for our work as much as possible.”

14. Finding: The extreme modes of expression used in many e-mails are 

characteristic of the medium. Crucially, the e-mails cannot always be relied upon 

as evidence of what actually occurred, nor indicative of actual behaviour that is 

extreme, exceptional or unprofessional. 

15. The Chapters which follow address specific behavioural allegations based in 

particular on a comparatively small number of e-mails. Despite the fact that all 

the e-mails released are unlikely to be representative of the larger set on the CRU 

back-up server, there nevertheless would appear to be a pattern of behaviour. 

There is little doubt about the polarisation of views in the world of climate 

science, which has overstepped the line dividing heated scientific debate from 

outright hostility. One camp comprises the main authors of the e-mails in 

paragraph 3, who are acknowledged leaders in majority climate science as 

indicated by the roles they all played in the IPCC. The other camp is their critics, 

for whom pejorative terms such as ―prat, dishonest, appalling, rubbish and crap‖ 
CHAPTER 4: CONTEXT OF THE E-MAILS 

35

were used by some CRU members to refer to them or their work. More generally 

the majority climate scientists appear to have been united in their defence against 

criticism. Whilst perhaps understandable, given the nature and methods of 

criticism, some of which impugns their personal integrity as well as challenging 

their work, this may have blinded some CRU members to the possibility of merit 

therein. Such denial then fuelled yet further antagonism. There needs to be better 

communication, as well as greater openness enabling more scientific debate. We 

comment on this in Chapters 5 and 10. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

36

CHAPTER 5: THE CHANGING CONTEXT OF MODERN 

SCIENCE 

1. In this Chapter we review the changing context of modern science as a framework 

against which CRU‘s practices can be judged. 

2. When CRU was created in the early 1970s, climate change was a relatively 

obscure area of science. But it has developed into an area of great political and 

public concern. CRU found itself ―in the eye of the storm‖ in generating much of 

the data necessary to address one of the most important issues of the century. 

3. This has come at a time of transformation in the need for openness in the culture 

of publicly-funded science. It is being driven by a range of pressures including 

new recommendations from national academies addressing the timely release of 

data, increased demands for scientific input to public policy, important changes in 

the law and challenge from the blogosphere. 

5.1 The Scientific Process

4. Scientific hypotheses and theories are presumed to be provisional: they can be 

refuted by testing but they cannot be verified as correct or true in an absolute 

sense. Verification is however possible of the results of the experiments upon 

which theories are built or that attempt to test a theory. If they can be repeated, 

and produce the same results, they are said to be validated. It is important to 

recognise that science progresses by substantive challenges based on rigorously 

logical, published arguments that present a different view of reality from that 

which they challenge. Criticising and attacking process and behaviour is not the 

same as an attack on a scientific hypothesis. Failure in one is not necessarily 

failure in the other. 

5. Given the nature of our remit, our concern is not with science, whether data has 

been validated or whether the hypotheses have survived testing, but with 

behaviour; whether attempts have been made to misrepresent, or ―cherry pick‖ 

data with the intention of supporting a particular hypothesis, or to withhold data so 

that it cannot be independently validated, or to suppress other hypotheses to 

prevent them being put to the test. 

5.2 The Nature of Climate Science 

6. Modern climate science is largely a creation of the last 30-40 years. It involves a 

wide range of disciplines, including geography, physics, chemistry, a wide variety 

of earth sciences, palaeo-biology and computer modelling. The synthesis of such 

diverse sources of evidence generated through wide variety of scientific 

approaches and traditions has been a major task, in which the IPCC has played an 

important role. 

7. One of the main contributions of CRU has been in the laborious assembly of 

datasets of instrumental measures of climate change and of data relating to tree 

rings that are proxies for climatic variables. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

37

5.3 Data Accessibility in the Digital Age

8. Modern digital technologies permit the acquisition and manipulation of very much 

larger datasets than formerly. To enable proper validation of the conclusions, 

such datasets must be made freely available, along with details of the associated 

computational manipulation. It is often difficult and time-consuming to describe 

this  ̳meta-data‘ in ways that permit others to verify experimental results and test 

the inferences drawn from them. The preservation of such data also presents a 

novel problem of archiving and stewardship compared with the archives of former 

times. 

9. An important shift in attitude is represented by a recent report by the US National 

Academies1

, which highlights these issues. It recommends a new approach to the 

integrity, accessibility, and stewardship of data in publicly-funded science, 

arguing that researchers should make all research data, methods, and other 

information underlying the results publicly accessible in a timely manner. These 

recommendations would require a substantial shift of behaviour amongst many 

scientists. The e-mails suggest that this would be true for CRU – for example: 

Jones: “We have 25 or so years invested in the work. Why should I make the data 

available to you, when your aim is to try and find something wrong with it”

2

.

10. Digital datasets in specific areas of science are increasingly being placed in 

international archives. The World Data System, run by the International Council 

for Scientific Unions3, for example, covers a very wide range of data types 

including the International Tree-Ring Data Bank. The majority of the source data 

used to create the CRUTEM gridded product is available through the Global 

Historical Climatology Network (GHCN) managed by the NCDC4

 and the US 

National Center for Atmospheric Research (NCAR)5.

11. Some funders of research require that applications for funding include plans for 

data archiving. The Natural Environment Research Council (NERC), the UK 

research council most relevant to the work of CRU, has a policy that requires 

recipients of grants to offer to deposit with NERC a copy of datasets resulting 

from the supported research6

. 

12. Over the period under review, data accessibility was also being driven by major 

changes in the law that had important implications for research scientists working 

in the UK. Whilst the precursors of the Environmental Information Regulations 

date back to the 1990s, the current EIR came into force on 1st January 2005, on the 

same date as the general right of access under the Freedom of Information Act 

2000 (FoIA). These are described in more detail in Chapter 10. The EIR applies 

exclusively within a rather broad definition of environmental and related 

 

1

 National Academy of Sciences. 2009. Ensuring the Integrity, Accessibility, and Stewardship of 

Research Data in the Digital Age. 188pp. 

2

 Referred to in Sherrington submission (No. 38), 17 February 2010, page 10) 

3

 http://www.ukssdc.ac.uk/wdcmain/guide/wdcguide html 

4

 http://www.ncdc noaa.gov/oa/ncdc.html 

5

 http://www.ncar.ucar.edu/ 

6

 NERC Data Policy (Version February 2010) 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

38

information; this area is then excluded from the FoIA. The extent to which both 

regimes apply to access to information from publicly funded (or part-funded) 

research is still an evolving area. Whilst it is broadly recognised that these do 

apply to data and supporting metadata, algorithms, etc. at the point when a 

research paper is published, how to interpret the legislation and regulations in 

terms of data held for a long period prior to publication and also pre-publication 

drafts and other correspondence is work in progress. The  ̳public interest test‘, 

applicable to both EIR and FoIA, again described in Chapter 10, remains key to 

such interpretation. 

5.4 Handling Uncertainty 

13. There is a widespread misconception that science produces unequivocal and 

absolutely precise answers. It does not, and cannot. All scientific results contain 

uncertainties, and it is important that these are made clear to and are understood 

by those who use them. There are two fundamental sources of uncertainty in 

science: uncertainty in measuring a phenomenon and uncertainty in determining 

causes and causal relationships. 

14. Over the period in question, the increasing amount of quantitative data available 

on climate change has made it easier to assess and represent uncertainty. This can 

clearly be seen in the representation of what is termed the Medieval Warm Period. 

Prior to the 1980s this was shown as a single line, without error bars7. Subsequent 

quantitative palaeo-temperature work on tree rings, in which CRU has been a 

leader, has permitted errors to be assessed as a consequence of the statistical 

correlations involved (see Chapter 7). Although the recent Oxburgh Scientific 

Assessment Panel8

 has been critical of some of the statistical work of CRU in 

relation to the tree ring series, that issue lies beyond our remit. Our concern is 

whether tree ring series have been improperly selected and whether the 

uncertainties have been properly presented. 

15. The processes of hypothesis, experiment, testing and refutation generally reduce 

uncertainties about causes and relationships but do not entirely remove them. 

Some of the conflicts involving CRU have been scientific disagreements about 

causes and relationships that in principle should be resolved by new data which 

exposes existing hypotheses to rigorous testing. One of the allegations against 

members of CRU has been that they have attempted improperly to prevent ideas 

with which they disagreed from being effectively discussed, for example in 

relation to the publication of papers (Chapter 8) and in parts of the IPCC process 

(Chapter 9). 

 

7

 e.g. Houghton, J.T., Jenkins, G.J. and Ephraums J.J. 1990. Climate Change: the IPCC Scientific 

Assessment. Cambridge University Press. 

8 Report of the International Panel set up by the University of East Anglia to examine the research of 

the Climatic Research Unit. 2010. Available at: 

http://www.uea.ac.uk/mac/comm/media/press/CRUstatements/oxburgh 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

39

5.5 Scientific Journals and the Peer Review Process 

16. Allegations against CRU include improper attempts to influence the editorial 

policy of scientific journals and the peer review process. To help it understand the 

context, the Review commissioned a contextual paper from a distinguished 

Journal editor – Dr Richard Horton of The Lancet – on which Elizabeth Wager, 

Chair of COPE, was invited to comment in a personal capacity. (Appendix 5). 

17. Peer review is important because of the way that scientific journals provide the 

frame of reference for most formal scientific debate. Scientific ideas that are not 

expressed in a scientific journal are rarely addressed. Edward Wilson, for 

example, commented that  ̳a discovery does not exist until it is safely reviewed 

and in print‟.

9

18. Access to publication in scientific journals is therefore a crucial issue. Editors 

have used peer review by acknowledged experts of papers submitted for 

publication to advise them whether they contain errors, are important, trivial or 

simply repeat what has previously been published. 

19. Richard Horton‘s essay explores typical patterns of behaviour relating to journals 

and peer review in contentious areas of science, and what behaviour should be 

regarded as improper. His conclusions are summarised as follows: 

What an editor seeks from a reviewer is a powerful critique of the manuscript 

– testing each assumption, probing every method, questioning all results, 

sceptically challenging interpretations and conclusions and ensuring that 

uncertainties are fully acknowledged, measured, and reported. 

Armed with such a critique, the editors decide, and take full responsibility for 

deciding, whether to publish. 

There is always the risk of group-think among experts which resists alternative 

perspectives. Editors try to reduce the risk of group-think by sending papers to 

different and widely dispersed reviewers, deliberately seeking or even 

provoking critical reviews. 

Editors send manuscripts to reviewers based on a principle of confidentiality. 

The author expects the editor to maintain a covenant of trust between the two 

parties. Disclosure to a third party without the prior permission of the editor 

would be a serious violation of the peer review process and a breach of 

confidentiality. 

Many who are far from the reality of the peer review process would like to 

believe that peer review is a firewall between truth on the one hand and error 

or dishonesty on the other. It is not. It is a means of sieving out evident error, 

currently unacceptable practices, repetition of previously published work 

without acknowledgement, and trivial contributions that add little to 

 

9

 Wilson, E.O. 1998. Consilience: The Unity of Knowledge. New York: Knopf. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

40

knowledge. 

It does not and cannot guarantee veracity. Many published papers have proved 

deeply flawed: many good ones have been rejected. Nor has it been efficient in 

identifying fraud, which has usually come to light by different routes. 

However, journals, as the gatekeepers of scientific publication, have come to 

exert an increasing influence on the reputations of scientists, research units and 

universities. Many measures of academic success, promotion, tenure, grants, 

fame, and personal wealth depend upon journal publication. It is not surprising 

therefore that journals, and peer review, are the subject of constant tension and 

occasionally explosive controversy. 

20. In conclusion, it is common for editors to have multiple, intense, and sometimes 

sharp and passionate interactions with authors and reviewers. The tone of their 

exchanges and communications with editors can be attacking, accusatory, 

aggressive, and even personal. If a research paper is especially controversial and 

word of it is circulating in a particular scientific community, third-party scientists 

or critics with an interest in the work may get to hear of it and decide to contact 

the journal. They might wish to warn or encourage editors. This kind of 

intervention is entirely normal. It is the task of editors to weigh up the passionate 

opinions of authors and reviewers, and to reflect on the comments (and 

motivations) of third parties. Occasionally, a line might be crossed into highly 

improper behaviour leading, for example, to censorship of ideas that might 

normally pass peer review. Defining that line is the crucial task when judging the 

role of CRU scientists, and determining whether, as has been alleged, they acted 

to subvert peer review by slowing or blocking the publication of research which 

disagreed with their own views. Was their activity part of the normal, robust 

hurly-burly surrounding publication in important highly contended fields, or was 

an important line crossed? We address this in Chapter 8. 

5.6 The Responsibilities of Scientists in Communicating in 

the Public Domain 

21. The scientific literature is relatively opaque to non-specialists. Scientific 

understanding that is transmitted into the public domain must be comprehensible 

to non-specialists, make appropriate and not excessive claims, and include careful 

statements of the uncertainties surrounding that understanding. These principles 

apply universally to scientific advice, and are embedded in the codes of practice 

for scientific advice that have been adopted by many governments10. One of the 

allegations against CRU is that they have not been sufficiently frank in 

communicating uncertainties about their reconstructions into the public domain. 

This is particularly relevant to graphical presentation such as the  ̳hockey stick‘ 

which has taken on iconic significance. Images have a great power to persuade, 

and this is particularly true when complex issues are faced by lay audiences, who 

may often infer a level of certainty that does not in fact exist. The danger is 

 

10 For example, the ―Code of Scientific Ethics‖ published in 2007 by the UK Chief Scientific Advisor, 

and accepted by UK Government, includes the responsibility to: ―present and review scientific 

evidence, theory or interpretation honestly and accurately.‖
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

41

obviously heightened where an image is being used to support arguments for 

policy change. Therefore, if images are likely to be used in this way, it is essential 

that qualifications such as uncertainties are given a closely coupled prominence 

and explanation. 

5.7 Communicating to Policymakers 

22. The interface between science and public policy is a crucial one in matters of great 

public importance such as climate change. The IPCC was set up to provide just 

such an interface. Its job was to draw on and synthesise the diverse strands that 

contribute to modern climate science (see paragraph 6) and to make this accessible 

to the public, policymakers and other stakeholders in a way that is comprehensible 

and that does justice to underlying uncertainties. The importance of this process is 

underlined by the potential magnitude of the economic and social consequences of 

governmental decisions in the domain of global climate change. It has been 

alleged that CRU scientists subverted IPCC processes by minimising uncertainties 

and blocking ideas that disagreed with their established views. 

23. The IPCC produces assessments of the current state of understanding of climate 

change, its causes and implications. Its approach is to produce the most probable 

account of these issues; together with their uncertainties, and to identify where 

there is insufficient evidence to discriminate between different interpretations of a 

phenomenon. Its purpose is to produce a  ̳best estimate‘ of what is currently 

understood, through the work of a group of scientists chosen for their expertise 

and experience to make reasoned assessments on the balance of evidence. It is not 

to produce a review of the scientific literature. 

24. The IPCC assessment reports were published in 1990, 1996, 2001 and 2007. AR4, 

published in 2007, consisted of three components: The Physical Science Base

(Working Group 1 – WG1); Impacts, Adaptation and Vulnerability (WG2); and 

Mitigation of Climate Change (WG3). The issues that concern CRU are a 

consequence of their involvement in WG1. The core activities of these WGs are 

undertaken by writing teams consisting of Coordinating Lead Authors (CLAs) and 

Lead Authors (LAs). There are two drafting cycles prior to the production of the 

Final Draft. The first draft is submitted to Expert Reviewers for comment. The 

second draft is submitted for both expert and governmental review. Review 

Editors are appointed for each of the review cycles to ensure that critical 

comments are properly dealt with. The final draft is the source of a Summary for 

Policymakers, which is also reviewed by governments prior to publication. 

5.8 The Changing Forum for Debate and the Blogosphere 

25. The development in recent years of the internet as a vehicle for easy, 

instantaneous transmission of news and opinion has changed the nature of the 

debate about scientific issues. Prior to these developments, scientific debate 

largely took place in journals and conferences that effectively excluded the public 

from active engagement. Experts tended to introduce their conclusions to the 

public in ways that were difficult to challenge. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

42

26. The mode has now changed and the field of climate change exemplifies this. 

There continues to be a scientific debate about the reality, causes and uncertainties 

of climate change that is conducted through the conventional mechanisms of peerreviewed publication of results, but this has been paralleled by a more vociferous, 

more polarised debate in the blogosphere and in popular books. In this the 

protagonists tend to be divided between those who believe that climate is 

changing and that human activities are contributing strongly to it, and those that 

are sceptical of this view. This strand of debate has been more passionate, more 

rhetorical, highly political and one in which each side frequently doubts the 

motives and impugns the honesty of the other, a conflict that has fuelled many of 

the views expressed in the released CRU emails, and one that has also been 

dramatically fuelled by them. It is difficult at the moment to predict whether and 

how the necessary cooler, rigorous scientific debate and the vital public policy 

interface will develop, or the effect that it will have on scientific publication or 

peer review. 

27. Arguably the most significant change produced by the blogosphere is a 

transformation in the degree of openness now required of scientists whose work 

directly affects policy making. Without such openness, the credibility of their 

work will suffer because it will always be at risk of allegations of concealment 

and hence mal-practice. The extent to which this change was fully recognised by 

both CRU and UEA administration is an important issue for the Review. 

28. Therefore, the Review would urge all scientists to learn to communicate their 

work in ways that the public can access and understand; and to be open in 

providing the information that will enable the debate, wherever it occurs, to be 

conducted objectively. That said, a key issue is how scientists should be 

supported to explain their position, and how a public space can be created where 

these debates can be conducted on appropriate terms, where what is and is not 

uncertain can be recognised. The learned societies may have an expanded role to 

play here in encouraging debate. We would also commend the work of bodies 

such as the Science Media Centre at the Royal Institution for encouraging and 

helping scientists to take their work to lay audiences through the media, and 

advising them on how best to do this. 
CHAPTER 5: THE CHANGING CONTEXT OF MODERN SCIENCE 

43
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

44

CHAPTER 6: LAND STATION INSTRUMENTAL 

TEMPERATURE DATA

6.1 Background 

1. Terrestrial temperature data is recorded at more than seven thousand land stations 

across the world. This is referred to as primary data. 

2. The steps involved in producing CRU‘s analysis of global temperature trends 

involve assembling and quality checking the primary data from approximately 

four thousand stations, determining the monthly ―anomalies‖1for each station, and 

then assembling averages based on a uniform 5x5 degree grid covering the globe. 

3. This work, requiring considerable experience and man-years of effort, was started 

by CRU in 1978 and it continues today. The results were published in a series of 

CRUTEM datasets over a period of 20 years between 1986 and 2006. 

4. In assembling a gridded temperature dataset, some adjustment of primary data 

may be needed in order to allow for what are referred to as  ̳non-climatic effects‘. 

A simple example might involve adjusting temperatures at a particular station to 

allow for obvious recording errors, or the elimination of an obvious discontinuity 

in results if a station had been moved (referred to as homogenization). Such 

adjustments are both necessary and ubiquitous in scientific research. 

5. This Chapter deals with the availability of these data to anyone wishing to carry 

out an equivalent study, the effect of adjustments, the comparison with 

independent studies and the availability of the information required to explicitly 

check the CRUTEM analysis. Finally, it addresses specific allegations about 

Chinese data used in a paper about the effect of urbanization on temperature. 

6.2 The Allegations 

6. The broad allegations which are prevalent in the public domain are: 

That CRU prevented access to raw data. 

That CRU adjusted the data without scientific justification or adequate 

explanation. Some allegations imply that this was done to fabricate evidence 

for recent warming. 

That CRU inappropriately withheld data and computer code, thus inhibiting 

others from checking the conclusions of the CRUTEM analysis. 

The overall implication of these allegations is to cast doubt on the extent to which 

CRU‘s work in this area can be trusted and should be relied upon. 

 

1 A station  ̳normal‘ is the average recorded temperature over a defined time period. In the case of 

CRUTEM this is 1961-1990. The temperature ―anomaly‖ is defined as the difference between the 

recorded temperature and the normal. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

45

7. While very few of the submissions to this Inquiry make the allegations above 

explicitly, they are nevertheless implied. The full text of the submissions is 

contained on the team website. It is not the intent here to review each one, but 

only to highlight the most relevant and significant. 

8. The most comprehensive and substantive submission critical of CRU is from 

McKitrick2. He addresses the question of data adjustments and also clarifies that 

the issue is not generally one of data availability, but more specifically the 

availability of a list of additional stations used in the CRUTEM analysis since 

1986. He draws attention to the series of requests to CRU for station identifiers 

over the period 2002-2007, eventually culminating in a series of FOI requests, 

also referred to by Matthews3

 and Cockroft4. Two submissions5 allege that CRU 

has withheld access to primary data and by implication stopped others from 

repeating analyses, but present no evidence. A common theme in the submissions 

from the Global Warming Policy Foundation (GWPF), McKitrick and McIntyre6

is the lack of cooperation exhibited by CRU in their dealings with selected third 

parties. 

9. In support of CRU, several submissions7,8,9

 point out that the data is in the public 

domain and can be accessed by anyone wishing to repeat a temperature trend 

analysis. Santer also defends the work done by CRU on regional adjustments, 

maintaining that it made very little difference to the global results. He also 

highlights that studies conducted independently from CRU, based on different 

choices and adjustment protocols, produce very similar results – a point also made 

by Joos10 and Tett.

10. In its own submission under issue 4, CRU11 defends its position with regard to 

adjustments and openness. The website also contains the minutes of the meeting 

which took place on the subject of data sets on March 4th, 2010 between members 

of CRU and the Review Team. 

6.3 The Approach Adopted by the Review Team 

11. In order to test the principal allegations of withholding data and making 

inappropriate adjustments, the Review undertook its own trial analysis of land 

station temperature data. The goal was to determine whether it is possible for an 

independent researcher to (a) obtain primary data and (b) to analyse it in order to 

produce independent temperature trend results. This study was intended only to 

test the feasibility of conducting such a process, and not to generate scientific 

conclusions. The process followed is described in Appendix 7. 

 

2

 McKitrick submission (no.15), 26 February 

3

 Matthews submission (no.16), 1 March 

4

 Cockroft submission (no. 6), 28 February 

5

 Global Warming Policy Foundation submission (no. 22), 28 February, and Cockroft (Ibid.) 

6

 McIntyre submission (no. 23), 2 March 

7

 Tett submission (no. 21), 1 March 

8

 Santer submission (no. 44), 28 February 

9

 Barnes submission (no. 1), 6 March 

10 Joos submission (no. 12), 1 March 

11 CRU submission (no. 5), 1 March 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

46

12. The Team then went on to address a further question, namely the extent to which 

an independent researcher could check the CRUTEM analysis. In particular, this 

considered: 

whether the data sources were properly explained 

the availability of software required in the analysis 

6.4 The Results of the Analysis 

13. To carry out the analysis we obtained raw primary instrumental temperature 

station data. This can be obtained either directly from the appropriate National 

Meteorological Office (NMO) or by consulting the World Weather Records 

(WWR) available through the National Centre for Atmospheric Research 

(NCAR), or the Global Historical Climatology Network (GHCN): 

NCAR: http://www.ncar.ucar.edu/tools/datasets/ 

WWR: http://dss.ucar.edu/datasets/ds570.0/ 

GHCN: http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php. 

14. Anyone working in this area would have knowledge of the availability of data 

from these sources. There are also other sources but we have not investigated 

these.

15. The Review extracted a comprehensive set of global temperature records from 

both GHCN and NCAR websites to test ease of availability from either. The 

stations included in these sources have much overlap as would be expected. In 

addition, as a test case, primary data was also obtained directly from the Japanese 

NMO. 

16. In a true analysis the researcher would at this point be able to make whatever 

quality control, station selection and adjustments they considered appropriate, 

which should be justified in the published record of research. 

17. Figure 6.1 below gives the results of the Team‘s trial analysis to produce three 

different global temperature anomaly average series. These use a 5x5 degree grid 

and a 5-year smoothing (for details see Appendix 7). In each case we started with 

the full dataset, requiring only that a station had enough data to construct the 

relevant normals when constructing the gridded averages. Apart from this we 

made no selection or other adjustments of our own. We show the result obtained 

using the GHCN unadjusted data set (blue), the GHCN adjusted data set (yellow) 

and the NCAR dataset which is unadjusted (green). On the same figure we show 

as a black line the global average series obtained from the land air temperature 

anomalies on a 5x5 degree grid-box basis (CRUTEM3) gridded analysis (which 

uses adjusted data). The green and blue lines are both from unadjusted datasets. 

The yellow and black lines are from adjusted datasets. 

18. All the lines, whether from adjusted or unadjusted datasets, agree very well. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

47

Figure 6.1: Temperature anomaly time series created by the Review Team‘s own trial analysis using a 

5x5 degree grid with 5 year smoothing. Shown are results obtained from GHCN (blue), GHCNadjusted (yellow) and NCAR (green). Also shown is the CRUTEM3 line (black). The Y-axis is 10 x 

the anomaly in degrees. The X-axis is year. 

19. In Figure 6.2 we show the results of independent scientific studies, where 

individual researchers have brought to bear their own judgment on station 

selection and adjustments to the data. Such studies have been carried out by 

NASA-GISS and NOAA-NCDC using the GHCN dataset. The figure from the 

IPCC 4th Report Chapter 3 is reproduced below to show these in comparison to 

CRUTEM312. Once again there is good agreement. 

 

12 The reader is referred to the comprehensive submission from the UK Met Office (CRU 54) to the 

House of Commons Committee on Science and Technology Inquiry which shows a similar unsmoothed 

figure. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

48

Fig 6.2 Reproduced from the IPCC 4th Report Chapter 3. 

20. Finding: This simple analysis and the comparisons in figures 6.1 and 6.2 give rise 

to the following findings: 

Any independent researcher may freely obtain the primary station data. It is 

impossible for a third party to withhold access to the data. 

It is impossible for a third party to tamper improperly with the data unless they 

have also been able to corrupt the GHCN and NCAR sources. We do not 

consider this to be a credible possibility, and in any case this would be easily 

detectable by comparison to the original NMO records or other sources such 

as the Hadley Centre. 

The steps needed to create a global temperature series from the data are 

straightforward to implement. 

The required computer code is straightforward and easily written by a 

competent researcher. 

The shape of the temperature trends obtained in all cases is very similar: in 

other words following the same process with the same data obtained from 

different sources generates very similar results. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

49

21. By performing this simple test one determines easily that the results of the 

CRUTEM analysis follow directly from the published description of the method, 

and that the resultant temperature trend is not significantly different from the other 

results regardless of stations used or adjustments made. The test is therefore 

sufficient to demonstrate that, with respect to the declared method, the CRUTEM 

analysis does not contain either error or adjustments which are responsible for the 

shape of the resultant temperature trend. 

22. A researcher can evidently produce a study which would test the CRUTEM 

analysis quite precisely, without requiring any information from CRU to do so. 

23. Finding: The high level trial analysis shows that adjustments make little difference 

to the overall conclusions on temperature trends. However, this has been a matter 

about which much comment has been made and it is therefore discussed in more 

detail in Appendix 7. Adjustments were made by CRU to the data from 10% of 

stations. The Review Team is satisfied that this had no significant effect upon 

results. Very strong evidence for this is furnished by the level of agreement 

between CRUTEM and the other lines shown in Figure 6.1, as well as other 

studies detailed in the Appendix, showing the effect of de-adjusting and of 

removing the adjusted stations. 

24. It should be noted that in making these findings, the Review Team is making no 

statement regarding the correctness of any of these analyses in representing global 

temperature trends. We do not address any alleged deficiencies such as allowance 

for non climatic effects or the significant drop in station number post 1991. We do 

not address any possible deficiencies of the method. These are entirely matters for 

proper scientific study and debate and lie outside the scope of this Review. 

6.5 Checking Specific Details in the CRUTEM Analysis 

25. The goal here is to consider the requirements for carrying out an exact replication 

of the CRUTEM work to produce an identical gridded analysis. The work above 

demonstrates that this procedure is not necessary if the goal is simply to produce 

an alternative temperature trend analysis. As discussed in Appendix 7, and as 

follows from the close agreement between the lines in Figures 6.1 and 6.2, it is 

also not necessary to test the robustness of the CRUTEM3 analysis. However, the 

Review upholds the important principle that any scientific result should be open to 

such scrutiny for whatever reason. It therefore considered whether all the 

necessary information was made available. 

6.5.1 Identification of Data Sources 

26. In order to reproduce exactly a CRUTEM study, the independent researcher 

requires the exact list of stations used in order to source the primary data 

themselves. 

27. The original version of the CRUTEM gridded averages was published in 1986 (in 

this report we call this CRUTEM1986) using 3276 stations (314 of which were 

adjusted). Subsequent revisions were produced in 1994, 2003 and 2006, referred 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

50

to as CRUTEM1, 2 and 3. CRUTEM3 used 4138 stations (298 were adjusted). 

Whereas the 3276 stations used in CRUTEM1986 are listed in the respective 

references13, this is not the case for the additional approximately 1000 stations 

included in CRUTEM314. The additional stations are described only in general 

terms in the respective publication via references to the sources. 

28. For some years prior to the coming into force of the general right of access to 

information under FoIA, CRU had received requests for data, including station 

identifiers. An example of the attitude to these requests is given in the following 

e-mail extract: 

Jones to Mann on 2nd February 2005 (1107454306.txt): 

“And don't leave stuff lying around on ftp sites - you never know who is trawling 

them. The two MMs have been after the CRU station data for years. If they ever 

hear there is a Freedom of Information Act now in the UK, I think I'll delete the 

file rather than send to anyone. Does your similar act in the US force you to 

respond to enquiries within 20 days? - our does ! The UK works on precedents, 

so the first request will test it. We also have a data protection act, which I will 

hide behind. Tom Wigley has sent me a worried email when he heard about it - 

thought people could ask him for his model code. He has retired officially from 

UEA so he can hide behind that. IPR should be relevant here, but I can see me 

getting into an argument with someone at UEA who'll say we must adhere to it !”

29. A request was made to CRU for both station identifiers and access to the raw data 

for these stations. This was formally logged by the UEA Information Policy & 

Compliance Manager (IPCM) in January 2007 and was dealt with under the FoIA. 

The request was initially refused entirely on the grounds that raw station data was 

publicly available (from GHCN, NCAR and the NMOs), but without reference to 

providing station identifiers. Following further correspondence, an internal UEA 

appeal process and further prompting by the applicant, the list of the 4138 stations 

used in CRUTEM3 was finally released on 1st October. The issues that this raises 

 

13 Bradley, R.S., Kelly, P.M., Jones, P.D., Goodess, C.M. and Diaz, H.F., 1985: A Climatic Data Bank 

for Northern Hemisphere Land Areas, 1851-1980, U.S. Dept. of Energy, Carbon Dioxide Research 

Division, Technical Report TRO17, 335 pp. 

Jones, P.D., Raper, S.C.B., Santer, B.D., Cherry, B.S.G., Goodess, C.M., Kelly, P.M., Wigley, T.M.L., 

Bradley, R.S. and Diaz, H.F., 1985: A Grid Point Surface Air Temperature Data Set for the Northern 

Hemisphere, U.S. Dept. of Energy, Carbon Dioxide Research Division, Technical Report TRO22, 251 

Jones, P.D., Raper, S.C.B., Cherry, B.S.G., Goodess, C.M. and Wigley, T.M.L., 1986: A Grid Point 

Surface Air Temperature Data Set for the Southern Hemisphere, 1851-1984, U.S. Dept. of Energy, 

Carbon Dioxide Research Division, Technical Report TR027, 73 pp. 

Jones, P.D., Raper, S.C.B., Bradley, R.S., Diaz, H.F., Kelly, P.M. and Wigley, T.M.L., 1986: Northern 

Hemisphere surface air temperature variations: 1851-1984. Journal of Climate and Applied 

Meteorology 25, 161-179. 

Jones, P.D., Raper, S.C.B. and Wigley, T.M.L., 1986: Southern Hemisphere surface air temperature 

variations: 1851-1984. Journal of Climate and Applied Meteorology 25, 1213 1230 

14 Brohan, P., Kennedy, J., Harris, I., Tett, S.F.B. and Jones, P.D., 2006: Uncertainty estimates in 

regional and global observed temperature changes: a new dataset from 1850. J. Geophys. Res. 111, 

D12106, doi:10.1029/2005JD006548. 

Jones, P.D. and Moberg, A., 2003: Hemispheric and large-scale surface air temperature variations: An 

extensive revision and an update to 2001. J. Climate 16, 206-223. 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

51

on compliance with the FoIA are dealt with in Chapter 10. 

30. The Review Team verified that matching of stations to a simple list of identifiers 

is not a straightforward process. The identification numbers are not unique across 

data sets, station names differ, and in the case of GHCN several replicas of the 

same data exist. This is primarily a problem caused by the lack of standardisation 

of metadata within the global climate science community. We make a 

recommendation on this below. 

31. The Review Team was able to match 90% of stations given in the CRU list to 

GHCN (see Appendix 7). CRU has stated in a written submission15 that that the 

remaining 10% can be obtained from other sources including the NMOs. Thus 

substantial work is required to take the CRU published list and assemble 100% of 

the primary station data from global repositories and NMOs. We make a 

recommendation for the future below. 

32. Finding: The Review finds that as a matter of good scientific practice, (and having 

established the precedent with CRUTEM1986) CRU should have made available 

an unambiguous list of the stations used in each of the versions of CRUTEM at 

the time of publication. In the absence of this, CRU was unhelpful and defensive 

and should have responded throughout to requests for this information in a more 

timely way. 

6.5.2 The Availability of Computer Codes 

33. Finding: The computer code required to read and analyse the instrumental 

temperature data is straightforward to write based upon the published literature. It 

amounts a few hundred lines of executable code (i.e. ignoring spaces and 

comments). Such code could be written by any research unit which is competent 

to reproduce or test the CRUTEM analysis. For the trial analysis of the Review 

Team, the code was written in less than two days and produced results similar to 

other independent analyses. No information was required from CRU to do this. 

 

34. The Met Office Hadley centre has published Practical Extraction and Reporting 

Language (PERL) codes which are a few hundred lines long. The submission by 

Barnes is also very helpful and illustrative in this context. 

6.6 Use of Local Temperature Data from China 

35. The above account of our work has focused on global data, its handling and 

accessibility. We also considered the implication in a submission to us by Dr 

Benny Peizer of the Global Warming Policy Foundation16 that Jones was 

complicit in malpractice in failing to respond appropriately to allegations of fraud 

made against a climate scientist at the State University of New York, Albany, 

Professor Wei-Chyung Wang in relation to more local data. An important paper 

 

15 Follow up request (20 April) to Professor Jones and response in relation to raw instrument station 

availability for each CRUTEM set 

16 Peizer/Global Warming Policy Foundation submission (no. 22), 28 February
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

52

on the effect of urbanization on temperature by Jones et al (1990)17 included 

data from China (Wang et al, 1990)18 that contained the statement that only 

instrument stations were selected where there had been few, if any, changes 

in instrumentation, location or observation times. It has been claimed by 

Keenan (2007)19 that this latter statement was knowingly untrue 

(“fabricated”), that stations had been moved, thus that their record of 

temperature change would be unreliable, and that hard copies of details of 

station histories had been lost or withheld. A subsequent paper by Jones et 

al. (2008)20 verified the original conclusions for the Chinese data for the 

period 1954–1983, showing that the precise location of weather stations was 

unimportant to the outcome 

36. Peizer, as Editor of Energy and Environment, to which the Keenan paper had 

been submitted, had contacted Jones asking him to comment on the Keenan 

paper, which Jones did, by denying its criticisms and advising rejection of the 

paper21. Jones was encouraged by colleagues to respond in writing to the 

journal, but chose not to do so "until the SUNY process has run its course."22

(The State University of New York, Albany, investigated the accusation of 

fraud against Wang, which it rejected23). Peizer has included some detail of 

email traffic on this issue in his submission to us.24

37. The allegation against Jones that is relevant to the Review25 is that since 

significant doubts about the reliability of Chinese climate data were raised, Jones 

has taken no public steps to clear up the discrepancies regarding Wang's claims 

and data, and that “it is unacceptable that the scientist who disseminates a data 

product on which international treaties are based, as well as IPCC reports and 

countless government policies, should actively seek to suppress information that 

calls the quality of the data into question”.

38. We note that, in at least a Nature interview referring to the loss of records26, Jones 

is reported as acknowledging “It‟s not acceptable...not best practice ...the stations 

„probably did move‟...I thought it was the right way to get the data. I was 

specifically trying to get more rural station data that wasn‟t routinely available in 

real time from [meteorological] services.” Although Jones chose not to respond in 

 

17 Jones P.D., Groisman P.Y., Coughlan M., Plummer N., Wang W.-C., Karl T.R. (1990), ―Assessment 

of urbanization effects in time series of surface air temperature over land‖, Nature, 347: 169–172. 

18 Wang W.-C., Zeng Z., Karl T.R. (1990), ―Urban heat islands in China‖, Geophysical Research 

Letters, 17: 2377–2380.

19 Keenan, D.J. Fraud allegation against some climatic research of Wei-Chyung Wang. Energy and 

Environment, 18 (7-8), 985-995. 

20 Jones, P.D., Lister, D.H. and Li, Q. 2008. Urbanization effects in large-scale temperature records, 

with an emphasis on China. Journal of Geophysical Research, 113 (D16122), 12pp. 

21 Ibid. 16 

22 Ibid. 16 

23 http://www.informath.org/apprise/a5620.htm. This site included a letter to Keenan from the State 

University of New York, Albany, that states that ―The Investigation Committee finds no evidence of 

the alleged fabrication of results and nothing that rises to the level of research misconduct having been 

committed by Dr. Wang.‖ We are not aware of a published version of this report. 

24 Ibid. 16 

25 Ibid. 16 

26 Nature News, published online 15 February 2010 

http://www.nature.com/news/2010/100215/full/news.2010.71.html 
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA 

53

writing when Keenan‟s claim was made, he did respond through an analysis in a 

paper in a front rank peer-reviewed journal only one year later that verified the 

conclusions of the earlier work27. By that time the SUNY investigation had 

concluded in Wang‟s favour.

6.7 Conclusions and Recommendations 

39. In summary, with regard to the allegations concerning the temperature data, the 

conclusions of the Review Team are as follows: 

Regarding data availability, there is no basis for the allegations that CRU 

prevented access to raw data. It was impossible for them to have done so. 

Regarding data adjustments, there is no basis for the allegation that CRU made 

adjustments to the data which had any significant effect upon global averages 

and through this fabricated evidence for recent warming. 

We find that CRU was unhelpful in dealing with requests for information to 

enable detailed replication of the CRUTEM analysis. 

Crucially, we find nothing in the behaviour on the part of CRU scientists that 

is the subject of the allegations dealt with in this Chapter to undermine the 

validity of their work. 

40. Reflecting the analysis in Appendix 7, the Review has the following 

recommendations: 

It would benefit the global climate research community if a standardised way 

of defining station metadata and station data could be agreed, preferably 

through a standards body, or perhaps the WMO. We understand that this is not 

straightforward and could be a lengthy process, but the process should start 

now. As example an xml based format would make the interpretation use, 

comparison, and exchange of data much more straightforward. 

Without such standardisation there will always be problems in issuing 

unambiguous lists, and assembling primary data from them. It would be in the 

public interest if CRU and other such groups developed a standard process to 

capture and publish a snapshot of the data used for each important publication.

 

 

27 Ibid. 20 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

54

CHAPTER 7: TEMPERATURE RECONSTRUCTIONS 

FROM TREE RING ANALYSIS

7.1 Background 

1. This Chapter considers the criticisms that have been made of CRU scientists in 

relation to their work on obtaining and analysing tree ring data to reconstruct land 

temperature records back over thousands of years, and in relation to the use made 

of that work in the IPCC. 

2. Data is recorded from tree core samples from many sites around the globe. These 

data comprise measurements of width and density taken from the cores and do not 

in themselves represent temperature directly. The processes of assembling and 

standardising the tree data, into what are termed tree ring chronologies, are 

separate from the later stage of producing temperature reconstructions. 

3. Temperature reconstructions are obtained through regression models which are 

calibrated against instrument temperature data during periods of overlap. A 

temperature reconstruction from any specific region does not in itself indicate 

global temperature patterns. Tree and other proxy reconstructions are combined 

to produce reconstructions pertaining to a wider area, and such reconstructions 

from many authors may be aggregated in order to provide an indication of global 

patterns. Even so the coverage is not as extensive as that of instruments. 

4. The uncertainty associated with temperature reconstructions in the distant past is 

much larger than that of present day instrument records. There is a wide and 

healthy debate in the published literature on all aspects of this. 

5. CRU scientists have developed temperature reconstructions based on a range of 

tree ring series. The current leader in CRU is Professor Briffa. Their work is 

extensively described in the relevant scientific literature. It also contributes to the 

development of a global picture such as that produced by IPCC in its 4th

Assessment Report (Working Group 1, Chapter 6, Palaeoclimate). In particular 

this contained conclusions on the likelihood that the second half of the 20th

Century was the warmest 50 year period in the last 500 or 1300 years in the 

Northern Hemisphere (different statements of probability are made for each – see 

paragraph 20).

7.2 The Allegations 

6. The release of the e-mails triggered the expression of a range of criticisms (many 

of long standing) of the work of CRU scientists. There was a clear implication 

that in carrying out their work, both the choices they made of data and the way it 

was handled, were intended to bias the scientific conclusions towards a specific 

result and to set aside inconvenient evidence. The overall implication is that this 

work is misleading and should not be trusted. 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

55

7. We address the following specific allegations which were explicit or can be 

inferred from the comments made on the e-mails and from the submissions we 

received. 

8. In relation to IPCC:

That the CRU work was ―flawed‖ and yet had a distinct influence on the 

balance of judgements made by IPCC Working Group 1 in Chapter 6 of their 

4

th Assessment Report, and therefore that less confidence should be ascribed 

to the conclusions reached by the authors, mentioned in paragraph 5 above. 

The criticism here is often captured by the proposition that today‘s 

temperatures are not unusual compared to the MWP. 

That Yamal and other chronologies constructed by CRU are unrepresentative 

of temperature trends (in recent years), and had an undue influence on all of 

the lines appearing in Chapter 6 of the 4th IPCC Report. 

That a majority of the reconstructions would look significantly different if 

certain component series were replaced with others, and that if this were done 

then the conclusions reached in respect of the likelihood associated with 

ranking of recent warmth with respect to the past would be significantly 

different. 

9. In relation to divergence:

That the phenomenon of ―divergence‖ (see discussion below) may not have 

been properly taken into account when expressing the uncertainty associated 

with reconstructions; and that this was not addressed with sufficient openness 

in IPCC reports. 

That the reference in a specific e-mail to a ―trick‖ and to ―hide the decline‖ in 

respect of a 1999 WMO report figure show evidence of intent to paint a 

misleading picture. 

10. In relation to withholding data:

That access to tree ring data from the Yamal peninsula in Siberia (used by 

Professor Briffa) was withheld, and that as a particular consequence of this the 

small sample size of the Yamal series in the most recent years was not 

acknowledged. 

11. In relation to mishandling data:

That improper manipulations have been performed – specifically with respect 

to the Yamal and Tornetrask tree series. 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

56

7.3 Findings 

7.3.1 IPCC Reports 

12. The IPCC reports are at the centre of allegations of presenting misleading 

messages produced by CRU. We concentrate upon AR4 as it is the most recent 

and most comprehensive. 

13. On the replacement of series in reconstructions, much of the critical material 

submitted presents several examples of differing results obtained from a tree 

series (A) compared to another tree series (B) in the same region. This is to be 

expected, and taken in isolation has no implication for the validity of either. To 

know whether replacing one with another would have a significant effect upon 

statements regarding Northern Hemisphere temperature series, would require the 

fulfilment of several conditions: 

That it has been proven that series B is more representative over the whole 

period than series A. 

That the difference between series A and B is important in a temperature 

reconstruction in which it is included.

That if A were replaced with B in each of the reconstructions in which A is 

included then the conclusions drawn in respect of the likely ranking of the 

past to the present would change significantly when considering all 

reconstructions together (including those in which it does not appear). 

To support an allegation of improper conduct would require that the scientists 

involved, knowing the above, still use and promote non-representative series A in 

order to reach improperly a false conclusion. 

14. Finding: We are unaware of any analysis to demonstrate that any of the above 

conditions are fulfilled for Yamal or any of the series cited in relation to CRU 

work (i.e. Tornetrask, Taymir). The Review is naturally aware that partial studies 

and comments referring to CRU‘s published work appear elsewhere. However 

these criticisms of CRU‘s work are not in peer reviewed journals, and we have not 

found that these are anywhere assembled into a coherent, comprehensive and 

scrutinised case which demonstrates the proposition in respect of any of the series 

cited. 

15. Finding: To make the case that replacing series in reconstructions calls in question 

the validity of the picture painted in Chapter 6 of the IPCC report (or to sustain a 

charge of impropriety on the part of its many authors) would require that all the 

conditions in paragraph 13 were met. No evidence of this has been presented to 

the Review. 

16. Finding: The influence of the Yamal series of Briffa 20001

 is often called into 

question. The conditions in paragraph 13 above have not been demonstrated 

 

1

 Briffa, K. R. 2000. Annual climate variability in the Holocene: interpreting the message of ancient 

trees. Quaternary Science Reviews 19:87-105
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

57

anywhere which has been brought to our attention. This series is used in none of 

the IPCC TAR and only 42out of 12 reconstructions in AR4.

17. We now turn to the question of uncertainty. There are multiple sources of 

uncertainty in respect of proxy temperature reconstructions and they are the 

subject of an ongoing and open scientific debate. 

18. Elements of that debate are touched on in the e-mails. For example, the following 

e-mails from Briffa show some concern to ensure that uncertainty was properly 

recognised. 

E-mail dated 29/4/03 (1051638938.txt): 

“Can I just say that I am not in the MBH camp - if that be characterized by an 

unshakable "belief" one way or the other , regarding the absolute magnitude 

of the global MWP. I certainly believe the " medieval" period was warmer 

than the 18th century - the equivalence of the warmth in the post 1900 period, 

and the post 1980s, compared to the circa Medieval times is very much still an 

area for much better resolution. I think that the geographic / seasonal biases 

and dating/response time issues still cloud the picture of when and how warm 

the Medieval period was . On present evidence , even with such uncertainties I 

would still come out favouring the "likely unprecedented recent warmth" 

opinion - but our motivation is to further explore the degree of certainty in this 

belief - based on the realistic interpretation of available data.”

E-mail dated 3/2/06 (1138995069.txt): 

“we are having trouble to express the real message of the reconstructions - 

being scientifically sound in representing uncertainty , while still getting the 

crux of the information across clearly...We have settled on this version 

(attached) of the Figure which we hoe [sic] you will agree gets the message 

over but with the rigor required for such an important document.”

19. What is clear is that the uncertainty associated with any estimate of past 

temperatures from reconstructions is much larger than that of recent instrument 

temperature data. This is demonstrated in the figure below taken from IPCC AR4 

Figure 6.10. The middle plot shows the variability both within and between 

different reconstructions each using an aggregation of proxy data (i.e. several tree 

and non-tree proxies). The lower plot gives an indication of uncertainty bands 

from those reconstructions. Simply looking at any individual reconstruction line 

alone makes only a partial statement about Northern Hemisphere temperatures 

with a large uncertainty. It is obviously even less meaningful to look at an 

individual tree series in isolation. 

 

2

 A fifth reconstruction is disputed. 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

58

Figure 7.1: From IPCC AR4 (Figure 6.10) 

20. Understanding requires proper statistical interpretation, i.e. to determine the 

confidence level associated with a statement such as “the present is likely warmer 

than the past”. To do this as objectively as possible would require a complex (and 

difficult) study to perform hypothesis testing in a mathematically rigorous way, 

taking proper account all of the uncertainties and their correlations. We are not 

aware that this has been done in the production of IPCC reports to date, but 

instead qualitative statements have been made based on definitions of ―likely‖, 

―very likely‖ etc according to criteria laid down by the IPCC ( ̳Likely‘ means a 

probability greater than 66%, and  ̳Very Likely‘ means a probability greater than 

90%). 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

59

21. Finding: We do not find that the data described in IPCC AR4 and shown in Figure 

6.10 is misleading, and we do not find that the question marks placed over the 

CRU scientists‘ input casts doubt on the conclusions. In particular: 

The variation within and between lines, as well as the depiction of 

uncertainty is quite apparent to any reader. 

It presents all relevant published reconstructions we are aware of, i.e. there 

has been no exclusion of other published temperature reconstructions 

which would show a very different picture. 

The general discussion of sources of uncertainty in the text is extensive, 

including reference to divergence and it therefore cannot be said that 

anything has been suppressed. Presenting uncertainty in this way is a 

significant advance on the TAR. 

7.3.2 Divergence 

22. Divergence: The phenomenon of divergence is an ongoing research area which is 

discussed widely in the literature and the submissions. Tree core data yields both 

width and density measurements and both of these are used to reconstruct 

temperature records. In some cases the temperature reconstruction from one or 

other of these will track instrumental temperature measurements on a short 

timescale, but diverges on a longer timescale. The phenomenon of divergence is 

not observed in all tree series. There is as yet no complete understanding of the 

source of divergence although studies are suggesting possible contributory effects. 

23. Finding: The question of whether divergence has been adequately taken into 

account when estimating uncertainties is a complex and ongoing question of 

science and is therefore outside the scope of this Review. We have however 

investigated this matter enough to be satisfied that it is not hidden and that the 

subject is openly and extensively discussed in the literature, including CRU 

papers. 

24. The WMO 1999 front cover figure: The allegation is that the figure appearing on 

the front cover of the WMO report of 1999 (Statement on the Status of Global 

Climate, 19993

) did not disclose that it combined proxy and temperature records 

and failed to show a declining proxy line due to divergence; and that this was 

done to deceive. 

Jones‘ e-mail on 16/11/99 (942777075.txt) 

“I've just completed Mike's Nature trick of adding in the real temps to each series 

for the last 20 years (ie from 1981 onwards) and from 1961 for Keith's to hide the 

decline.”

is perhaps the prime example of the fact that opposing interpretations can be 

obtained from the same statement. The word ―trick‖ has been widely taken to 

 

3

 http://www.wmo.ch/pages/prog/wcp/wcdmp/statement/wmo913.pdf 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

60

confirm the intention to deceive, but can equally well, when used by scientists, 

mean for example a mathematical approach brought to bear to solve a problem. It 

is the latter explanation that Jones has given the Review, quoted in paragraph 6 of 

Chapter 4. 

25. The WMO report is a short document produced annually. It does not have the 

status or importance of the IPCC reports. The figure in question was a frontispiece 

and there is no major discussion or emphasis on it in the text. The caption of the 

figure states: ―Northern Hemisphere temperatures were reconstructed for the past 

1000 years (up to 1999) using palaeoclimatic records (tree rings, corals, ice 

cores, lake sediments, etc.), along with historical and long instrumental records”.

26. Finding: In relation to ―hide the decline‖ we find that, given its subsequent iconic 

significance (not least the use of a similar figure in the TAR), the figure supplied 

for the WMO Report was misleading in not describing that one of the series was 

truncated post 1960 for the figure, and in not being clear on the fact that proxy and 

instrumental data were spliced together. We do not find that it is misleading to 

curtail reconstructions at some point per se, or to splice data, but we believe that 

both of these procedures should have been made plain – ideally in the figure but 

certainly clearly described in either the caption or the text. 

7.3.3 Withholding Data

27. Yamal - withholding data and information on sample size: This issue concerns 

the Briffa 20004 and Briffa 20085 publications. The underlying raw data were not 

archived at the time of publication. Since it was not owned by CRU, they had no 

right to archive it. The data were owned by others including Hantemirov and 

Shiyatov. CRU has stated that it directed the request it received for data to the 

owners. Whether it was a result of this, or otherwise, the requester was given a 

copy

6

 of the data in 2004 by Hantemirov. Later, following publication of Briffa 

20087

, and again following requests for the data, CRU has stated that it asked the 

owners to archive the data, which they did. 

28. Because the raw data was not archived, the information on the number of core 

counts used in each year was not easily available. As a result the reader could not 

know that the core count in the most recent years had dropped substantially. This 

fact grew in importance as the need to understand uncertainty grew. The 

information was actually published and in the public domain in 20028

 but it is 

arguable that a user of the CRU Yamal chronology would not have known to look 

 

4

 Ibid. 1 

5

 Briffa, K. R., Shishov, V.V., Melvin, T. M., Vaganov, E. A., Grudd, H., Hantemirov, R. M., Eronen, 

M., and Naurzbaev, M. M. 2008. Trends in recent temperature and recent tree growth spanning 2000 

years across Northwest Eurasia. Philosophical Transactions of the Royal Society B-Biological Sciences 

363:2271-2284 

6

 It has also been pointed out to us by CRU that McIntyre acknowledged this in a comment (number 

61) made underneath a Climate Audit post which states that McIntyre did not realize that this was the 

series used by Briffa when he received it. http://climateaudit.org/2009/10/05/yamal-and-ipcc-ar4-

review-comments/. 

7

 Ibid. 5 

8

 Hantemirov, R. M., and Shiyatov, S. G. 2002. A continuous multimillenial ring-width chronology in 

Yamal, northwestern Siberia. Holocene 12:717-726 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

61

there unless explicitly directed. Hantemirov and Shiyatov 2002 was referenced in 

Briffa 20089.

29. Finding: It is evidently true that access to the raw data was not simple until it was 

archived in 2009 and this can rightly be criticized on general principles of 

transparency, although it may have been common practice at the time of the 

original publication. We find that CRU has not withheld the data (having correctly 

directed the single request to the owners). However, in the interests of 

transparency, we believe that CRU should have ensured that the data it did not 

own, but had relied upon in publications, was archived in a more timely way. 

This is another example where a more open approach would have been helpful. 

7.3.4 Mishandling Data 

30. We have not focussed upon disagreements over comparisons of results using 

individual tree series for two reasons. First, because this is a science question and 

should be addressed through the normal channels of peer reviewed publication. 

Secondly, because as we have indicated above, it requires much more work to 

show that one is unequivocally more representative than the other, and even then 

requires the conjunction of several additional conditions to lead to an allegation of 

wrongdoing. We have addressed these conditions above, and we have found no 

evidence to substantiate them. Nevertheless we comment briefly upon Yamal as it 

has received so much attention and the Tornetrask series as it is subject of much 

misunderstanding. 

31. Finding on Yamal: The Briffa 2000

10 paper presents a collection of many 

chronologies (Yamal is simply one of them) assembled from many authors. It is

quite clear that Briffa simply reprocessed data from other authors (Hantemirov 

and Shiyatov in this case) and no selection was performed. In their submission to 

the Review CRU included a copy of work they had made available on the web11. 

This was posted in response to criticisms published at Climate Audit12 which 

described examples of variability between specific tree series in the region. This 

CRU response demonstrates the consistency of the original Yamal publication 

with that obtained using an updated analysis incorporating a comprehensive set of 

contemporary data. The Review is not aware of any equivalent comprehensive 

analysis which demonstrates that the conclusions of Briffa 2000 are inconsistent 

with the best analysis available today. 

32. Finding on ―Bodging‖ in respect of Tornetrask. The term ―bodging‖ has been 

used, including by Briffa himself, to refer to a procedure he adopted in 199213. 

The  ̳bodge‘ refers to the upward adjustment of the low-frequency behaviour of 

the density signal after 1750, to make it agree with the width signal. This ad hoc 

process was based on the conjecture that the width signal was correct. There is 

nothing whatsoever underhand or unusual with this type of procedure, and it was 

 

9

 Ibid. 5 

10 Ibid. 1 

11 http://www.cru.uea.ac.uk/cru/people/briffa/yamal2009/ 

12 http://climateaudit.org/2009/09/27/yamal-a-divergence-problem/ 

13 Briffa, K. R., Jones, P. D., Bartholin, T. S., Eckstein, D., Schweingruber, F. H., Karlen, W., 

Zetterberg, P., and Eronen, M. 1992. Fennoscandian Summers from AD 500: temperature changes on 

long and short timescales. Climate Dynamics 7:111-119. 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

62

fully described in the paper. The interpretation of the results is simply subject to 

this caveat. The conjecture was later validated14 when it was shown to be an effect 

due to the standardisation technique adopted in 1992. Briffa referred to it as a 

―bodge‖ in a private e-mail in the way that many researchers might have done 

when corresponding with colleagues. We find it unreasonable that this issue, 

pertaining to a publication in 1992, should continue to be misrepresented widely 

to imply some sort of wrongdoing or sloppy science. 

7.4 Conclusions and recommendations 

33. We do not find that the data described in AR4 and shown in Figure 6.10 is 

misleading, and in particular we do not find that the question marks placed 

over the CRU scientists’ input cast doubt on the conclusions. 

34. The variation within and between lines, as well as the depiction of uncertainty is 

quite apparent to any reader. All relevant published reconstructions of which we 

are aware are presented, and we find no evidence of exclusion of other published 

temperature reconstructions which would show a very different picture. The 

general discussion of sources of uncertainty in the text is extensive, including 

reference to divergence and it therefore cannot be said that that anything has been 

suppressed. Presenting uncertainty in this way is a significant advance on the 

TAR.

35. We have seen no evidence to sustain a charge of impropriety on the part of CRU 

staff (or the many other authors) in respect of selecting the reconstructions in AR4 

Chapter 6. This would require that all the conditions in paragraph 13 were met in 

respect of tree chronologies either used by, or created by, CRU. No evidence of 

this has either been presented to the Review, nor has it been assembled as a 

scientific study published elsewhere and subjected to scrutiny. For the same 

reasons we found no evidence that there is anything wrong with the CRU 

publications using the Yamal or other tree series. 

36. We find that divergence is well acknowledged in the literature, including CRU 

papers. 

37. In relation to ―hide the decline‖ we find that, given its subsequent iconic 

significance (not least the use of a similar figure in the IPCC TAR), the figure 

supplied for the WMO Report was misleading in two regards. It did not make 

clear that in one case the data post 1960 was excluded, and it was not explicit on 

the fact that proxy and instrumental data were spliced together. We do not find 

that it is misleading to curtail reconstructions at some point per se, but that the 

reason for doing so should have been described. 

38. We find that CRU has not withheld the underlying raw Yamal data (having 

correctly directed the single request to the owners). But it is evidently true that 

access to the raw data was not simple until it was archived in 2009 and this can 

rightly be criticised on general principles. In the interests of transparency, we 

believe CRU should have ensured that the data they did not own, but had relied 

upon in publications, was archived in a more timely way. 

 

14 Briffa KR and Melvin TM (2010 in press) 
CHAPTER 7: TEMPERATURE RECONSTRUCTIONS FROM TREE RING ANALYSIS 

63

39. It is a matter for the IPCC Review to determine whether the conclusions were in 

line with IPCC processes and guidelines for levels of likelihood. In respect of that 

Review we offer the suggestion that putting the combination of different 

reconstructions upon a more rigorous statistical footing would help in the future to 

make confidence levels more objective. 
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

64

CHAPTER 8: PEER REVIEW AND INFLUENCING 

EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

8.1 Background: Peer Review, Testing and Verification 

1. The rigorous development of scientific ideas leans heavily on peer-reviewed 

publication in scientific journals that are subject to testing and challenge by other, 

published, scientifically derived evidence. 

2. The processes whereby new scientific concepts are peer-reviewed and published 

can be complex, contentious and partial (see Chapter 5 and Appendix 5). Peer 

review is not a ―gold standard‖ that ensures validity, as some claim. The processes 

of testing, validation and replication are the means whereby error is discarded, 

uncertainty reduced and robust scientific progress made. Peer review attempts 

initial sieving-out of the demonstrably erroneous or trivial, thereby setting a high 

standard that encourages rigour. But it is not infallible. Many well-founded 

concepts are rejected and many erroneous ideas accepted. 

3. If peer review is subverted to exclude apparently well-founded alternative views, 

or if journal editors are intimidated from considering their publication, progress on 

an important issue such as climate change can be seriously slowed or skewed. 

8.2 The Allegations 

4. The broad allegation is that CRU made improper attempts to influence the peer 

review system, pressuring journals to reject submitted articles that did not support 

a particular view of climate change. 

5. This can only be properly assessed by analysis of the individual cases cited. 

Specific examples used to support this allegation are the events surrounding the 

publication of a paper by Soon and Baliunas (2003)1, the issues relating to Dr 

Boehmer-Christiansen, editor of the journal Energy and Environment, and certain 

actions of Professor Briffa when he was editor of the Journal Holocene.

8.3 The Soon and Baliunas Affair & Climate Research

6. The paper by Soon and Baliunas (2003) entitled Proxy climatic and environmental 

changes of the past 1000 years, published in the journal Climate Research, 

reviewed 240 previously published papers on temperature trends during the last 

millennium. It challenged the conclusion of Mann et al (1998, 1999)2 that the late 

20th century was the warmest period of the last millennium on a hemispheric 

scale, and claimed that recent temperatures were by no means unprecedented over 

 

1

 Soon, W. and Baliunas, S. 2003. Proxy climatic and environmental changes of the past 1000 years. 

Climate Research 23, 89-110.

2 Mann ME, Bradley RS, Hughes MK (1998) Global-scale temperature patterns and climate forcing 

over the past six centuries. Nature 392:779–787. 

Mann ME, Bradley RS, Hughes MK (1999) Northern Hemisphere temperatures during the past 

millennium: inferences, uncertainties, and limitations. Geophys Res Lett, 26:759 – 762.
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

65

this period. It was greeted with enthusiasm by those sceptical of the hypothesis of 

anthropogenic global warming3. However it received a negative reception from 

many other climate scientists4 on scientific grounds, viz. that it conflated 

qualitative data on temperature and precipitation from many sources that could not 

be combined into a consistent proxy record. That hostility is reflected in the 

released CRU e-mails (e.g. CRU 1051156418.txt, 1051202354.txt), and the 

following e-mail from Jones to Mann on 11.3.03 (1047388489.txt): 

“I think the skeptics will use this paper to their own ends and it will set paleo back 

a number of years if it goes unchallenged. I will be emailing the journal to tell 

them I'm having nothing more to do with it until they rid themselves of this 

troublesome editor (de Freitas), a well-known skeptic in NZ. A CRU person is on 

the board but papers get dealt with by the editor assigned by Hans von Storch." 

7. The S&B paper had been seen by four reviewers, none of whom had 

recommended rejection5, and had been accepted by de Freitas (one of 10 Climate 

Research review editors; papers could be submitted to any one of them). A 

number of review editors resigned as a reaction against the publication of what 

they regarded as a seriously flawed paper. The journal‘s publisher admitted that 

the journal should have requested appropriate revisions of the manuscript prior to 

publication6

. The Editor in Chief resigned on being refused permission by the 

publisher to write an editorial about what he regarded as a failure of the peer 

review system. De Freitas was said to have described these events as ―a mix of a 

witch-hunt and the Spanish Inquisition‖7

.

8. These events, and the e-mail quote in paragraph 6, have led to the allegation that 

normal procedures of publications were being improperly undermined by a group 

that included Jones8.

9. Jones has responded in evidence9 to us that the reaction to the S&B paper was not 

improper or disproportionate given what he saw as the self evident errors of the 

paper. The arguments presented in the Eos article, if correct, are strongly put, and 

suggest that the reaction was based on a belief, for which evidence was adduced, 

that the science was poor. In light of the reaction of the Journal‘s publisher, we do 

not believe that any criticism of Jones can be justified in this regard. 

10. Finding: This was clearly a bruising experience for all concerned. But Richard 

Horton‘s paper (Appendix 5) and the comments on it in Chapter 5 suggest to the 

Team that this scale of reaction is not unusual in contested areas, and the peer 

review process does not provide insulation from it. The Review makes no 

judgement about the correctness or otherwise of the Soon and Baliunas paper, but 

we conclude that the strong reaction to it was understandable, and did not amount 

to undue pressure on Climate Research. 

 

3 New York Times, 5 August 2003. 

4

 e.g. Eos, Vol. 84, No. 27, 8 July 2003.

5

 http://www.sgr.org.uk/climate/StormyTimes_NL28 htm 

6

 A statement from Inter-Research, 2003. Climate Research, 24, 197-198

7

 http://www.sgr.org.uk/climate/StormyTimes_NL28 htm 

8

 Montford, A.W. 2010. The Hockey Stick Illusion. Stacey International, London. 

9

 See the note of the CCER meetings held on 9 April 2010 
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

66

8.4 The Conflict with Dr Boehmer-Christiansen 

11. Dr Boehmer-Christiansen is Reader Emeritus in Geography at the University of 

Hull and the editor of the scientific journal Energy and Environment. As part of 

her evidence to us, she submitted a copy of her evidence to the House of 

Commons Science and Technology Committee Select Committee for its inquiry 

into ―The disclosure of climate data from CRU at the UEA‖. In it she wrote10

: 

 

“As editor of a journal which remained open to scientists who challenged the 

orthodoxy, I became the target of a number of CRU manoeuvres. The hacked 

emails revealed attempts to manipulate peer review to E&E‟s disadvantage, and 

showed that libel threats were considered against its editorial team. Dr Jones 

even tried to put pressure on my university department. The emailers expressed 

anger over my publication of several papers that questioned the „hockey stick‟ 

graph and the reliability of CRU temperature data. The desire to control the peer 

review process in their favour is expressed several times”.

12. Apart from the allegation of  ̳attempts to manipulate peer review‘ for which 

Boehmer-Christiansen presents no evidence in relation to the journal that she 

edits, her statement11 to us implies that pressure on her University was designed to 

undermine her role as editor of Energy and Environment. She comments that ―a 

message was send to my head of department late last year by Phil Jones in 

relation to some other matter, which suggested that the University of Hull should 

dissociate itself from me as editor of Energy & Environment (Multi-Science) as I 

was causing difficulties for CRU.‖ 

13. This episode appears to have been precipitated by an email sent on 2 October 

2009 (1254746802.txt) from Boehmer-Christiansen to Stephanie Ferguson at the 

UK Climate Impacts Programme, and copied to a number of others. Its title was: 

“Please take note of potetially [sic] serious scientific fraud by CRU and Met 

Office.” The email was brought to the attention of Jones by the ―help desk‖ of 

UKCP09 in the UK Department for Environment, Food and Rural Affairs 

(DEFRA). He then wrote an e-mail on 27 October 2009 (1256765544.txt) to 

Professor Graham Haughton, Boehmer-Christiansen‘s Head of Department, 

complaining about the e-mail as ―very malicious”. Haughton responded 

sympathetically, but commented that “I‟d want to protect another academic‟s 

freedom to be contrary and critical”, to which Jones replied: “I don‟t think there 

is anything more you can do. I have vented my frustration and have had a 

considered reply from you”. 

14. Finding: We see nothing in these exchanges or in Boehmer-Christiansen‘s 

evidence that supports any allegation that CRU has directly and improperly 

attempted to influence the journal that she edits. Jones‘ response to her accusation 

of scientific fraud was appropriate, measured and restrained. 

 

10 Memorandum submitted by Dr Sonja Boehmer-Christiansen (CRU 26), at 4.1 

11 Dr Sonja Boehmer-Christiansen submission (no.43), 23 February 
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

67

8.5 Peer Review and Professor Briffa‟s Editorship of 

Holocene

15. An e-mail (1054748574.txt) was sent by Briffa, as Editor of the journal Holocene, 

on 4.06.03 to Ed Cook, a reviewer, as follows: 

“I am really sorry but I have to nag about that review – Confidentially I now need 

a hard and if required an extensive case for rejecting – to support Dave Stahle‟s 

and really as soon as you can.” 

This has been assumed to be an editor who wishes to have the paper rejected, for 

reasons that are not explicit, seeking help from a reviewer to do so. In general, 

reviewers are not, or should not be invited to reject, merely to review. This e-mail 

has been widely interpreted as Briffa perverting the purpose of peer review by 

asking a colleague for help in rejecting a paper that contained research findings 

contradictory to his own views12. 

16. However, put in context, the e-mail‘s significance changes. The reality is reflected 

in a series of emails provided by Briffa13 and which have been redacted to remove 

names, apart from that of Stahle, as follows: 

17.06.02 – Briffa requests references from Dr Dave Stahle and referee B for a 

paper submitted to Holocene on the use of a specific tree ring record. 

07.08.02 – Stahle submits a referee report to Briffa. It is strongly critical of 

some of the underlying analysis, though it comments that the paper is well 

written. Advises rejection but suggests that if substantial additional work were 

done, it could be suitable for publication. 

09.08.02 – Briffa to Stahle, thanking him for the review, and committing 

himself to reading the paper very carefully in anticipation of receipt of a 

second review. 

28.05.03 – Briffa to referee B, pressing to send a second review. 

04.06.03 – Briffa reiterates his request to referee B, sending the email copied 

in paragraph 15.

04.06.03 – Referee B sends review to Briffa. In the event, the review is not as 

negative as that of Stahle: the paper is ―marginal at best, could justifiably be 

rejected‖. This is an immediate reply to the email copied in paragraph 15.

23.07.03 – One of the authors e-mails to Briffa enquiring about a decision on 

the paper. 

24.07.03 – Briffa to author. Apologises for the delay (―the manuscript could 

have been dealt with much better‖) and encloses referees‘ comments. Offers 

possible fast-track publication if the referees comments could be dealt with. 

We can find no evidence that the article was subsequently published but the 

evidence above demonstrates that the possibility of publication was not 

rejected. 

 

12 e.g. Channel 4 News 3.2.10 

13 Copies of Communications relating to Keith Briffa‟s editorial treatment of a submitted manuscript. 

Available at www.cce-review.org/Evidence.php 
CHAPTER 8: PEER REVIEW AND INFLUENCING EDITORIAL POLICY OF SCIENTIFIC JOURNALS 

68

17. Finding: Although much has been made of the e-mail in paragraph 15 as evidence 

of an unprincipled approach to the role of editor, we see nothing in these 

exchanges that supports the interpretations of subverting the peer review process 

that have been placed upon it. It appears to reflect an Editor with a strongly 

negative review in hand, and who presumably has read the paper, asking for 

confirmation that the paper should be rejected, possibly to reduce one of the many 

complications that assail an editor; and in view of the delay in communicating to 

authors, hoping for a strong decision from the referee. On receiving a second, 

more equivocal review, he offers the authors the opportunity to re-submit. These 

exchanges illustrate some of the complications of an Editor‘s life as referred to in 

Chapter 5 and in Appendix 5. They do not provide evidence of subversion of 

process in rejecting contradictory ideas as has been alleged. 

8.6 Conclusions 

18. In our judgement none of the above instances represents subversion of the peer 

review process nor unreasonable attempts to influence the editorial policy of 

journals. It might be thought that this reflects a pattern of behaviour that is partial 

and aggressive, but we think it more plausible that it reflects the rough and tumble 

of interaction in an area of science that has become heavily contested and where 

strongly opposed and aggressively expressed positions have been taken up on both 

sides. The evidence from an editor of a journal in an often strongly contested area 

such as medicine (Appendix 5) suggests that such instances are common and that 

they do not in general threaten the integrity of peer review or publication. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

69
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

70

CHAPTER 9: COMMUNICATING INTO THE PUBLIC 

DOMAIN THROUGH THE IPCC 

9.1 Background

1. Where scientific results and concepts are relevant to issues of public or policy 

interest, their complexities are often such that they need to be communicated in 

simpler, comprehensible language that includes explicit statements about 

uncertainties and errors. 

2. The IPCC process and its assessment reports have been the prime routes through 

which the complexities of climate change are transmitted into the international 

governmental domain. Actions that undermined the rigour, honesty and 

expressions of scientific uncertainty in such communications would be serious 

infringements of good scientific practice. 

9.2 The Allegations 

3. The above principles are the frame for allegations levelled at CRU that relate to 

the Review‘s remit. In broad terms the claim is that CRU attempted to prevent 

proper consideration of views which conflicted with their own through their roles 

in the IPCC. 

4. If this is the case it would represent a failure to represent current scientific 

understanding impartially at the vital interface between science and policy. If this 

has been done, it has the potential to understate uncertainty, and to undermine the 

rigour of risk-based evaluations by government, with the potential for severe 

social and economic consequences. 

5. There are two specific, analogous allegations. They primarily relate to papers that 

challenge important elements of IPCC work; the first about the interpretation of 

the CRUTEM instrumental temperature series and the second about the 

reconstruction of tree ring proxy temperature series, discussed earlier in different 

contexts in Chapters 6 and 7 respectively. It is alleged that these papers were not 

properly considered by the relevant IPCC writing groups, and that members of 

CRU played the major role in ensuring that this was not done. 

6. The following approach is adopted for each of these two sets of allegations: 

summary of the scientific basis of the challenge; 

allegations about the improper treatment of the papers by CRU members in 

their roles in IPCC writing groups; 

evidence in support of the allegations; 

the response of the CRU members alleged to have been primarily involved; 

evidence from the IPCC Review Editors (see Chapter 5, section 5.7) about the 

procedures of the relevant IPCC writing groups; 

findings of the Review. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

71

9.3 The CRUTEM Temperature Series 

9.3.1 The Scientific Challenge 

7. A paper by McKitrick & Michaels (2004, hereafter MM2004)1

argued that a large 

proportion of the measured late 20th century warming was a consequence of 

increased economic activity and that many meteorological measurement sites had 

become increasingly influenced by the warming effect of urbanisation around 

them, the so-called ―heat island effect‖, and that the CRUTEM data set was not 

adequately adjusted for such non-climatic effects. They argued that removing the 

effects of this socio-economic transition would reduce the trend of land surface 

temperatures from 0.27 deg C/decade to 0.11 deg C/decade and possibly to 0.06 

deg C/decade. If this claim were correct it would, for many, fundamentally 

undermine the argument that the increasing concentration of atmospheric carbon 

dioxide as a consequence of human activity was a major driver of climate change. 

9.3.2 The Allegations 

8. The allegation that CRU withheld their data from scrutiny and adjusted it without 

justification is dealt with in Chapter 6. Here we are concerned with the specific 

allegation that CRU improperly dismissed contrary views about the interpretation 

of the CRUTEM series and prevented its serious consideration by the IPCC. 

9.3.3 Evidence in Support of the Allegations 

9. The reaction of Jones, who was then one of two CLAs for Chapter 3 

(Observations: Surface and Atmospheric Climate Change) of the Working Group 

1 report of IPCC AR4, to the MM2004 paper is revealed in an e-mail 

(1089318616.txt) sent on 8th July 2004 to Mann, that included: “The other paper 

by MM (McKitrick & Michaels, 2004) is just garbage. ... I can‟t see either of 

these papers being in the next IPCC report. Kevin (Trenberth, the other 

coordinating lead author for ch. 3) and I will keep them out somehow – even if we 

have to redefine what the peer-review literature is!” It indicates clearly his 

awareness of the paper prior to any drafts of IPCC AR4, and his determination to 

exclude it from consideration by IPCC.

10. It has been suggested2 that this determination to exclude is reflected in the fact 

that MM2004 was not referred to in either the first order draft of Chapter 3 in 

August 2005 or in the second order draft of March 2006. Nor was a paper drawing 

similar conclusions by de Laat and Maurellis (2006)3 referred to. It has been 

suggested to us by McIntyre4 that it was Jones who kept reference to these papers 

out of the first and second order drafts, with the implication that this was done 

improperly to prevent incorporation of conclusions contrary to those held by the 

CRU group. 

 

1

 McKitrick, R. and Michaels, P.J. 2004. A test of corrections for extraneous signals in gridded surface 

temperature data. Climate Research, 26, 159-173.

2

 McKitrick submission (no.15), 26 February 

3

 de Laat, A.T.J. and Maurellis, A.N. 2006. Evidence for influence of anthropogenic surface processes 

on lower tropospheric and surface temperature trends. International Journal of Climatology, 26, 897-

913.

4

 McIntyre submission (no. 23), 2 March 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

72

11. The omission of reference to MM2004 was criticised by expert reviewer Vincent 

Gray for Chapter 3, but this was rejected by the IPCC writing team for this 

Chapter (see Chapter 5 paragraph 24) with the argument that large scale patterns 

of atmospheric circulation over the continents and stronger warming of the 

continents would produce patterns of warming such as those recorded by 

temperature stations5. 

12. The final draft of Chapter 3 did however include reference both to McKitrick & 

Michaels (2004) and de Laat and Maurellis (2006), in the following paragraph: 

“McKitrick and Michaels (2004) and De Laat and Maurellis (2006) attempted to 

demonstrate that geographical patterns of warming trends over land are strongly 

correlated with geographical patterns of industrial and socioeconomic 

development, implying that urbanisation and related land surface changes have 

caused much of the observed warming. However, the locations of greatest 

socioeconomic development are also those that have been most warmed by 

atmospheric circulation changes (Sections 3.2.2.7 and 3.6.4), which exhibit largescale coherence. Hence, the correlation of warming with industrial and 

socioeconomic development ceases to be statistically significant. In addition, 

observed warming has been, and transient greenhouse-induced warming is 

expected to be, greater over land than over the oceans (Chapter 10), owing to the 

smaller thermal capacity of the land.”

13. It has been surmised in a submission to the Review by McKitrick6

 that Jones 

wrote the above paragraph and bears responsibility for its inclusion. The same 

submission alleges that it represents a ―fabricated conclusion‖ or ―invented 

evidence‖ and that only the derivation of a p-value7 from a statistical test that is 

compatible with the claim of statistical insignificance would rebut this allegation 

of fabrication. It alleges that this is evidence of bias, and that after attempts to 

exclude evidence that conflicted with the preferred CRU interpretation of the 

CRUTEM data series from drafts 1 and 2, reasons were contrived in the published 

draft for the specific purpose of rejecting this evidence. If this were correct, such 

actions would appear to violate the principles in Chapter 5, of the duty of 

scientists to ensure that uncertainties are clearly transmitted to those that have the 

responsibility for deciding on any contingent actions. 

 

5 Gray: “The „corrections‟ to the surface temperature record have always been based on very poor 

evidence. The many references to studies on individual or regional stations which find the need for 

much higher corrections than are currently applied, are ignored. Now you have ignored the persuasive 

evidence of McKitrick and Michaels 2004 Climatic Research 26 156-173 who have shown a significant 

influence on your „corrected‟ figures of a series of socioeconomic factors. You cannot just ignore this 

paper.”

Response of the IPCC writing team: “References are plentiful. Those of value are cited. Rejected.

The locations of socioeconomic development happen to have coincided with maximum warming, not 

for the reasons given by McKitrick and Michaels (2004) but because of the strengthening of the Arctic 

Oscillation and the greater sensitivity of land than ocean to greenhouse forcing owing to the smaller 

thermal capacity of land. Parker (2005) demonstrates lack of urban influences.”

http://pds.lib harvard.edu/pds/view/7795947?n=7&s=4&imagesize=1200&jp2Res=.25&rotation=0 

(comment 3-34). 

6

 McKitrick submission (no.15), 26 February 

7

 The p-value in statistics is the probability of obtaining a statistic at least as extreme as the one 

actually observed, assuming that there is no relationship between the measured phenomena. The lower

the p-value, the less likely that there is no relationship. McKitrick and Michaels (2004) obtained a pvalue of 0.002, suggesting a significant relationship between climatic and socio-economic trends. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

73

14. It has been further alleged8 that the changing response to the McKitrick and 

Michaels (2004) paper between the response to Gray and the statement in 

paragraph 13 of AR4, both assumed to have been by Jones, is evidence of a 

preparedness to accept any argument, irrespective of how well founded, that 

would refute conclusions to which he was opposed. 

9.3.4 Jones‟ Response 

15. These allegations were put to Jones. We summarise his oral and written responses 

as follows9,10: 

Jones comments that the e-mail of July 8, 2004 was sent on the spur of the 

moment and quickly forgotten. No pattern of behaviour with respect to his 

IPCC work could be construed from this one email. 

The reason for the strong response of the email and the justification for not 

including reference MM2004 in the early drafts is that it can be readily shown 

to be scientifically flawed. 

The basis for this statement is that if the CRUTEM3 trend is reduced by the 

factor claimed by MM2004, the land-based record then becomes incompatible 

with the ocean and the satellite record. MM2004 make no mention of this in 

their paper. In writing Chapter 3 of AR4 the author team were mindful of this. 

MM2004‘s analysis of the land surface temperature record is completely at 

odds with the rest of the surface and lower tropospheric temperature records. 

MM2004 also fails to take into account the effects of changes in the 

atmospheric circulation. 

In summary, the atmospheric circulation has been shown, in numerous studies, 

to account for large scale patterns of temperature change. Before undertaking 

the kind of analysis in MM2004, it is essential to account for known signals 

(i.e. the North Atlantic Oscillation (NAO)11 and El Nino Southern Oscillation 

(ENSO)12 and possibly others) and then examine the residuals. It does not 

make sense to calculate the p-value without allowing first for the atmospheric 

effects and their spatial autocorrelation in that calculation. 

In view of these arguments, it was reasonable to exclude MM2004, as the 

IPCC reports are assessments, not reviews of the literature. The author teams 

were chosen for their experience and expertise, and reliance was placed on this 

in determining which work should be included and discussed in the 

 

8

 McKitrick submission (no.15), 26 February 

9 Response from Professor Jones to questions from Professor Geoffrey Boulton (20 April) 

10 Clarification from Professor Jones to additional questions from Professor Geoffrey Boulton (3 May) 

11 The North Atlantic Oscillation (NAO) and the associated Arctic Oscillation are parts of the global 

wave structure in the northern hemisphere atmosphere. The Icelandic low and the Azores high are parts 

of this wave structure, and their east-west oscillations control the strength and direction of westerly 

winds and storm tracks across the North Atlantic. 

12 ENSO (El Niño-Southern Oscillation) is a 5-year (on average) oscillation between warm and 

upwelling cold waters in the tropical eastern Pacific, which has a major impact on atmospheric pressure 

in the region and knock-on effects on weather in many other parts of the world. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

74

assessment. 

Jones commented that the decision about which papers to include and which to 

exclude was a collective one of the author team of Chapter 3 of AR4. Jones 

stated that he did not write the relevant text in the final report as has been 

assumed: the lead on the relevant section (3.2) was another member of the 

writing team. The suggestion to make a response did not come from Jones, as 

he was not the responsible person for the section. He did, however, agree with 

its inclusion in the final draft as a part of the overall writing team and a CLA 

for Chapter 3. 

Jones explained that the decision to include MM2004 (and de Laat and 

Maurellis, 2006) was made at the final plenary team meeting in Bergen, and as 

stated the text was seen by the whole writing team. It had not been possible to 

include de Laat and Maurellis (2006) until then, as it was not published until 

after the third Lead Author‘s meeting. Discussion of MM2004 can be seen in 

comments numbered 3-283 to 3-289 of the Second Order Draft of Chapter 313. 

In two of these comments (3-284 and 3-285) it was stated that reference would 

be made to MM2004 in Section 3.2.2.2 with some text, which would point out 

that the papers by MM2004 and de Laat and Maurellis were biased. The fact 

that the Chapter author team had now read de Laat and Maurellis is referred to 

in response to comment 3-289. The comments were signed off by the two 

Review Editors for the Chapter. 

Jones also noted to us that there were three criticisms made of the draft of the 

Summary for Policy Makers (SPM) based on those of MM2004, but that these 

were rejected by the SPM writing team (which did not include any members 

of the Chapter 3 team). They regarded them as inconsistent with a large body 

of the climatological literature addressed in Chapter 3 (comments numbers 

482, 864 and 1005)14

.

9.3.5 Evidence from IPCC Review Editor for Chapter 3 

(Professor Sir Brian Hoskins) 

16. The role of Review Editors was to ensure that all comments from expert and 

government reviewers were given appropriate consideration, to advise lead 

authors how to handle contentious and controversial issues and to ensure that 

genuine controversies were reflected adequately in the text. Professor Hoskins 

was one of three Review Editors for Chapter 3. Telephone evidence was sought 

from Hoskins about the ways of working of the author team for Chapter 3. The 

full summary of the interview with Hoskins is available on the Review website15

.

17. Hoskins confirmed that LAs, working individually and as small groups, were 

responsible for the collation and primary assessment of material relevant to the 

 

13 Comments are available at http://hcl.harvard.edu/collections/ipcc/ by scrolling down to appropriate 

comments. 

14 Ibid. 13 

15 Evidence from Review Editors for Chapters 3 and 6 of the IPCC Fourth Assessment Report on “The 

Physical Science Basis” (June 2010) 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

75

topics for which they were responsible. The CLAs led the plenary meetings of the 

writing team prior to production of each of the drafts, led the process of overall 

collation of the Chapter material and the production of the initial drafts of the First 

and Second Draft Reports and the Final Draft Report of the Chapter. These drafts 

were discussed and agreed during plenary meetings of the whole writing team. 

18. There were a very large number of comments from reviewers, of which a majority 

were from a relatively small group. The Review Editors made sure that they were 

all given proper consideration, and that they were either responded to by a change 

in the text or by an adequate reason for omission that was recorded in the author 

responses to expert and governmental review comments16. Hoskins, as a Review 

Editor, took part in the Chapter 3 plenary discussions and ensured that conflicting 

views were addressed. 

19. Led by the two CLAs, Jones and Trenberth, the writing team for Chapter 3 was 

assiduous in dealing with comments. Hoskins was very impressed by Jones‘ 

attention to detail, and the rigour of the Chapter 3 process. 

20. The levels of confidence and uncertainty reflected in the drafts were based on the 

consensus of a group of CLAs and LAs who were chosen for their expertise and 

experience in relevant fields. Irrespective of whether a paper is published in a peer 

reviewed journal, it is the responsibility of the whole team to assess whether a 

paper‘s conclusions are robust and to justify whether its arguments should carry 

weight in the assessment. These decisions for each draft were taken in plenary 

sessions of the whole team. Hoskins said that it is inconceivable that a paper 

making significant claims relevant to the work of IPCC and the Chapter 3 team 

would not be considered by the team as a whole. The basis for rejecting one of the 

papers that is a focus of the allegation is included in IPCC records17. Decisions 

about the inclusion of the MM2004 paper would have been taken by the whole 

team. Jones‘ voice would have been one amongst many. 

9.3.6 Findings 

21. The essential issues in determining whether there is substance to the allegation 

that Jones misused his position to exclude well-founded conclusions that 

conflicted with his prior views and ―invented‖ an explanation in order to reject 

them are as follows: 

i) Was the decision to exclude the MM2004 paper from the first and second drafts 

an unreasonable one? 

ii) Was the reason given for the rebuttal of MM2004 in the final draft ―invented‖? 

iii) If the answer to either or both of these questions is ―yes‖, was Jones the major 

 

16 Available at: http://hcl.harvard.edu/collections/ipcc

17 The issue is discussed further in discussion of reviewers comments on the 2nd order draft of Chapter 

3 (footnote to comments 3-275 to 3-389), in part of which McKitrick objected to the exclusion of the 

MM2004 and de Laat and Michaelis 2004, to which the writing team responded by the addition of text 

to section 3.2.2.2 of the draft. This was included in the final draft of Chapter 3. The comments and 

responses can be found at: 

http://pds.lib harvard.edu/pds/view/7786376?n=40&s=4&imagesize=1200&jp2Res=.25&rotation=0 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

76

influence in excluding early consideration and/or in providing an ―invented‖ 

explanation? 

22. The answer to i) depends upon the scientific credibility of the article. The 

contemporary emails indicate a highly critical response to it, and Jones‘ email 

(paragraph 9) is very strongly negative. Was the absence of a discussion of 

MM2004 in the first and second drafts reasonable, or could it reflect suppression 

of a view merely because it conflicted with that of the writing team or of Jones? 

Was the analysis in MM2004, and later in de Laat and Maurellis (2006), so 

evidently flawed in an issue as important as the significance of the instrumental 

record of climate change (see Chapter 6), that it could be readily rejected in the 

AR4? Jones believes that it was (paragraph 15, bullets 2-4), although the 

persistence of a debate on the issue in peer reviewed journals, including on one 

side MM2004, de Laat and Maurellis (2006), McKitrick and Michaels (2007)18

and McKitrick (2010)19, and on the other side Benestad (2004)20, Parker (2010)21, 

Jones et al (2008)22 and Schmidt (2009)23, suggests at least a continuing margin of 

doubt. Those within the writing team took one view, and a group outside it took 

another. It is not in our remit to comment on the rights and wrongs of this debate, 

but those within the team had been entrusted with the responsibility of forming a 

view, and that is what they did. They initially rejected inclusion of reference to 

MM2004, and in the final draft included a commentary on it explaining why they 

disagreed with its conclusions. It may be that the conclusions of MM2004 

conflicted so strongly with a generally held view among climate scientists that 

rejection was made too easily; but in the absence of better evidence, this is mere 

speculation. The mechanisms of the IPCC did however ensure that a reference to 

the article (and to de Laat and Maurellis, 2006) was contained in the final draft. 

23. The answer to ii) depends upon the implication that the response to MM2004 in 

the published Chapter 3 was not scientifically credible. Having read most of the 

relevant papers however, we observe a consistence of view amongst those who 

disagree with MM2004 that has been sustained over the last 6 years, that the large 

scale organisation of atmospheric circulation produces a spatially integrated 

response to forcing. Although we do not comment on the relative merits of the 

two views, we see no justification of the view that that this response was 

―invented‖, or even that its various expressions in the response to reviewer Gray24

or the final text are fundamentally different. 

24. Irrespective of the above comments on issues i) and ii), the evidence of the 

Review Editor underlines the team responsibility for the text, and the unlikelihood 

 

18 McKitrick R and Michaels P. 2007. Quantifying the influence of anthropogenic surface processes 

and inhomogeneities on gridded global climate data. Journal of Geophysical Research 112: D24S09. 

19 McKitrick, R., 2010: Atmospheric circulations do not explain the temperature-industrtrialization 

correlation. Statistics, Politics and Policy (in press – summer 2010). 

20 Benestad, R. 2004. Are temperature trends affected by economic activity? Comment on McKitrick 

and Michaels. Climate Research, 27, 171-173.

21 Parker, D. E., 2010: Urban heat island effects on estimates of observed climate change, Wiley 

Interdisciplinary Reviews: Climate Change, 1(1), 123-133, doi:10.1002/wcc.21. 

22 Jones, P.D., Lister, D.H. and Li, Q., 2008: Urbanization effects in large-scale temperature records, 

with an emphasis on China. J. Geophys. Res. 113, D16122, doi:10.1029/2008/JD009916. 

23 Schmidt, G.A., 2009: Spurious correlations between recent warming and indices of local economic 

activity. Int. J. Climatol. 29, 2041-2048. 

24 Ibid. 5
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

77

that a single voice could dominate on an important issue. Jones‘ evidence is that 

McKitrick‘s ―surmise that Professor Jones ... wrote the paragraph (in Chapter 3) 

alone or in consultation with Trenberth, and bears responsibility for its 

inclusion”25 is false, and that the lead responsibility for the relevant section was 

another specified member of the writing team. 

25. We conclude that there is evidence that the text was a team responsibility. It is 

clear that Jones (though not alone) had a strongly negative view of the paper but 

we do not find that he was biased, that there was any improper exclusion of 

material or that the comments on the MM2004 paper in the final draft were 

―invented‖ given the (continuing) nature of the scientific debate on this issue. 

9.4 The Tree Ring Proxy Temperature Series 

9.4.1 The Scientific Challenge 

26. A paper by McIntyre and McKitrick (2003; hereafter referred to as M&M2003)26

argued that the so called ―hockey stick‖ plot (Mann, Bradley and Hughes, 1998; 

hereafter referred to as MBH98)27 contained both simple errors and serious 

statistical errors. It suggested that the ―hockey stick‖ shape of the MBH98 

reconstruction was largely an artefact of these errors and of the selection of 

specific tree ring series. 

27. This work posed a significant challenge to interpretation of the assessment by the 

IPCC of the history of climate of the last millennium prior to the instrumental 

record of the last 150 years. It cast doubt on the validity of the claim of MBH98 

and of Mann, Bradley and Hughes (1999; hereafter MBH99)28 that the northern 

hemisphere was warmer in the late 20th century than at any time during the last 

millennium, which has been assumed by many to be evidence in support of the 

argument for strong, human induced greenhouse warming over recent decades. 

 

9.4.2 The Allegations 

28. It has been alleged that Briffa, in his role as lead author for Chapter 6 in Working 

Group 1 for AR4, and as the member of the writing team with the most relevant 

expertise, attempted to bias the scientific conclusions towards those of the 

MBH98/99 and to set aside the inconvenient evidence of M&M2003. It is alleged 

this behaviour was calculated to favour one particular view of climate change and 

its causes, and to discredit or ignore opposing views, without, at the time, an 

adequate scientific reason for doing so. It would therefore represent a failure to 

discharge a scientist‘s responsibility impartially to represent current scientific 

understanding at the vital interface between science and policy. 

 

25 McKitrick submission (no. 15), 26 February 

26 McIntyre, S. and McKitrick, R. 2003. Corrections to the Mann et al (1998) proxy database and 

northern hemisphere average temperature series. Energy and Environment, 14, 751-771.

27 Mann, M.E., Bradley, R.S. and Hughes, M.K. 1998. Global-scale temperature patterns and climate 

forcing over the past six centuries. Nature, 392, 779-787. 

28 Mann, M.E., Bradley, R.S. and Hughes. M.K. 1999. Northern hemisphere temperatures during the 

last millenium: inferences, uncertainties and limitations. Geophysical Research Letters, 26(6), 759-762.
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

78

9.4.3 Evidence in Support of the Allegations 

29. The Chapter 6 writing team relied heavily on a paper that was in preparation by 

Wahl and Ammann (eventually published as Wahl and Ammann, 2007, hereafter 

referred to as WA2007) that purported to refute the arguments of M&M2003. It 

has been alleged that Briffa played a central role in improperly using WA2007 to 

refute M&M2003, to discredit a paper that conflicted with his core hypothesis, 

and in doing so was willing to break IPCC rules: 

The claim of refutation made in the second order and the published drafts of 

Chapter 6 was knowingly misleading, because the method used to support the 

conclusions of WA2007 was dubious, and in any case, relied upon material 

that was rejected for publication in Geophysical Research Letters, and was not 

made available until August 2008 in an online supplement to Ammann and 

Wahl 2007, hereafter referred to as AW2007, long after acceptance of the final 

draft of Chapter 6. Thus even if the paper did represent an effective refutation 

of the M&M2003 argument, it was not available prior to the acceptance of the 

final draft of Chapter 6 of AR4.

IPCC rules require papers that are to be referenced should be at least 

 ̳accepted‘ by journals by specific deadline dates. WA2007 missed these 

deadlines and should not have been quoted as evidence.

Briffa broke confidence by asking Wahl, who was not involved in the IPCC 

process, to comment on Chapter 6 text. 

30.The second order draft text of IPCC 2007 WGI Chapter 6, assumed in the 

allegation to have been written by Briffa, and sent late in March 2006 to the 

Government and Expert Reviewers, included on page 29 the following text that 

relied on WA2007 to rebut M&M2003: 

“McIntyre and McKitrick (2003) reported that they were unable to replicate the 

results of Mann et al. (1998). Wahl and Ammann (accepted) demonstrated that 

this was due to the omission by McIntyre and McKitrick of several proxy series 

used by Mann et al. (1998). Wahl and Ammann (accepted) were able to 

reproduce the original reconstruction closely when all records were included”. 

31. This text was criticised by the Reviewer for the Government of the United States 

of America, who wrote in comment 6-75029

: 

“The use of Wahl and Ammann (accepted) does not comply with WG1‟s deadlines 

and all text based on this reference should be deleted. WG1‟s rules require that 

all references be “published or in print” by December 16, 2005. Wahl and 

Ammann was “provisionally accepted” on that date, and not fully accepted until 

February 28, 2006, at which time no final preprint was available. Substantial 

changes were made in the paper between December 16, 2005 and February 28, 

 

29 Comments are available at http://hcl.harvard.edu/collections/ipcc/
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

79

2006, including insertion of tables showing that the MBH98 reconstruction failed 

verification with r-squared statistics, as had been reported by McIntyre and 

McKitrick in 2003. These tables were not available in the draft considered by 

WG1 when developing the second-order draft.” The response to this was:

“Rejected – the citation is allowed under current rules” (Comment 6-1158)30. 

32. It was alleged that the material derived from WA2007 that was the rationale for 

the text in the final version of Chapter 6 was based on material that was not 

published or openly available until after the last deadline for the final draft. Their 

evidence should therefore not have been included.

33.In an email dated 18 July 2006 (1153470204), Briffa wrote to Wahl, who was not 

an official Expert Reviewer, as follows: 

“Gene I am taking the liberty (confidentially) to send you a copy of the 

reviewers comments (please keep these to yourself) of the last IPCC draft 

chapter. I am concerned that I am not as objective as perhaps I should be and 

would appreciate your take on the comments from number 6-737 onwards, that 

relate to your reassessment of the Mann et al work. I have to consider whether 

the current text is fair or whether I should change things in the light of the 

sceptic comments. In practise this brief version has evolved and there is little 

scope for additional text, but I must put on record responses to these comments 

- any confidential help , opinions are appreciated . I have only days now to 

complete this revision and response. note that the sub heading 6.6 the last 2000 

years is page 27 on the original (commented) draft. Cheers Keith”

34. It is alleged that this e-mail is an appeal to a strong proponent of the ―hockey 

stick‖ plot for assistance in coping with the comments of reviewers sceptical about 

it, that it hands confidential material to Wahl for him to help rebut the comments 

from Expert Reviewers critical of the Wahl and Ammann paper, and that it breaks 

rules of confidentiality at a stage when even official reviewers were denied ready 

access to review comments. It is implied that Briffa was prepared to go to 

exceptional and improper lengths to bolster a case that he supported and to defend 

it against alternative views. 

35. Finally, it is alleged that the relevant paragraph on p.466 in Chapter 6 of the AR4 

Final Report leaves the last word to Wahl and Ammann, and the reader is left with 

the clear impression that the M&M2003 criticisms have been rebutted, although 

the work claimed to be the rigorous basis of this rebuttal had missed or was long 

after IPCC deadlines. 

 

30 Ibid. 29 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

80

9.4.4 Responses from Briffa 

36. These allegations were put to Briffa. We summarise his oral and written responses 

as follows31,32,33,34

: 

a) Evaluation of the M&M2003 work by the Chapter 6 writing team 

Briffa responded that the M&M2003 work was taken very seriously by the 

Chapter 6 writing group. There was genuine concern about whether its claim 

that the MBH98 reconstruction could not be replicated was correct. It was 

important to assess the merits of this, as a failure in replication would, if 

substantiated, be crucial in assessing confidence in the MBH99 reconstruction 

and the trend of past climate. Did M&M2003 provided sufficient grounds for 

dismissing the view of temperature change in MBH98? 

The evaluation of this issue was very similar in both the second order and 

final, published versions of Chapter 6. The final version is: 

“McIntyre and McKitrick (2003) reported that they were unable to 

replicate the results of Mann et al. (1998). Wahl and Ammann (2007) 

showed that this was a consequence of differences in the way McIntyre and 

McKitrick (2003) had implemented the method of Mann et al. (1998) and 

that the original reconstruction could be closely duplicated using the 

original proxy data. McIntyre and McKitrick (2005a,b) raised further 

concerns about the details of the Mann et al. (1998) method, principally 

relating to the independent verification of the reconstruction against 19thcentury instrumental temperature data and to the extraction of the 

dominant modes of variability present in a network of western North 

American tree ring chronologies, using Principal Components Analysis. 

The latter may have some theoretical foundation, but Wahl and Ammann 

(2006NOTE) also show that the impact on the amplitude of the final 

reconstruction is very small (~0.05°C; for further discussion of these 

issues see also Huybers, 2005; McIntyre and McKitrick, 2005c,d; von 

Storch and Zorita, 2005).”

The first sentence above reflects the concern that the results of MBH98 could 

not be replicated. 

Briffa noted however that M&M2003 had not followed the MBH98 method and 

that Rutherford et al. (2005)35 had pointed out that the M&M2003 approach to 

calculation resulted in the elimination of 77 out of the 95 pre-1500 tree ring 

 

31 Response to specific questions raised by Professor Geoffrey Boulton, in his letter of 6 May 2010, in 

his role as a member of the Muir-Russell Review team (19 May) 

32 Issues for discussion with Briffa and Jones on 9 April 2010 (April 10) 

33 Response to Additional Question regarding Keith Briffa‟s request to Eugene Wahl and his response 

(June 2010) 

34 Copies of Communications relating to Professor Briffa‟s editorial treatment of a submitted 

manuscript (June 2010)

35 Rutherford, S., M.E. Mann, T.J. Osborn, R.S. Bradley, K.R. Briffa, M.K. Hughes and P.D. Jones, 

2005: Proxy-based Northern Hemisphere surface temperature reconstructions: Sensitivity to 

methodology, predictor network, target season and target domain. Journal of Climate 18, 2308-232. 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

81

proxy series used by MBH98. 

WA2007 had then shown that the results of MBH98 could be replicated very 

closely using their implementation of the MBH98 methods and using the same 

data. This is pointed out in the second sentence of the above paragraph from 

Chapter 6. 

Briffa pointed out that the AR4 text did not state that WA2007 had disproved 

the concerns of M&M2003. Instead it considered their possible impact on the 

final reconstruction, citing papers that had assessed this impact including, but 

not exclusively, WA2007. These results indicated that the impact on the final 

reconstruction might be relatively small, leading to the view, contained in the 

AR4 text, that the criticisms raised against the MBH98 reconstruction were not 

sufficient to discount the evidence that it represented. 

Briffa commented that he believed the above quoted paragraph from the 

published report represented a proper and balanced response to the issues raised 

by M&M2003, that the issues were important to the assessment and that the 

reference to WA2007 was a significant part of the debate. 

Briffa and his colleague Osborn36 commented that in any case the MBH98 was 

only one of 12 such reconstructions in figure 6.10 in Chapter 6, and does not 

therefore dominate the picture. The M&M2003 series is not presented in that 

figure as an alternative reconstruction, as McKitrick commented on the first 

order draft that, the draft ―trots out the straw man that we are selling an 

alternative climate history, despite our repeated and persistent statements that 

we are not trying to offer „our‟ climate history curve.‖

37

Briffa also rejected the implication that this text was his responsibility, asserting 

that it was the responsibility of the whole writing group, not of any one person. 

 b) Deadlines 

Briffa rejected the allegation that IPCC rules on deadlines were broken because 

of a determination to include reference to WA2007 (which was claimed to have 

been in press at the time). Details of these allegations and the responses to them 

from Briffa and Osborn are on the Review website38

.

Briffa pointed out that the scientific content in AW2007, referred to by 

WA2007, is equivalent to the content in the manuscript rejected by Geophysical 

Research Letters (this was not rejected for its scientific content, but for other 

editorial reasons). 

 

 c) Breaching confidentiality 

 

36 Response to specific questions raised by Professor Geoffrey Boulton, in his letter of 6 May 2010, in 

his role as a member of the Muir-Russell Review team (May 2010) 

37 Minute 6-1319 at http://hcl.harvard.edu/collections/ipcc/

38 Ibid. 33 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

82

Briffa responded to the allegation of having broken confidentiality in sending 

draft text to Wahl to comment on, that there is no proscription in the IPCC rules 

to prevent the author team seeking expert advice when and where needed. The 

Technical Support Unit (TSU) and the CLAs of Chapter 6 agree that the author 

team was allowed to seek such advice. Copies of communications from both 

CLAs (Jansen and Overpeck) and the IPCC WG1 TSU are provided by Briffa 

(and published on the website) to provide support to Briffa‘s claim that his 

actions did not contravene IPCC procedures. 

Briffa asserts that Wahl was asked for comment on text as a knowledgeable and 

objective arbiter and as such was a wholly reasonable judge of whether the 

responses were appropriate. Given his particular expertise on the details of the 

Mann et al. methodology and most importantly the implications for the 

character of the Mann et al. reconstruction, Briffa felt justified in seeking his 

advice and in using specific wording in a very few responses that were based on 

the text of a paper co-authored by Wahl (AW2007). Wahl did not write any of 

the main text, though he did make some suggestions for very minor edits. 

Briffa‘s evidence includes copies of the relevant email exchanges with Wahl39

, 

which also confirm that both Jansen and Overpeck, as CLAs, were aware of the 

approach to Wahl. 

9.4.5 Evidence from IPCC Review Editor for Chapter 6 

(Professor John Mitchell) 

37. Mitchell was one of two Review Editors for Chapter 6 of the AR4 Working Group 

1 report. A telephone interview with him was conducted to establish how 

unpublished and ―in press‖ material was handled and how decisions about the text 

of successive drafts were made. He commented as follows40: 

“I was not aware of the debate about whether the Wahl and Ammann paper had 

or had not met the deadline for the 2nd order draft for chapter 6, until after the 

event. The concentration on specific deadlines however misses the larger point. It 

must be recognised that if only published sources were used, the report would be 

two years old by the time of publication. In a fast-moving area such as climate 

change research, assessments could be significantly behind the times if important, 

but as yet unpublished, new results could not be used. The assessments for 

policymakers could also therefore be behind the times”. 

“In earlier assessments, there had been a relatively liberal regime in using 

unpublished material provided that there was a sound basis for regarding it as 

rigorous or reliable, although priority was always given to finding published 

sources. In AR4 however, the regime was tightened significantly, so that such 

material was only to be used under exceptional circumstances, but the use of 

unpublished material was not prohibited. „Hockey-stick‟ issues were regarded at 

 

39 Response to Additional Question regarding Keith Briffa‟s request to Eugene Wahl and his response 

(June 2010), available at: www.cce-review.org/Evidence.php 

40 Evidence from Review Editors for Chapters 3 and 6 of the IPCC Fourth Assessment Report on “The 

Physical Science Basis”
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

83

the time as sufficiently important to justify using new data. The dilemma between 

using only published material and being out of date, or using more recent 

unpublished material was increased in AR4 as the „latest publication date‟ was 

about 12 months earlier than in the process than in the previous assessment”.

“The email is problematic (para 33). On the one hand it appears to reflect an 

honest request to an expert for a comment about the extent to which the author is 

being balanced and fair. On the other hand, it stresses the need for confidentiality 

in three places, implying that the author realizes that the approach may be 

improper. There was also a leak of an early draft of the WG1 report to the press 

which led to IPCC emphasizing the need to maintain confidentiality in general 

which may have been at the back of the author‟s mind”. 

“In principle however there is nothing in IPCC rules that prevents an author from 

seeking external help, comment or judgement on text through consulting his peers. 

It is questionable whether expert reviewers‟ comments, in some* ways analogous 

to the comments of a peer reviewer for a journal, should be shared with an third 

party without their consent. (*Note that unlike most peer-reviewed journals, IPCC 

reviewers names and comments are made available at the end of the process).”

9.4.6 Findings 

38. The essential issues to be resolved in relation to the allegations are: 

Was the McIntyre and McKitrick (2003) paper dealt with in a reasonable 

fashion in the draft and final versions of Chapter 6? 

The evidence and narrative provided by Briffa is persuasive that these issues were 

dealt with in a careful and reasonable fashion that took into account the 

importance of the issues addressed in M&M2003. McIntyre or McKitrick may 

have wished to see them addressed in a different way, but they were addressed 

seriously and cogently. 

Is there any evidence of any personal desire from Briffa to protect a 

fundamentalist line in defending the conclusions of MBH98/99? 

The evidence of the Review Editor suggests that no one person in the writing team 

could have overridden the team responsibility for the text. Indeed, the evidence of 

a contemporary e-mail (1140039406; Feb 2006; to Overpeck) suggests that Briffa 

was unlikely to be an uncritical defender of the MBH view of the  ̳hockey stick‘, 

and wished to respect the view of the writing team as a whole: 

―Peck, you have to consider that since the TAR, there has been a lot of argument 

re„hockey stick‟ and the real independence of the inputs to most subsequent 

analyses is minimal. True, there have been many different techniques used to 

aggregate and scale data - but the efficacy of these is still far from established. 

We should be careful not to push the conclusions beyond what we can securely 

justify - and this is not much other than a confirmation of the general conclusions 

of the TAR.....Just need to show the "most likely"course of temperatures over the 

last 1300 years - which we do well I think. Strong confirmation of TAR is a good 

result, given that we discuss uncertainty and base it on more data. Let us not try 

to over egg the pudding. For what it worth, the above comments are my (honestly 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

84

long considered) views - and I would not be happy to go further. Of course this 

discussion now needs to go to the wider Chapter authorship, but do not let Susan 

[Solomon – co-chair of IPCC WG1 for AR4] (or Mike) [Mann] push you (us) 

beyond where we know is right.”

Is the inclusion of references to WA2007 allegedly against the rules of IPCC 

evidence of a determination to rebut M&M2003 at all costs? 

Taking into account the evidence of the Review Editor, IPCC papers, the 

statement from the CLA, and the importance of the issues raised by M&M2003, 

we consider it to be reasonable that work that might throw further light on these 

issues and to ensure that assessments were as up to date as possible should be 

included. We do not consider therefore that these were exceptional unwarranted 

efforts to defend a particular position, but reasonable attempt so use up to date 

information that might resolve an issue. They appear to be consistent with IPCC 

principles and to reflect a concern for objectivity. 

Was there breach of confidentiality in having Wahl comment on draft text of the 

report and does this reflect determination to sustain a predetermined line? 

Although Briffa‘s e-mail41 stressing confidentiality does imply an awareness of 

questionable conduct, the e-mail correspondence with Wahl42 stresses in several 

places Briffa‘s concern to be fair to sceptical views. We see no evidence in the 

correspondence of anything other than a detailed determination to resolve a 

scientific issue. Nor do the IPCC Review Editor‘s comments to us suggest that 

what would normally be regarded in the research community as conventional 

requests for advice and help, were ruled out. 

39. We conclude, in line with the comments made by Professor Mitchell, it was not 

unreasonable to include the WA2007 paper alongside the M&M2003 paper in 

presenting an up-to-date picture of the relevant scientific arguments. We do not 

find that Briffa acted improperly in the part he played in this, and note that in any 

case he did not have sole responsibility for the outcome. 

9.5 Conclusions 

40. In summary, we have not found any direct evidence to support the allegation that 

members of CRU misused their position on IPPC to seek to prevent the 

publication of opposing ideas. 

41. In addition to taking evidence from them and checking the relevant minutes of the 

IPCC process, we have consulted the relevant IPCC Review Editors. Both Jones 

and Briffa were part of large groups of scientists taking joint responsibility for the 

 

41 Briffa to Wahl email: 1140039406.txt; Feb 2006 "I am taking the liberty (confidentially) to send you 

a copy of the reviewers comments (please keep these to yourself) of the last IPCC draft chapter. I am 

concerned that I am not as objective as perhaps I should be and would appreciate your take on the 

comments from number 6-737 onwards, that relate to your reassessment of the Mann et al work. I have 

to consider whether the current text is fair or whether I should change things in the light of the sceptic 

comments. " 

42 Response to Additional Question regarding Keith Briffa‟s request to Eugene Wahl and his response 

(June 2010) 
CHAPTER 9: COMMUNICATING INTO THE PUBLIC DOMAIN THROUGH THE IPCC 

85

relevant IPCC Working Group texts and were not in a position to determine 

individually the final wording and content. We find that neither Jones nor Briffa 

behaved improperly by preventing or seeking to prevent proper consideration of 

views which conflicted with their own through their roles in the IPCC. 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

86

CHAPTER 10: COMPLIANCE WITH 

FoIA/ENVIRONMENTAL INFORMATION 

REGULATIONS 

10.1 Introduction and Method of Enquiry 

1. This Chapter addresses the third part of our remit: “Review the Climatic Research 

Unit‘s compliance or otherwise with the University of East Anglia‘s policies and 

practices regarding requests under the Freedom of Information Act 20001

( ̳the 

FoIA‘) and the Environmental Information Regulations2( ̳the EIR‘) for the release 

of data‖. It also reviews subject access requests under the Data Protection Act 

1998 – as amended3

. Whilst compliance with FoIA, EIR and DPA fall within the 

more general topic of  ̳Governance‘ (covered in the following Chapter), they are 

extracted and covered in detail here, given their very specific relevance to the 

Review. 

2. Interviews were carried out with the: 

Information Policy & Compliance Manager – IPCM; 

Science Faculty FoIA/EIR Contact; 

Director of Information Services; and 

Key staff within CRU.

3. Discussions were also held with representatives of the ICO, both to co-ordinate 

the work of the Independent Review with that of the concurrent ICO investigation 

and to seek advice. 

4. Notes of all these interviews are available on the Review website. 

10.2 The Allegations 

5. It is alleged by a number of correspondents (for example the submission by 

Matthews4) and commentators that requests under the FoIA and the EIR were 

incorrectly denied. Other correspondents (for example the submission by Mann5) 

have suggested that a number of these FoIA requests were inappropriate or 

frivolous. Similarly it is alleged that subject access requests under the DPA for 

access to e-mails specifically referencing the applicant were not fully complied 

with. 

10.3 General Context

6. The Freedom of Information Act 2000 created new statutory rights of access, 

subject to a number of defined exemptions, to information held by a wide range of 

public bodies. It replaced the earlier  ̳Code of Practice on Access to Government 

Information‘ that was a non-statutory scheme. The general right of access under 

 

1

 See: http://www.opsi.gov.uk/Acts/acts2000/ukpga_20000036_en_1 

2

 See: http://opsi.gov.uk/si/si2004/uksi_20043391_en.pdf 

3

 See: http://www.ico.gov.uk/what_we_cover/data_protection/legislation_in_full.aspx 

4

 Matthews submission (no. 16), 1 March 

5

 Mann submission (no. 42), 28 February 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

87

this Act came into force on 1st January 2005. In cases where access to information 

is refused, the Act requires the public authority to give reasons for that refusal, 

including detailing any exemptions the public authority believes may apply. 

There is also a duty under the Act to offer reasonable advice and assistance to 

applicants seeking information. 

7. The current Environmental Information Regulations (EIR) also came into force on 

1

st January 2005. These contain a similar general right of access to that defined 

under the FoIA. Their genesis however dates back to initial regulations based on 

the European Community (EC) Council Directive 90/313 published in 1992, to 

which UEA would have been subject. It is not clear whether this was widely 

understood either by the University or by those seeking information. 

8. Both the FoIA and EIR: 

encourage the use of formal publication policies (―Schemes‖) and pro-active 

dissemination of information; 

have initial 20 working day time limits for the public authority to respond to 

requests; 

require public authorities to provide advice and assistance to applicants and to 

provide the information in the form or format requested wherever reasonably 

possible. 

use the  ̳Public Interest‘ test. Information can only be withheld if: ―in all the 

circumstances of the case, the public interest in maintaining the exception 

outweighs the public interest in disclosing the information‖;

have defined appeal processes (albeit with some subtle differences on time 

limits); and 

have common recourse to the Information Commissioner and to the 

Information Tribunal. 

9. Key differences6

 between the FoIA and EIR regimes are that, under the EIR: 

requests can be made verbally and do not have to be submitted in writing (or 

by e-mail) with a name and address; 

there is a clearer definition of  ̳information held‘ than under the FoIA: ―If the 

information is in the authority's possession and has been produced or received 

by the authority, or if it is held by another person on behalf of the authority‖; 

a request cannot be rejected purely on grounds of cost; 

the allowed initial response deadline can be extended to 40 working days for 

particularly complex requests; 

withholding of information under exemptions remains subject to the  ̳public 

interest‘ test;

there is an emphasis on improved decision making and participation; and 

there is no direct equivalent to the FoIA  ̳vexatious requests‘ provisions.

There are no guarantees of absolute confidentiality under the EIR. 

10. The EIRs only cover environmental information, albeit this is subject to a very 

broad definitioni (see endnote after paragraph 35). The FoIA covers all other 

 

6

 See: http://www.defra.gov.uk/corporate/policy/opengov/eir/slides-leaflets htm 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

88

information held by public authorities, but with a specific exemption for 

environmental information. 

11. The DPA 1998 (as amended) gives individuals the right to know what information 

is held about them. It provides a framework to ensure that personal information is 

handled properly. The Act states that anyone who processes personal information 

must comply with eight principles, which make sure that personal information is: 

Fairly and lawfully processed 

Processed for limited purposes 

Adequate, relevant and not excessive 

Accurate and up to date 

Not kept for longer than is necessary 

Processed in line with your rights 

Secure 

Not transferred to other countries without adequate protection 

12. More specifically in the context of this Chapter, the Act provides individuals with 

important rights, including the right to find out what personal information is held 

on computer and most paper records. 

13. UEA‘s (and thus CRU‘s) formal processes are described in 3 documents:

―The Code of Practice for Responding to Requests for Information under the 

Freedom of Information Act 2000‖7

, which is based on the Lord Chancellor‘s 

Code and sets out the manner in which UEA processes requests. 

―Guidance for Staff‖8, which directs how UEA staff should respond to 

requests for information. 

―Requests for Information‖9, which explains how to make a request for 

information, and how the UEA expects to respond. 

14. UEA‘s FoIA/EIR processes are based on concentric circles. The IPCM is 

considered the central point. He: 

should always be the co-ordinator for FoIA/EIR requests; 

maintains the formal FoIA/EIR logs on requests received; 

establishes a case file for each request; 

makes the initial determination as to whether to treat a request under the FoIA

or EIR regime; 

determines in cooperation with the Faculty contact and relevant staff whether 

the information sought is indeed ―held‖ in statutory terms; and 

identifies and approaches the key provider via the appropriate faculty or 

central unit contact. 

15. The IPCM is the key person within UEA with a detailed understanding of the 

FoIA and the EIR. He maintains liaison with the ICO. He is also the key training 

provider to others within UEA. 

 

7

 See: 

http://www.uea.ac.uk/is/strategies/infregs/FOIA+Code+of+Practice+for+Responding+to+Requests 

8

 See: http://www.uea.ac.uk/is/foi/guidance 

9

 See: http://www.uea.ac.uk/is/foi/foi_requests 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

89

16. The next circle comprises FoIA/EIR contacts at the Faculty or Central Unit level. 

There are FoIA/EIR contacts in place for each of the four University Faculties and 

for each of the central university units (such as Information Systems and the 

Registrar‘s Department). These receive initial training and then yearly updates. It 

is understood that participation in these training sessions has been good. Beyond 

the initial training, it is understood that much learning is ―on the job‖. 

17. The final circle comprises individual members of staff/researchers. An initial 

brochure was sent to all units in the University when the FoIA/EIR regimes came 

into force in 2005. Awareness training is made available on a voluntary basis. 

10.4 Investigation

18. As stated in this Review‘s  ̳Issues for Examination‟ document (Appendix 3), it is 

alleged that requests under the FoIA, EIR and DPA were incorrectly denied by 

UEA on advice from CRU. This is the subject of a separate inquiry by the ICO, 

but does fall within the terms of reference of the Review. The Review has 

remained in regular contact with the ICO to ensure that both investigations could 

continue in parallel (see paragraph 19). The Review is particularly concerned to 

address: 

What formal processes were in place both centrally and within CRU to ensure 

fair and impartial assessment of FoIA requests? 

Were there any processes in place centrally to review recommendations from 

CRU that information should not be released? 

19. The improperly released e-mails contain a number of references that raise specific 

concerns with respect to FoIA, EIR and the DPA. In particular, since CRU uses 

data obtained from other bodies, guidance was sought from the ICO on the extent 

to which a public authority that was not a primary repository of data might be 

expected to act as a secondary source of that data for the purposes of FoIA/EIR. 

The advice received was that: 

Neither FoIA nor EIR make a distinction as to whether a public body is a 

primary or secondary source of information or data. The point is simply 

whether they hold the information for the purposes of the legislation, i.e. 

section 3(2) of FoIA and Regulation 3(2) EIR. 

If the public authority holds the information for any reason or in any form 

they must provide it or rely on a provision in the legislation to refuse the 

request. 

If however a request is received and the information requested is publicly 

available, and reasonably accessible, the public body can rely on either 

section 21 FoIA or Regulation 6 EIR and point the requester to these 

sources. “Reasonably accessible” will normally mean the information is 

published or available on demand. There is no obligation for the public 

body to apply section 21 or Regulation 6: the public body is free to choose 

to supply the data.
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

90

20. The Review recognises three time periods into which information requests should 

be grouped: 

before 2005, where the FoIA was not in operation but a precursor regime to 

the current EIR was operating; 

from 2005 until the start of 2007, when responses to requests were being 

handled informally by CRU; and 

after 25th January 2007 when the first request was formally logged by the 

IPCM. 

21. The Review had access to the complete formal log of FoIA/EIR requests, 

compiled by the IPCM, with respect to CRU since both current regimes came into 

force at the start of 2005. CRU only started to treat such requests formally via the 

IPCM in 2007. It should be noted however that there are extensive indications, 

from e-mails now in the public domain, of requests for information and some 

release of information prior to 2007. An example is the e-mail from Jones on 21st

February 2005 (1109021312.txt), ―PS I'm getting hassled by a couple of people to 

release the CRU station temperature data. Don't any of you three tell anybody 

that the UK has a Freedom of Information Act!‖. Indeed, three e-mails from 2004 

also made various information requests. 

22. Formal FoIA/EIR requests were initially quite limited. 

In 2007 four requests were received, of which two were given full release of 

the requested information but two, despite appeals, were rejected. 

In 2008 two requests were received, one was granted full release, but the other 

was rejected, both initially and upon appeal.

In the first half of 2009 only one request was received and this was responded 

to in full. 

23. But in the third quarter of 2009 a wave of requests was received. In the five days 

starting on 24th July, some 60 requests were logged by the IPCM. A further 10 

requests were logged between the 31st July and 14th August. Some related to the 

raw station data underpinning the CRUTEM data sets and the vast majority sought 

details of any confidentiality agreements related to this data. The wordings bear 

the hallmarks of an organised campaign. One applicant (UEA Log 09/97) appears 

to have forgotten to customise the request before dispatch. The text reads: ―I 

hereby make a EIR/FOI request in respect to any confidentiality agreements) 

restricting transmission of CRUTEM data to non-academics involving the 

following countries: [insert 5 or so countries that are different from ones already 

requested] 

the date of any applicable confidentiality agreements; 

the parties to such confidentiality agreement, including the full name of 

any organization; 

a copy of the section of the confidentiality agreement that "prevents further 

transmission to non-academics". 

a copy of the entire confidentiality agreement.”

24. In the final quarter of 2009 a further wave of 41 requests was received, starting on 

20th November. These were mostly related to the unauthorised release of e-mails 

and seeking information in the context of that release. However a number raised 

other issues such as: the extent of UEA staff training in FoIA/EIR issues; sources 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

91

of funding; and statistics on FoIA/EIR requests received. Still others harked back 

to previous themes of access to the CRUTEM data set and related confidentiality 

agreements. 

10.5 Findings

25. Education and culture. Whilst we did identify evidence that UEA had widely 

distributed initial guidance at the introduction of the FoIA/EIR regimes in 2005, 

we also found a lack of engagement by core CRU team in understanding 

EIR/FoIA legal requirements and how these might legitimately impact them. 

There was evident confusion within the CRU as to how these requirements might 

be applied for example to data, code, and personal correspondence. We found a 

tendency to assume that no action was required until precedents had been set. As 

an example, on 21st January 2005 Jones wrote (1106338806.txt): ―On the FOI Act 

there is a little leaflet we have all been sent. It doesn't really clarify what we might 

have to do re programs or data. Like all things in Britain we will only find out 

when the first person or organization asks. I wouldn't tell anybody about the FOI 

Act in Britain. I don't think UEA really knows what's involved.‖ There was 

insufficient priority given from the UEA centre to motivating staff and to 

prompting continuing education. Various requests received by the CRU between 

2005 and the start of 2007 had not been formally logged with the IPCM. 

26. Foresight lacking. We found a lack of recognition by both CRU, and the 

University‘s senior management of the extent to which earlier action to release 

information or give full guidance to locate primary sources and to provide station 

identifiers might have minimized the problems. There are many references in the 

e-mails now in the public domain to ―hiding information‖, ―finding ways around 

releasing‖, or finding excuses not to release information. There was a fairly swift 

shift towards a lack of sympathy with the requesters, as seen in an e-mail from 

Jones sent on 7th May 2005 (1083962092.txt): ―Mike and I are not sending 

anything, partly because we don't have some of the series he wants, also partly as 

we've got the data through contacts like you, but mostly because he'll distort and 

misuse them.‖ 

We do not suggest that the allegations made against McIntyre are correct. 

27. Unhelpful responses. We found a tendency to answer the wrong question or to 

give a partial answer. For example the very first formal FoIA request (UEA Log 

07-04) asked for the list of stations used in the preparation of HadCRUT3 and the 

raw data for these stations. The initial response refers simply to the availability of 

the raw data from other sources and not to the station list. The requester then 

enters into an extended correspondence trying to extract the station identifiers. An 

extract from 14th April 2007 is given below: 

“While it is good to know that the data is available at those two web sites, that 

information is useless without a list of stations used by Jones et al. to prepare 

the HadCRUT3 dataset. As I said in my request, I am asking for: 

1) A list of the actual sites used by Dr. Jones in the preparation of the 

HadCRUT3 dataset, and 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

92

2) A clear indication of where the data for each site is available. This is quite 

important, as there are significant differences between the versions of each 

site's data at e.g. GHCN and NCAR." 

Without knowing the name and WMO number of each site and the location of 

the source data (NCAR, GHCN, or National Met Service), it is not possible to 

access the information. Thus, Exemption 21 does not apply - I still cannot 

access the data. 

I don't understand why this is so hard. All I am asking for is a simple list of the 

sites and where each site's data is located. Pointing at two huge piles of data 

and saying, in effect, "The data is in there somewhere" does not help at all. 

To clarify what I am requesting, I am only asking for a list of the stations used 

in HadCRUT3, a list that would look like this: 

 WMO# Name Source 

 58457 HangZhou NCAR

 58659 WenZhou NCAR

 59316 ShanTou GHCN 

 57516 ChongQing NMS 

etc. for all of the stations used to prepare the HadCRUT3 temperature data. 

That is the information requested, and it is not available "on non-UEA 

websites", or anywhere else that I have been able to find. 

I appreciate all of your assistance in this matter, and I trust we can get it 

resolved satisfactorily.”

These station identifiers are finally promised on appeal, but not provided until 

some six months later following further prompting. 

28. Deliberate actions to avoid release. There seems clear incitement to delete emails, although we have seen no evidence of any attempt to delete information in 

respect of a request already made. Two e-mails from Jones to Mann on 2nd

February 2005 (1107454306.txt) and 29th May 2008 (in 1212063122.txt) relate to 

deletion: 

2

nd February 2005: ―The two MMs have been after the CRU station data for 

years. If they ever hear there is a Freedom of Information Act now in the UK, I 

think I'll delete the file rather than send to anyone”.

29th May 2008: ―Can you delete any emails you may have had with Keith re 

AR4? Keith will do likewise. He's not in at the moment - minor family crisis. 

Can you also email Gene and get him to do the same? I don't have his new 

email address. We will be getting Caspar to do likewise”. 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

93

There is a clear statement that e-mails had been deleted – for example, an e-mail 

from Jones to Santer sent on 3rd December 2008 (1228330629.txt ): ―About 2 

months ago I deleted loads of emails, so have very little - if anything at all.‖ 

It seems likely that many of these  ̳deleted‘ e-mails subsequently became public 

following the unauthorised release from the backup server. There is evidence that 

the IPCM did try to warn Prof. Jones about deliberate deletion of information; for 

example, an email from Jones to Santer (1228922050.txt) 10th December 2008 

states: ―I did get an email from the FOI person here early yesterday to tell me I 

shouldn't be deleting emails - unless this was 'normal' deleting to keep emails 

manageable!”. 

29. Imbalance of authority. The current structure for handling FoIA/EIR 

requests within UEA focuses very much on the IPCM. At interview he 

indicated that he felt: “Very much the bull’s eye at the centre of the target”. We 

believe that the UEA senior staff need to take more explicit responsibility for 

these processes, thus enhancing the authority and standing of the IPCM. We 

found that the IPCM may have lacked such standing within the University 

structure and the authority to challenge the assertions of senior professors. 

In this context, the IPCM and the Faculty FoIA contact may not have been 

empowered to be sufficiently rigorous: Jones to Santer sent on 3rd December 

2008 (1228330629.txt): When the FOI requests began here, the FOI person 

said we had to abide by the requests. It took a couple of half hour sessions - one 

at a screen, to convince them otherwise showing them what CA was all about. 

Once they became aware of the types of people we were dealing with, everyone 

at UEA (in the registry and in the Environmental Sciences school - the head of 

school and a few others) became very supportive.” However at interview the 

IPCM explicitly denied that he behaved in the way suggested in this e-mail. 

30. Lack of constructive challenge. Whilst we found that efficient basic control, 

logging and progress chasing processes for FoIA/EIR requests were in place, we 

found a lack of independent working-level challenge in these systems. We found 

that the appeals mechanisms in place lacked the resources for effective challenge 

to basic assumptions. Similarly, the escalation processes failed to react sufficiently 

quickly to the dramatic change in the volume and character of requests and to 

provide timely high-level review and resources. 

31. Limited internal communication. We found a lack of understanding within 

University central functions of the presence of extensive, and long duration, 

backups of e-mail and other materials despite these being on a server housed 

within the central Information Technology (IT) facilities. Awareness of these 

might have led to much greater challenge of assertions regarding non-availability 

of material by CRU, notably in the case of a subject access request made under the 

DPA for material naming the requesting individual. 

32. The Review found an ethos of minimal compliance (and at times non-compliance) 

by the CRU with both the letter and the spirit of the FoIA and EIR. We believe 

that this must change and that leadership is required from the University‘s most 

senior staff in driving through a positive transformation of attitudes. Public trust 

in science depends on an inherent culture of honesty, rigour and transparency. 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

94

The requirements of FoIA and EIR must not be seen as impositions. They are a 

necessary part of the implicit contract between the scientist and broader society. 

Such an open culture will also lead to the best science. 

10.6 Recommendations

33. The Review offers the following recommendations for action within the UEA: 

Change fundamentally the perception that responsibility for FoIA/EIR 

compliance lies with administrative staff. University senior staff need to make 

clear their commitment to a culture of honesty, rigour and transparency, plus 

the supporting processes and resources. 

Review the resourcing and standing of the FoIA/EIR/DPA compliance and 

request handling processes. Our findings have highlighted significant 

problems in the areas of: imbalance of authority; lack of effective challenge at 

appeal; over dependence on single individuals; inadequate escalation 

processes and limited strategic oversight. 

A concerted and sustained campaign to win hearts and minds. This should 

include: promotion of the University‘s formal publication policy; 

incorporating more information on FoIA/EIR/DPA responsibilities in the 

induction processes for new staff members; developing a rolling awareness 

campaign to focus the attention of established staff, particularly in the context 

of the changing landscape e.g. Queens University judgment (see paragraph 

34); and issuing annual reminders of the importance of transparency and of 

key FoIA/EIR/DPA responsibilities; 

Once the improved awareness measures and processes are in place, to run a 

programme of independent, external, tests with requests for information to 

verify the continuing effectiveness of these operations. This is a special case 

of the more general recommendation on  ̳Audit processes‘ given in the 

Governance Chapter. 

34. The Review offers the following more general recommendations: 

Definition of research data. There is extensive confusion and unease within the 

academic community as to exactly how FoIA/EIR should be applied in terms 

of the materials developed during a research process. The Review believes 

that all data, metadata and codes necessary to allow independent replication of 

results should be provident concurrent with peer-reviewed publication. 

However the situation regarding supporting materials such as early drafts, 

correspondence with research colleagues and working documents is widely 

regarded as unclear. The American experience is instructive here. The so 

called ―Shelby Amendment‖ in 1998 directed the US ―Office of Management 

& Budget (OMB)‖ to produce new standards requiring all data produced under 

Federally funded research to be made available under the US Freedom of 

Information Act. This resulted in great concern within the US Scientific 

community, expressed through Congressional testimony, that a very broad 

interpretation of this requirement could seriously impair scientific research 

and collaboration. In the final OMB guidelines10, recognising these concerns, 

 

10 Federal Register: March 16, 2000 Volume 65, Number 52 Page 14406. 

http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=2000_register&docid=00-5674-filed 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

95

―research data‖ is defined as: ―the recorded factual material commonly 

accepted in the scientific community as necessary to validate research 

findings, but not any of the following: preliminary analyses, drafts of scientific 

papers, plans for future research, peer reviews, or communications with 

colleagues‖. The Review recommends that the ICO should hold consultations 

on a similar distinction for the UK FoIA/EIR. 

Orchestrated campaigns. As detailed in paragraph 23, CRU was the subject of 

an orchestrated campaign of FoIA/EIR requests in late July and early August 

2009. The Review believes that CRU helped create the conditions for this 

campaign by being unhelpful in its earlier responses to individual requests for 

station identifiers and the locations from which specific, detailed station raw 

data could be downloaded. Similarly a clearer publication policy, reflecting 

the wishes of both the University and the research funders might have avoided 

these challenges. The Review team can however conceive of situations where 

such orchestrated campaigns might recur, with literally overwhelming impacts 

on small research units. We urge the ICO to give guidance on how best to 

respond to such organised campaigns, consistent with the underlying 

principles of openness. 

Greater clarity in an evolving landscape. Particularly in the light of the recent 

Queens University Belfast determination by the ICO in respect of the release 

of Irish Tree Ring data11, it would be helpful if the ICO could re-engage more 

generally with the Higher Education sector about their understanding of FoIA

and EIR obligations and also consider what further guidance could be 

provided for that sector. It would be particularly useful if guidance were 

available as to how long it is reasonable to retain data without release, pending 

full publication as part of a peer reviewed paper. It is however recognised that 

often such determinations have to be made on a case-by-case basis against a 

―public interest‖ test. 

35. As a final comment we find that a fundamental lack of engagement by the CRU 

team with their obligations under FoIA/EIR, both prior to 2005 and subsequently, 

led to an overly defensive approach that set the stage for the subsequent mass of 

FoIA/EIR requests in July and August 2009. We recognise that there was deep 

suspicion within CRU, as to the motives of those making detailed requests. 

Nonetheless, the requirements of the legislation for release of information are 

clear and early action would likely have prevented much subsequent grief. 

 

11 ICO Case Ref: FS50163282 

Date: 29/03/2010 

Public Authority: Queen‘s University Belfast

Summary: The complainant requested electronic data relating to tree ring research (dendrochronology). 

The public authority confirmed that it held the requested information but refused to provide it citing 

section 12 of the Act. The Commissioner indicated to the public authority that the withheld information 

fell within the definition of environmental information under the EIR. The public authority 

subsequently cited the exceptions at regulations 12(4)(d), 12(4)(b), 12(5)(c) and 12(5)(e) to refuse the 

13 cont information. The Commissioner finds that none of the exceptions is engaged and the withheld 

information should therefore be disclosed. The Commissioner also recorded a number of procedural 

breaches in the public authority‘s handling of the request.
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

96

i EIR definition of environmental information (see paragraph 10):

Any information in written, visual, aural, electronic or any other material form on: 

(a) the state of the elements of the environment, such as air and atmosphere, 

water, soil, land, landscape and natural sites including wetlands, coastal and 

marine areas, biological diversity and its components, including genetically 

modified organisms, and the interaction among these elements; 

(b) factors, such as substances, energy, noise, radiation or waste, including 

radioactive waste, emissions, discharges and other releases into the 

environment, affecting or likely to affect the elements of the environment 

referred to in (a); 

(c) measures (including administrative measures), such as policies, legislation, 

plans, programmes, environmental agreements, and activities affecting or 

likely to affect the elements and factors referred to in (a) and (b) as well as 

measures or activities designed to protect those elements;(d) reports on the 

implementation of environmental legislation; 

(d) reports on the implementation of environmental legislation; 

(e) cost-benefit and other economic analyses and assumptions used within the 

framework of the measures and activities referred to in (c); 

(f) the state of human health and safety, including the contamination of the food 

chain, where relevant, conditions of human life, cultural sites and built 

structures inasmuch as they are or may be affected by the state of the elements 

of the environment referred to in (a) or, through those elements, by any of the 

matters referred to in (b) and (c). 
CHAPTER 10: COMPLIANCE WITH FoIA/ENVIRONMENTAL INFORMATION REGULATIONS 

97
CHAPTER 11: GOVERNANCE 

98

CHAPTER 11: GOVERNANCE

11.1 Introduction and Method of Enquiry 

1. This Chapter addresses the fourth part of the Review‘s remit, ―Review and make 

recommendations as to the appropriate management, governance and security 

measures for CRU and the security, integrity and release of data it holds‖, to the 

extent that these issues are not addressed in previous Chapters. The previous 

Chapter deals extensively with the major governance issue of FoIA and EIR 

compliance. This Chapter considers other relevant aspects of the governance 

framework, first as it relates to Research Management Systems in Section 11.2 

and then secondly to the specific issue of the Management of Software, Data 

Management and Security in Section 11.3. The Chapter concludes with a set of 

recommendations in Section 11.4 on governance matters relating to both issues. 

2. The Review Team decided to focus on the efficacy of current control systems in 

UEA relevant to CRU‘s research, while developing a broad understanding of the 

evolution of UEA‘s policies and practices. 

3. Interviews were carried out with the: 

Vice-Chancellor 

PVC, Research Enterprise and Engagement 

Registrar and Secretary 

Associate Dean for Research, Faculty of Science 

Director of Information Services; 

Information Communications Technology (ICT) Systems Director; and 

CRU (part-time) IT Manager 

Director of Research, Enterprise and Engagement and Manager of Research 

Services 

Research Finance Management Accountant and Faculty of Science Finance 

Manager 

Director of Human Resources 

4. Notes of the relevant meetings and supporting documentation are accessible on 

the website. 

11.2 Research Management Systems 

11.2.1 Background

5. CRU is located within the School of Environmental Sciences (ENV) of UEA. 

Historically CRU had a high degree of autonomy, with separate reporting lines 

and distinct budgets. Apart from the Director, its staff were dependent on ―soft‖ 

money, that is on grants and other sources of funding mainly from sources such as 

UK Research Councils, EU programmes and the US Department of Energy. The 

position changed with the University‘s decision to invest strongly in 

environmental science. More recently up to 5 of the CRU staff have held 
CHAPTER 11: GOVERNANCE 

99

established positions and the current full-time equivalent is 3.5. 

6. The research policies and strategy of the University are set by a hierarchy of 

committees, with CRU being represented on the ENV Committee. The Director of 

CRU reports to the Head of ENV. 

11.2.2 Funding Management 

7. Responsibility for the University‘s funding management process rests with 

Research, Enterprise and Engagement (REE), through which major bids for 

funding are channelled. More routine funding bids are handled by Faculty 

Research Officers. The documentation supporting bids requires sign-off by the 

relevant principal investigator (PI) and Head of School, confirming their 

collective and individual responsibilities in delivering the research project in 

accordance with the funder‘s terms and conditions and the University‘s and 

School‘s policies and procedures.

11.2.3 Funders‟ Requirements 

8. Funders‘ requirements vary considerably and have developed over time. All seek 

publication of results from the work they support. The traditional expectation has 

been that this will be in peer reviewed journals. There is now some pressure for 

open access publication, which also involves peer review. The emphasis on 

protecting intellectual property varies. Regular progress reports are generally 

required, often as a condition of drawing down funds; and reports with specific 

formats and deadlines may be required on completion. There are rarely detailed 

requirements for the release of data or code. 

11.2.4 Good Research Practice

9. The university has a well developed set of policies on good research practice, 

research ethics, misconduct in research (including whistle blowing and grievance 

procedures), which apply to all those working in research, as well as regular 

performance reviews. The system is continuously developing; the good practice 

guidelines were first introduced in 2003 (we saw the 2009 version). Details of 

these structures, policies and procedures are referenced with the records of our 

interviews with the relevant senior managers, on our website. 

10. Prospective students and staff are sent weblinks before arrival. There is training 

for undergraduates carrying out research as part of their coursework, and for 

postgraduate students and for new members of staff there is probationary training. 

It is the responsibility of Principal Investigators to ensure that their people are 

fully trained 

11. The University‘s statement of terms and conditions of appointment of academic, 

teaching and research staff cross-refers to the requirements of a wide range of 

university policies. There is an induction process that includes coverage of what 

the requirements are and includes the allocation of a ―mentor‖, to act as an 

advisory colleague to a new member of staff. There is a standard annual training 

programme for both supervisors and new recruits. Compliance with the 
CHAPTER 11: GOVERNANCE 

100

requirements under the terms and conditions is the responsibility of Heads of 

School, with support from Human Resources as necessary. 

12. Outputs such as publication and citations are monitored on a yearly basis, research 

plans are prepared at institutional level and have to be signed off each year, and 

individuals receive appraisal with feedback every 2 years. UEA attaches 

importance to peer review as a check on people‘s work and publications, and 

encourages academics to seek outside involvement. 

13. Whistle blowing has never been used for any research related issue (it has been 

used a couple of times in all, in relation to other matters). The research 

misconduct procedure has been used on only a few occasions, mainly in relation 

to individuals‘ doctorate or master‘s theses. 

11.2.5 Financial Controls

14. Lead universities are normally responsible for administration of grants; payments 

are controlled by funders against a profile of expenditure and pass through central 

accounts only. 

15. Where cash advances are required, there are specific, senior approval procedures. 

In exceptional circumstances advances could be made to an employee‘s bank 

account. 

16. It was reported by UEA Finance that audit procedures are robust, both in general 

and in relation to certain EU projects where these are mandatory. UK Research 

Councils are increasingly asking for specific audit checks. 

11.2.6 Risk Management

17. The University maintains a risk register, regularly reviewed by Senior 

Management and annually by the Council and the Audit Committee. In relation to 

Research, the main concern has been the risk to reputation from slippage in league 

tables, with a range of mechanisms to drive performance in the desired direction. 

The register did not address the pressures on CRU as a result of the significance of 

its work for the climate change debate and the severity of the challenge to that 

work, and therefore did not consider the support CRU might need in handling its 

data and in making it available transparently. The upsurge of FoIA requests in 

2009 was not brought to the attention of senior management soon enough; and, 

when it was, there was no prepared framework for addressing the implications. 

Action is now being taken to raise awareness of the implications of FoIA and EIR, 

with leadership from senior management. 

11.2.7 Findings on Research Management Systems 

18. The Review team‘s reading of the improperly released e-mails together with 

Sherrington1

 suggested that cash advances to researchers in Russia might not have 

been adequately controlled, see also Chapter 4, paragraph 13. Whereas nearly all 

records from the mid-1990s have now been destroyed, in keeping with UEA‘s 

policy, we have confirmed that CRU did not have its own bank account and any 

 

1

 Sherrington submission (no. 107), 4 March 
CHAPTER 11: GOVERNANCE 

101

such payment would have required authorisation in writing by the Director of 

Finance at that time. While we have been unable to deal exhaustively with the 

issue, we are advised that there is a payment of $5000 in November 1998 properly 

recorded in the UEA ledger which is consistent with e-mails 911405082.txt and 

914256033.txt at that time. 

19. Apart from those issues addressed in Chapters 6 to 10, UEA‘s Research 

Management Systems appear adequate. Requirements have developed 

significantly over the period relevant to the Review, from fairly general 

expectations that terms and conditions set by funders would be complied with, to 

the highly developed system now in operation. 

20. To summarise, the focus of risk management in research was on quality and 

standards. The critical pressure on CRU, with its attendant risk to the University‘s 

reputation, was not on the radar screen. Senior management were not sufficiently 

aware of the issue, were not alerted in time to its emergence, and had no 

preparation in place to respond adequately. 

11.3 Software, Data Management and Data Security 

11.3.1 General Context 

21. In submissions, concerns were expressed to the Review over the quality of 

software development, testing and operational processes within CRU, though it 

was also asserted that much of such software is transient, and, though poorly 

structured and documented, may still be fit for the limited purposes envisaged. 

Submissions from Bratby2 and Barnes3 are relevant. We noted that a number of 

requests received by UEA under FoIA/EIR also sought access to code. Concerns 

have also been expressed about potential loss of data by CRU and the level of 

information security for CRU systems, which were vulnerable to unauthorised 

disclosure of the personal e-mails and other material subsequently published 

widely on the Internet.

22. IT Organisation. In common with other areas of the Science Faculty, CRU 

operates largely independently of the central IS functions of the UEA. Central IS 

has, in recent years, made significant efforts to better support the Science Faculty 

and some use of central facilities (such as the Storage Area Network) has been 

achieved. The University IS team does not provide desktop, remote access, 

hosting, database or software support to CRU, nor any quality control or 

assessment. CRU has its own local architecture based on a mix of individual PC 

based and server based processing. In common with many other research groups 

across the university, this is distinct from the UEA preferred model of client –

server operation. Internet communications for CRU is however routed over the 

university network and through the university firewall. CRU has its own IT 

Manager for whom CRU is 40% of his workload. CRU originally had no central 

backup arrangements for the individual researchers‘ PCs however its IT Manager 

introduced automated backup (using open source software) to a simple server held 

securely within the Central IS machine room. 

 

2

 Bratby submission (no. 3), 22 February 

3

 Barnes submission (no. 1), 6 March 
CHAPTER 11: GOVERNANCE 

102

 

23. Policy. A high level  ̳Information Systems Policy‘ and a related  ̳Information 

Security Policy‘4 were agreed and put in place in 2005 under the aegis of UEA‘s 

Information Systems Strategy Committee (ISSC), which includes representatives 

of all four Faculties. Low level, detailed, security policies had been developed 

and put in place by 20075

.

24. Standards. The Review is aware of a number of industry standards relevant to this 

Chapter dealing with: software development, data stewardship and security. For 

further details see Appendix 8. 

11.3.2 Issues and the Investigation 

25. The Director of Information Services indicated that, whilst the central IT function 

were aware of the existence of the CRU Backup Server, they had no knowledge of 

the nature of the information held on the server as it was managed from CRU. 

26. The CRU IT Manager indicated that researchers within CRU worked individually 

or in small groups. There was no master index of resources, be these data, 

algorithms or software. No systematic approach to the creation of metadata 

existed. There was no central database of underlying climate data; rather 

individual researchers assembled sets of data drawn from different primary 

sources outside CRU (for example the Met. Office Hadley Centre for Climate 

Change). These might arrive by network (The United Kingdom‘s Educational and 

Research Network, or JANET), or on portable hard disks. 

27. With the exception of a small amount of tree ring data, CRU does not generate 

new raw data. It relies instead on accessing existing primary data sources. 

Nonetheless the importance of maintaining clear records of what data has been 

accessed for what purpose – for example in terms of station records processed – is 

clear. The benefits of such a central data catalogue (or ―data dictionary‖) had long 

been recognised within CRU and past attempts had been made to create such a 

resource. These attempts had foundered on the lack of resources – research grants 

made no provision for this and central UEA funding had not been available. 

Whilst there was no policy for the systematic archiving of data, many of CRU‘s 

processed results datasets are available on the Web6

. 

28. Individual researchers were responsible for acquiring or developing their own 

software applications (usually written in Fortran or Interactive Data Language, 

IDL). There was no formal quality control policy or review policy. However 

individual projects would comply with whatever quality control processes were 

specified as part of their funding arrangements. 

29. At interview, the ICT Systems Director indicated that ―lessons had been learnt‖ 

and he expected (subject to the results of a security audit report) to bring forward 

proposals within the University for: 

 

4

 These can be downloaded from: http://www.uea.ac.uk/is/itregs/ictpolicies 

5 A draft ―Security Manual‖ (not available for public download) was received by the Review on 8th 

February. 

6

 See: http://www.cru.uea.ac.uk/cru/data/hrg/ 
CHAPTER 11: GOVERNANCE 

103

Greater compliance with centrally defined IS policies and architecture; 

An audit of research data held in digital storage across the University; and 

Clear data retention (and destruction) policies. 

11.3.3 Findings on Software, Data Management and Data 

Security 

30. The software development process. We found that, in common with many other 

small units across a range of universities and disciplines, CRU saw software 

development as a necessary part of a researcher‘s role, but not resourced in any 

professional sense. Small pieces of software were written as required, with 

whatever level of skill the specific researcher happened to possess. No formal 

standards were in place for: 

Software specification and implementation; 

Code reviews; and 

Software testing 

31. Data management and archiving. We found that there were no formal processes 

in place within CRU with respect to the systematic retention and archiving of data 

sets, or more particularly of metadata (data that allows the data set to be correctly 

interpreted). Individual researchers took whatever actions they deemed 

appropriate within the context of specific agreements with research funding 

bodies. 

32. Information security. We found that the basic security processes had been 

appropriately specified and documented by the UEA‘s Information Systems 

Strategy Committee. We are constrained in our detailed findings by the fact that a 

police investigation into the unauthorised release of information is ongoing. 

11.4 Recommendations 

33. Risk management. UEA should be alert to the implications for their reputation of 

the sort of challenges we have seen in this case to the work of CRU and any other 

key groups. The risk register should reflect the range of external attitudes towards 

its key units and growing criticism or the attentions of pressure groups should be 

noted. Mitigation measures should be put in place, including increased security 

and a bias for openness and properly resourced policy on data management and 

availability. Reporting arrangements should ensure that key senior management 

are in touch with the issues and are informed quickly of problems; and response 

plans should be in place and rehearsed. These points are no doubt relevant to 

many other universities. 

34. Training for researchers. We believe that Universities should develop formal 

approaches to the training of researchers in basic software development 

methodologies and best practice, as well as best practice in the handling and 

sharing of research data. 

35. Provision of a formal metadata repository. Whilst we recognize and accept that 

CRU relies on other bodies both nationally and internationally to provide and to 
CHAPTER 11: GOVERNANCE 

104

archive basic weather station data, we believe that a formal approach to the 

storage and archiving of metadata is required. Such a repository would, for 

example, have made it far easier to respond quickly to requests for the list of 

station identifiers associated with particular CRUTEM datasets. Where a 

University is hosting a unit of such international significance, we believe that it 

should ensure funding is available for such a repository either through the research 

grant process or from central resources. 

36. Role of research sponsors. We note the recent statement by the US National 

Science Foundation (NSF)7 that, from October 2010, NSF plan to make inclusion 

of a ―Data Management Plan‖ a requirement for all research proposals. It will be 

important for such plans to recognize that in some areas of science huge volumes 

of data are created and a degree of processing and compression is inevitable 

before data suitable for storage is created. We agree that the way in which 

important research data (and the associated meta data to make that data useful) 

should be preserved, should be specified by those funding such research. Explicit 

budgetary and resource provision must be made. Sponsorship arrangements 

should include a clear statement of requirements on the extent to which such data 

should be placed in the public domain and any constraints on the timing of such 

release. The guidance from the UK Research Integrity Office (UKRIO) is helpful 

in this respect.8

 

37. Making source code publicly available. We believe that, at the point of 

publication, enough information should be available to reconstruct the process of 

analysis. This may be a full description of algorithms and/or software programs 

where appropriate. We note the action of NASA‘s Goddard Institute for Space 

Science in making the source code used to generate the GISTEMP gridded dataset 

publically available. We also note the recommendation of the US National 

Academy of Sciences in its report ―Ensuring the Integrity, Accessibility and 

Stewardship of Research Data in the Digital Age‖ that: “...the default assumption 

should be that research data, methods (including the techniques, procedures and 

tools that have been used to collect, generate or analyze data, such as models, 

computer code and input data) and other information integral to a publically 

reported result will be publically accessible when results are reported...‖. We 

commend this approach to CRU. 

38. Audit processes. It is entirely acceptable that the central functions of a University 

should set, document and disseminate the standards expected across all 

governance areas, but without necessarily mandating the precise means by which 

these will be achieved. These standards will reflect the University‘s interpretation 

of applicable law (Data Protection, Computer Misuse, Health & Safety, 

Environmental Information Regulations) and best practice. In areas such as 

Information Systems, it may well be appropriate to allow a degree of local 

autonomy. However it is then essential that robust audit procedures are in place to 

ensure that, where local solutions are implemented, these do meet fully the 

standards specified. 

 

7

 NSF Press Release 10-077 

http://www.nsf.gov/news/news_summ.jsp?cntn_id=116928&org=NSF&from=news 

8

See UKRIO  ̳Code of Practice for Research‘ Section 3.12 ―Collection and Retention of Data‖ at 

http://www.ukrio.org 
CHAPTER 11: GOVERNANCE 

105
106

ACKNOWLEDGEMENTS 

The Review Team would like to acknowledge the help and support they received 

during the conduct of the Review and in the preparation of this report. Our thanks go 

to: 

William Hardie who provided Secretariat support. 

David Walker who acted as our Project Manager. 

Kate Moffat, Adrian Dias, Helen Davison and Mike Granatt who provided 

communications support and advice, set up and handled the Review website, and 

managed the production of the report. 
107
APPENDIX 1

108

APPENDIX 1: REVIEW TEAM MEMBERS 

TEAM CV‘s

Sir Muir Russell, KCB, FRSE (Chair) 

Sir Muir Russell was Principal and Vice-Chancellor of the University of Glasgow 

from 2003 to 2009. During that period he was Convener of Universities Scotland, a 

member of the Universities UK Main Board, a Trustee of the Universities 

Superannuation Scheme, and a member of the UCAS Board. He graduated from the 

University of Glasgow in 1970 with a First in Natural Philosophy and took up a career 

in the civil service. He was appointed Permanent Secretary to the Scottish Office in 

1998, and was the first Permanent Secretary to the Scottish Executive following the 

establishment of the Scottish Parliament in 1999. He has honorary degrees from the 

Universities of Strathclyde, Glasgow and Edinburgh. He currently chairs the Judicial 

Appointments Board for Scotland. He is also a Trustee of the Glasgow School of Art, 

a Member of the Board of the Moredun Research Institute, and the Chairman of the 

Dunedin Concert Trust. 

Professor Geoffrey Boulton, OBE, FRS, FRSE 

Professor Geoffrey Boulton is Regius Professor Emeritus of Geology and former Vice 

Principal of the University of Edinburgh. His research is in the fields of glaciology, 

glacial geology and Quaternary science, and has been awarded several international 

awards and honorary degrees for his scientific work. He currently has research 

projects in Antarctica and Iceland. He has been the UK representative to the 

International Union of Geosciences and to the International Union of Quaternary 

Sciences. He is a member of the UK Prime Minister‘s Council for Science and

Technology, chairs the Advisory Board of the University of Heidelberg and is 

General Secretary of the Royal Society of Edinburgh, Scotland‘s national academy. 

He has been a member of the Councils of the Natural Environment Research Council 

and the Royal Society, a member of the Royal Commission on Environmental 

Pollution, the Scottish Science Advisory Committee and the Scottish Higher 

Education Funding Council. He was formerly Head of the Department of Geology 

and Geophysics and Provost of Science and Engineering in the University of 

Edinburgh. 

Professor Peter Clarke, FInstP, CPhys, FIET, CEng 

Peter Clarke is Professor of Physics at the University of Edinburgh. He has a B.Sc in 

Electronics Engineering (Southampton University, 1980) and a D.Phil in Particle 

Physics (Oxford 1985). He was a CERN Fellow before being appointed as a lecturer 

first at Brunel University in 1987 and then University College London in 1993. He 

was promoted to Reader and then Professor in 2001 and was Head of the Particle 

Physics Research Group between 2001-04. He moved to the University of Edinburgh 

in 2004 to take up the Chair of eScience and later become Director of the National 

eScience Centre 2006-09. 

David Eyton MA, MIoM3, CEng. 

David Eyton is Group Head of Research & Technology at BP, and was appointed in 

April 2008. He is accountable for technology strategy and its implementation across 

BP and conducting research and development (R&D) in areas of corporate renewal. 
APPENDIX 1

109

In this role, David oversees the R&D capability of the company and also sits on the 

UK Energy Technologies Institute and Science¦Business Innovation Boards. During 

his career he has held a number of Petroleum Engineering, Commercial and Business 

Management positions in the UK, Australia, Trinidad and USA. 

Professor James Norton, CDir, CEng, CITP, FIoD, FIET, 

FBCS, FRSA 

Aged fifty-seven, Jim Norton is an independent director and policy adviser. He is an 

external member of the Board of the UK Parliament's Office of Science & 

Technology (POST) and council member of the Parliamentary IT Committee 

(PITCOM). Jim is a Non-Executive Director of F&C Capital & Income Investment 

Trust plc, where he chairs the Audit & Management Engagement Committee. He is a 

Board Member and Trustee of the Foundation for Information Policy Research 

(FIPR), as well as a member of the 'Electronic Communications Expert Advisory 

Panel' for the Irish Commission for Communications Regulation (ComReg). Jim is a 

Vice-President and Trustee of the BCS – Chartered Institute for IT, an External 

Examiner for the Institute of Directors, and Chair of the IT Policy Panel for the 

Institution of Engineering & Technology. He also chairs the Steering Group for the 

Secure Software Development Partnership (SSDP) of the Technology Strategy Board. 
APPENDIX 2

110

APPENDIX 2: APPROACH AND WORK PLAN

1. This document sets out how the Review Team has approached its terms of 

reference, and the way in which it will fulfil its remit. 

2. The terms of reference are: 

―The Independent Review will investigate the key allegations that arose from 

a series of hacked e-mails from the University of East Anglia‘s Climatic 

Research Unit (CRU). The review will: 

2.1. Examine the hacked e-mail exchanges, other relevant e-mail exchanges 

and any other information held at CRU to determine whether there is any 

evidence of the manipulation or suppression of data which is at odds with 

acceptable scientific practice and may therefore call into question any of the 

research outcomes.

2.2. Review CRU‘s policies and practices for acquiring, assembling, 

subjecting to peer review and disseminating data and research findings, and 

their compliance or otherwise with best scientific practice.

2.3. Review CRU‘s compliance or otherwise with the University‘s policies 

and practices regarding requests under the Freedom of Information Act ( ̳the 

FoIA‘) and the Environmental Information Regulations ( ̳the EIR‘) for the 

release of data.

2.4. Review and make recommendations as to the appropriate management, 

governance and security structures for CRU and the security, integrity and 

release of the data it holds.‖

3. The remit requires the Review to address the specific allegations about the way in 

which CRU has handled its data, reflecting comments in the e-mail exchanges that 

have been made public. In a separate paper – Issues for examination - the Team 

has set out its initial view of the questions that need to be addressed. It will seek 

written submissions from CRU and other appropriate parts of UEA. It will also 

invite interested parties to comment on what the Issues paper covers, and to 

propose any further matters that clearly fall within the Remit and should also be 

examined. 

4. The Review‘s remit does not invite it to re-appraise the scientific work of CRU. 

That re-appraisal is being separately commissioned by UEA, with the assistance 

of the Royal Society. The Review‘s conclusions will complement that re-appraisal 

by pointing to any steps that need to be taken in relation to data, its availability 

and its handling. 

5. The Team wishes to focus on the honesty, rigour and openness with which CRU 

handled its data. It wishes to gain a proper understanding of: 

The range of data involved, and how it has been indexed and archived. 
APPENDIX 2

111

The procedures, processes and relevant protocols used to handle the data, 

recognizing that these may have changed over time as data-handling 

capacity has developed. 

The associated metadata, algorithms and codes used for analysis. 

The extent to which other independent analysis produces similar 

conclusions. 

The peer review process, examining how much was in common between 

the work of the reviewers and the reviewed. 

6. In making its analysis and conclusions, the Team will test the relevant work 

against pertinent standards at the time it was done, recognizing that such standards 

will have changed. It will also test them against current best practice, particularly 

statements of the ethics and norms such as those produced by the UK Government 

Office for Science and by the US National Academy of Sciences. These identify 

principles relating to rigour, respect and responsibility in scientific ethics and to 

integrity, accessibility and stewardship in relation to research data. This overall 

approach will allow the Team to establish a conceptual framework within which it 

can make judgements and comment about key issues such as the level of 

uncertainty inherent in all science, and the particular confidence limits associated 

with the CRU work. 

7. The police and the Information Commissioner are also considering issues in 

connection with the leaked e-mails, and the Team has established appropriate, 

continuing liaison with them. 

8. The Team‘s analysis and conclusions will include not only a view of what has 

happened in the past, but also comments and recommendations on best practice 

for the future. This will be done both at the level of CRU and the University as a 

whole, and may have broader implications for institutions undertaking scientific 

work. 

9. The Team will operate as openly and transparently as possible. It is establishing a 

website which will eventually display all of the submissions received, 

correspondence, analyses and conclusions. The aim will be to publish all received 

submissions quickly, unless there are wholly exceptional reasons to delay, for 

example legal issues. 
APPENDIX 3

112

APPENDIX 3 

The questions, criticisms and allegations raised in the „Issues for Examination‟ paper 

below are not intended to be read in isolation. Readers are referred to the Review‟s 

findings on these points as set out in the body of the report. 

ISSUES FOR EXAMINATION

1. The Review Team’s remit is set out in its terms of reference as follows: 

―The Independent Review will investigate the key allegations that arose from a 

series of hacked e-mails from the University of East Anglia‘s Climatic Research 

Unit (CRU). The review will: 

Examine the hacked e-mail exchanges, other relevant e-mail exchanges and any 

other information held at CRU to determine whether there is any evidence of the 

manipulation or suppression of data which is at odds with acceptable scientific 

practice and may therefore call into question any of the research outcomes.

Review CRU‘s policies and practices for acquiring, assembling, subjecting to peer 

review and disseminating data and research findings, and their compliance or 

otherwise with best scientific practice.

Review CRU‘s compliance or otherwise with the University‘s policies and 

practices regarding requests under the Freedom of Information Act ( ̳the FoIA‘) 

and the Environmental Information Regulations ( ̳the EIR‘) for the release of data.

Review and make recommendations as to the appropriate management, 

governance and security structures for CRU and the security, integrity and release 

of the data it holds.‖

1. The remit reflects the reaction to the e-mails that became public, much of it 

questioning and critical. In following up those questions and criticisms the Team 

stresses that it has formed no view on whether they are fair or justified. In 

formulating in its own words an expression of the issues for examination the 

Team is not adopting those issues as its own criticisms. 

2. The Team‘s approach is to distill the questions and criticisms into the broad 

questions set out below. Using its own enquiries and experience, it has added 

questions about the handling and dissemination of data, including the response to 

FOI requests. The issues addressed in the first three paragraphs of the terms of 

reference will inform recommendations for paragraph four, as to the appropriate 

management, governance and security structures for CRU and the release of data.

 

3. The Team will invite CRU and other parts of UEA to respond in writing to these 

questions, and will follow up those responses as required. The Team expect the 

CRU to provide original documentary evidence to support its responses. 

4. The Team invites those with an interest in the matter to comment on this Issues 

paper. The Team stresses that its remit does not involve re-evaluation of the 
APPENDIX 3

113

scientific conclusions of the CRU work, still less a reappraisal of the scientific 

debate about the existence and suggested causes of global warming. Please 

confine any comments on this paper to matters within the remit at paragraph 1 of 

this paper. 

5. Written responses from CRU and others are requested by 1 March 2010 to the 

address below:

Email: correspondence@cce-review.org 

Or post to, 

Climate Change E-Mails Review 

Box 18 

196 Rose Street 

Edinburgh 

EH2 4AT

ISSUES ARISING ON Para 1.1 OF THE TERMS OF 

REFERENCE 

6. The allegation of ignoring potential problems in deducing 

palaeotemperatures from tree ring data that might undermine the validity of 

the so-called “hockey-stick” curve.

7. In the late 20th century, the correlation between the tree ring record and 

instrumental record of temperature change diverges from that for the earlier 

period. The cause of this divergence does not appear to be understood. If the 

method used to deduce temperatures from tree ring proxy metrics for the earlier 

tree ring record is applied to the late 20th century tree ring series, then declining 

temperatures would be deduced for the late 20th century. It is alleged that if the 

cause of divergence between the tree ring and instrumental temperature record is 

unknown, it may have existed in earlier periods. Therefore if tree rings had 

similarly failed to reflect the warming of the early Middle Ages, they may 

significantly under-estimate the warming during the Medieval Warm Period, thus 

falsely enhancing the contrast between the recent warming and that earlier period. 

(It is this contrast that has led to statements that the late 20th century warming is 

unprecedented during at least the last 1000 years.) 

QUESTIONS TO ADDRESS: 

What method do you use to deduce palaeotemperatures from tree ring 

data? 

Does not the problem of divergence for the late 20th century record 

invalidate the deduction of tree ring palaeotemperatures for the period 

prior to the instrumental record? 

How open have you been about this issue? 

What attempts have you made to resolve it? 
APPENDIX 3

114

What is the evidence that the amplitude of warming during the Medieval 

Warm Period (MWP) is not underestimated by tree ring evidence? 

How does the tree ring evidence of the MWP compare with other proxy 

data? Have you showed how data from different sources compare or have 

you conflated them? If the latter, what is the justification? 

If tree ring proxies are removed from reconstructions, what evidence 

remains of the MWP? 

Have you been selective in utilizing tree ring evidence from Yamal in 

Siberia; and if so, what is the justification for selectivity and does the 

selection influence the deduced pattern of hemispheric climate change 

during the last millennium? 

8. The allegation that CRU has colluded in attempting to diminish the 

significance of data that might appear to conflict with the 20th century global 

warming hypothesis 

9. The CRU group, in consultation with Professor Michael Mann, is alleged to have 

systematically attempted to diminish the significance of the Medieval Warm 

Period, evidenced by an email from Mann 4th June 2003: ―I think that trying to 

adopt a timeframe of 2K, rather than the usual 1K, addresses a good earlier point 

that Peck made w/ regard to the memo, that it would be nice to try to "contain" the 

putative "MWP", even if we don't yet have a hemispheric mean reconstruction 

available that far back [Phil and I have one in review--not sure it is kosher to show 

that yet though--I've put in an inquiry to Judy Jacobs at AGU about this].‖ The use 

of the words ―contain‖ and ―putative‖ are alleged to imply an improper intention 

to diminish the magnitude and significance of the MWP so as to emphasise the 

late 20th century warming. 

QUESTIONS TO ADDRESS

What does the word “contain” mean in this context? 

What involvement have you had in “containing” the MWP?

How important is the assertion of “unprecedented late 20th century 

warming” in the argument for anthropogenic forcing of climate?

10. It is alleged that proxy temperature deductions and instrumental 

temperature data have been improperly combined to conceal mismatch 

between the two data series 

11. An attempt to hide the difficulty of combining these two data series and to mislead 

is alleged to be revealed in the following sentence in a November 1999 email from 

Professor Phillip Jones which is alleged to imply a conscious attempt to mislead: 

"I've just completed Mike's Nature trick of adding in the real temps to each series 

for the last 20 years (i.e. from 1981 onwards) and from 1961 for Keith's to hide 

the decline‖. 

QUESTIONS TO ADDRESS

What is the meaning of the quotation from the 1999 email? 
APPENDIX 3

115

How do you justify combining proxy and instrumental data in a single 

plotted line? 

What method do you use? 

12. It is alleged that there has been an improper bias in selecting and adjusting 

data so as to favour the anthropogenic global warming hypothesis and details 

of sites and the data adjustments have not been made adequately available 

13. It is alleged that instrumental data has been selected preferentially to include data 

from warmer, urban in contrast to rural sites; that the rationale for the choice of 

high/low latitude sites is poor; and that the processes by which data has been 

corrected, accepted and rejected are complex and unclear. 

QUESTIONS TO ADDRESS

What is the rationale for the choice of data stations worldwide? 

How has this choice been tested as appropriate in generating a global or 

hemispheric mean temperature (both instrumental and proxy data)? 

Describe as clearly as possible the protocols you have followed in 

selecting, correcting and rejecting data and stations. 

Has this been an orderly and objective process applied to all datasets? 

To what extent have different procedures for data of different vintages and 

different sources been unified?

What means do you use to test the coherence of the datasets? 

ISSUES ARISING ON Para 1.2 OF THE TERMS OF 

REFERENCE 

14. It is alleged that there have been improper attempts to influence the peer 

review system and a violation of IPCC procedures in attempting to prevent 

the publication of opposing ideas.

15. It is alleged that there has been an attempt to subvert the peer review process and 

exclude publication of scientific articles that do not support the Jones-Mann 

position on global climate change. A paper by Soon & Balunias was published in 

the Journal Climate Research arguing that the 20th century was abnormally warm. 

An email from Professor Michael Mann on 11th March 2003 contained the 

following: "I think we have to stop considering Climate Research as a legitimate 

peer-reviewed journal. Perhaps we should encourage our colleagues in the climate 

research community to no longer submit to, or cite papers in, this journal." The 

allegation is that journals might be pressured to reject submitted articles that do 

not support a particular view of climate change. 

16. In an email to a fellow researcher in June 2003, Briffa wrote: ―Confidentially I 

now need a hard and if required extensive case for rejecting (an unnamed paper) 

to support Dave Stahle‘s and really as soon as you can.‖

 

17. In an email to Mann on 8th July 2004, Jones wrote: "The other paper by MM is 

just garbage. [...] I can't see either of these papers being in the next IPCC report. 
APPENDIX 3

116

Kevin and I will keep them out somehow — even if we have to redefine what the 

peer-review literature is!" The allegation is of an attempt to prevent ideas being 

published and the author being prepared to subvert the peer review process for a 

journal and to undermine the IPCC principle of accounting properly for 

contradictory views. 

QUESTIONS TO ADDRESS 

Give full accounts of the issue in relation to the journal Climate Research, 

the June 2003 email, and the March 2004 email to Mann (“recently 

rejected two papers (one for Journal of Geophysical Research & one for 

Geophysical Research Letters) from people saying CRU has it wrong over 

Siberia. Went to town over both reviews, hopefully successfully. If either 

appears I will be very surprised”.

Are the first two instances evidence of attempts to subvert the peer review 

process? 

In relation to the third, where do you draw the line between rejecting a 

paper on grounds of bad science etc, and attempting to suppress contrary 

views? 

To what extent is your attitude to reviewing conditioned by the extent that 

a paper will set back the case for anthropogenic global warming and the 

political action that may be needed to mitigate it? 

What is the justification for an apparent attempt to exclude contrary views 

from the IPCC process? 

18. The scrutiny and re-analysis of data by other scientists is a vital process if 

hypotheses are to rigorously tested and improved. It is alleged that there has 

been a failure to make important data available or the procedures used to 

adjust and analyse that data, thereby subverting a crucial scientific process. 

19. It is alleged that there has been a systematic policy of denying access to data that 

has been used in publications, referring to an email from Jones to Mann on 2nd

February 2005 which contains the following: 

"And don't leave stuff lying around on ftp sites - you never know who is trawling 

them. The two MMs have been after the CRU station data for years. If they ever 

hear there is a Freedom of Information Act now in the UK, I think I'll delete the 

file rather than send to anyone. Does your similar act in the US force you to 

respond to enquiries within 20 days?—our does! The UK works on precedents, so 

the first request will test it. We also have a data protection act, which I will hide 

behind‖. 

QUESTIONS TO ADDRESS

Do you agree that releasing data for others to use and to test hypotheses 

is an important principle? 

If so, do you agree that this principle has been abused? 

If so, should not data be released for use by those with the intention to 

undermine your case, or is there a distinction you would wish to make 

between legitimate and illegitimate use? 
APPENDIX 3

117

If not, do others have reasonable access to the data at all levels and to the 

description of processing steps, in order to be able to carry out such a reanalysis? 

Can you describe clearly the data-sets and relevant meta-data that have 

been released; what has not been released and to what extent is it in 

useable form? Where has it been released? 

Where access is limited, or not possible, or not meaningful, for legitimate 

reasons please explain why? 

20. The keeping of accurate records of datasets, algorithms and software used in 

the analysis of climate data. 

21. A key concern expressed by a number of correspondents and commentators has 

been as to whether datasets, and analyses based thereon, were deleted. 

QUESTIONS TO ADDRESS 

Were formal „data dictionaries‟ kept of the data sets acquired by the CRU 

at various times from other bodies such as the UK Meteorological Office 

Hadley Centre and its equivalents around the World? 

Were comprehensive records kept of the way these various data sets were 

used, the statistical and other algorithms used in processing them, and the 

various software programmes and modules used to carry out that 

processing? 

Does a formal library of algorithms and software used by the CRU exist? 

What quality control measures were used to test the various algorithms 

and software modules developed by the CRU? 

What techniques did members of the CRU employ to ensure the integrity 

of the various applications used to process climate data? 

What policies are in place to ensure the formal archiving of data sets and 

resultant analyses for future use and review. 

ISSUES ARISING ON Para 1.3 OF THE TERMS OF 

REFERENCE 

22. Response to Freedom of Information requests. 

23. A number correspondents and commentators assert that requests under the 

Freedom of Information Act (FoIA) and the Environmental Information 

Regulations (EIR) were incorrectly denied by the University of East Anglia on 

advice from the CRU. This is the subject of a separate inquiry by the Information 

Commissioner, but does fall within the terms of reference of the Review Team. 

QUESTIONS TO ADDRESS 

What formal processes were in place both centrally and within the CRU to 

ensure fair and impartial assessment of FoIA requests? 
APPENDIX 3

118

Were there any processes in place centrally to review recommendations 

from the CRU that information should not be released? 

Over the five years to November 2009: 

o how many requests were received? 

o how many were rejected, and on what grounds? 

o how many received full release of information? 

o how many received partial release of information? 

Independent Climate Change E-Mails Review, February 2010 
APPENDIX 3

119
APPENDIX 4

120

APPENDIX 4: INDEX OF MEETINGS, INTERVIEWS, 

SUBMISSIONS, FOLLOW UP ENQUIRIES AND 

RESPONSES 

Team Meetings were held on the following dates in 2010. Confirmed notes of Team 

Meetings can be viewed at: http://www.cce-review.org/Meetings.php 

12 January 

4 February 

25 February 

20 March 

1 April 

13 April 

22 April 

28 April 

11 May 

26 May 

7 June 

11 June 

15 June 

Other Meetings and Interviews conducted. Notes of the meetings and supporting 

documentation can be accessed from: http://www.cce-review.org/Evidence.php 

18 December 2009, meetings conducted by Sir Muir Russell at the UEA. Notes were 

taken by Lisa Williams (Senior Assistant Registrar), UEA: 

Brian Summers (Registrar and Secretary), Professor Trevor Davies (PVC 

Research, Enterprise and Engagement), Professor David Richardson (Dean, 

Faculty of Science), UEA 

Stuart Holmes, (Chair, UEA Council), UEA 

Jonathan Colam-French (Director of Information Services), Iain Reeman (ICT 

Systems Director), Steve Mosley (ICT Policy Manager), UEA 

Brian Summers (Registrar and Secretary), David Palmer (Information Policy 

& Compliance Manager) and Jonathan Colam-French (Director of Information 

Services), UEA 

Alan Preece (Director of Marketing and Communications) and Annie Ogden 

(Head of Communications), UEA 

Professor Trevor Davies (PVC Research, Enterprise and Engagement) and 

Professor Philip Jones (Climatic Research Unit), UEA 

Professor Trevor Davies (PVC Research, Enterprise and Engagement) and 

Professor Keith Briffa (Climatic Research Unit), UEA 

Superintendent Julian Gregory, Norfolk Constabulary (meeting note withheld 

to avoid prejudicing ongoing police investigation). 
APPENDIX 4

121

27 January 2010, meetings conducted by Sir Muir Russell and Professor Jim Norton 

at the UEA: 

Mike Gorrill (Head of Enforcement) and David Clancy (Investigations 

Manager) of the Information Commissioner‘s Officer 

Jonathan Colam-French (Director of Information Services) and Mike Salmon 

(IT Manager to the CRU - 40% time), UEA 

Professor Philip Jones and Professor Keith Briffa, Climatic Research Unit, 

UEA 

Superintendent Julian Gregory and Andy Guy from Norfolk Constabulary 

(meeting note withheld to avoid prejudicing ongoing police investigation) 

4 March 2010, interviews conducted by Professor Peter Clarke and Professor Jim 

Norton at the UEA: 

Professor Philip Jones, Dr Tim Osborn and Ian Harris, Climatic Research 

Unit, UEA 

24 March 2010, meeting conducted by Professor Jim Norton at the Information 

Commissioner‘s Office:

Mike Gorrill (Head of Enforcement) and Steve Wood (Assistant 

Commissioner FOI) 

26 March 2010, meetings conducted by Sir Muir Russell and David Eyton at the 

UEA: 

Ian McCormick (Director of Research, Enterprise and Engagement) and Alan 

Walker (Manager of Research Services), UEA 

Rob Bell (Research Finance Management Accountant) and Laura McGonagle 

(Faculty of Science Finance Manager), UEA 

Professor Trevor Davies (PVC Research, Enterprise and Engagement), UEA 

Professor Trevor Davies (PVC Research, Enterprise and Engagement) and 

Professor David Russell (Associate Dean for Research, Faculty of Science), 

UEA 

Cecile Piper (Director of Human Resources), UEA 

Brian Summers (Registrar and Secretary), UEA 

Professor Edward Acton (Vice-Chancellor), UEA 

30 March 2010, meetings conducted by Sir Muir Russell and Professor Jim Norton at 

the UEA: 

Michael McGarvie (Science Faculty FoIA contact), UEA 

David Palmer (Information Policy & Compliance Manager) and Jonathan 

Colam-French (Director of Information Services), UEA 

9 April 2010, meetings conducted by Professor Geoffrey Boulton and Professor Peter 

Clarke at the UEA: 

Professor Keith Briffa, Professor Philip Jones, Dr Tim Osborn and Dr Tom 

Melvin (Climatic Research Unit), David Palmer (Information Policy & 
APPENDIX 4

122

Compliance Manager) and Jonathan Colam-French (Director of Information 

Services), UEA 

15 June 2010, telephone interview conducted by Sir Muir Russell and David Eyton 

with Professor Trevor Davies, PVC Research, Enterprise and Engagement 

Telephone interviews conducted by Professor Geoffrey Boulton: 

1 May, Professor Sir Brian Hoskins, IPCC AR4 Review Editor Ch 3 (Surface 

and Atmospheric Climate Changes) 

1 June, Professor John Mitchell, IPCC AR4 Review Editor Ch 6 

(Palaeoclimate). 

Follow- up enquiries, responses and related correspondence. This material can be 

accessed from: http://www.cce-review.org/Evidence.php 

Presentations from 20 March Review Team Meeting 

Independent forensic analyst report and commentary on e-mail examination 

Redacted set of FOI and EIR requests received relating to the Climatic 

Research Unit since 2005 

Correspondence with Lord Oxburgh in respect of the separate reviews 

Follow-up request to Professor Jones and response on issues relating to the 

IPCC 

Clarification request to Professor Jones and response on issues relating to the 

IPCC 

Follow-up request to Professor Jones and response in relation to raw 

instrument station availability for each CRUTEM data set 

Follow-up request to Professor Briffa and response in relation to best scientific 

practice in dissemination scientific evidence into the public domain 

Follow-up request and response on the scope of material that was backed up 

on the server that was compromised 

Follow-up request and response in relation to the UEA‘s main university email server domain 

Follow-up request and response in relation to governance and risk 

management issues 

Follow-up request and response in relation to financial controls 

Follow-up request to ICO and response in relation to the holding of 

information or data 

Response to the Science and Technology Committee‘s Eight Report on The 

Disclosure of Climate Data from the Climatic Research Unit (Note: at the time 

of writing, the Science and Technology Commons Select Committee has not 

given permission to publish this response)

Submissions

The Review was launched on 11 February and submissions were invited from this 

date and accepted up to 16 April. The Review accepted a small number of 

submissions that it deemed to have significant relevance to its remit after this close 

date. 
APPENDIX 4

123

Submissions to the Review can be viewed at: http://www.ccereview.org/Evidence.php 

Submissions Published

Submission Name Date received 

Number 

1 Nicholas Barnes 06-Mar

2 David Archer 25-Feb

3 Dr Phillip Bratby 22-Feb

4 Philip Brohan 24-Feb

5 Climatic Research Unit 01-Mar

6 David Cockroft 28-Feb

7 Gill Chant 13-Feb

8 Tor Berge S. Gjersvik 12-Feb

9 Professor Gabriele Hegerl 02-Mar

10 Michael Hughes 17-Feb

11 Professor Mike Hulme 26-Feb

12 Professor Dr Fortunat Joos 01-Mar

13 Dr D.R. Keiller 14-Mar

14 Andrew MacIntyre 12-Feb

15 Dr Ross McKitrick 26-Feb

16 Dr P.C. Matthews 01-Mar

17 Forrest M.Mims III 18-Feb

18 Andrew Montford 11-Feb

19 Professor Raymond Bradley 01-Mar

20 Jim Stathos 26-Feb

21 Professor Simon Tett 01-Mar

22 The Global Warming Policy Foundation 28-Feb

23 Stephen McIntyre 02-Mar

24 Professor Rob Wilby 26-Feb

25 Dr Austin Woods 11-Feb

26 Professor Malcolm Hughes 01-Mar

27 W.F. Lenihan 15-Feb

28 Richard Calhoun 14-Feb

29 Dr Henry Barnard 27-Feb

30 Ron Cram 15-Feb

31 Charlie Kilgore 13-Feb

32 Vincent Moran 14-Feb

33 Dr Richard North 11-Feb

34 Stephen Richards 15-Feb

35 Cameron Rose 16-Feb

36 Dr C.W. Schoneveld 15-Feb

37 David Shepherd 13-Feb

38 Geoffrey Sherrington 17-Feb

39 Martin Vermeer 27-Feb

40 Robert Wright 15-Feb
APPENDIX 4

124

41 Brent Hargreaves 16-Feb

42 Professor Michael Mann 28-Feb

43 Dr Sonja Boehmer-Christiansen 23-Feb

44 Dr Benjamin Santer 28-Feb

45 Trevor Jones 28-Feb

46 D R G Andrews 08-Apr

47 Professor Joshua Halpern 08-Apr

48 Mike Haseler 22-Feb

49 John R. Smith 17-Feb

50 Dr D.R. Keiller 17-Feb

51 Dr D.R. Keiller 17-Feb

52 Dr David Lehmiller 15-Feb

53 Dr D.R. Keiller 24-Mar

54 Bob Smith 19-Feb

55 Robert Wright 19-Feb

56 Ian MacDonald 28-Feb

57 Dr David Lehmiller 15-Feb

58 Stephen Richards 11-Feb

59 Ray Soper 17-Feb

60 David Sandeman 18-Feb

61 Dr Josep Verges 15-Feb

62 Dominic O'Kane 15-Feb

63 Vid Stimac 17-Feb

64 Chris Allen 12-Feb

65 Ian MacDonald 20-Feb

66 Tony Brown 13-Feb

67 Gerry Morrow 12-Feb

68 Stephen Graves 12-Feb

69 Will Hawkes 12-Feb

70 C. Barling 12-Feb

71 Susan Ewens 13-Feb

72 Malcolm McClure 13-Feb

73 Robert Owen 13-Feb

74 Bruce Garrick 13-Feb

75 Stephen Johnson 13-Feb

76 Jim Stathos 13-Feb

77 Dr Barry Napier 14-Feb

78 Joe Olson 16-Feb

79 Roderick Campbell 16-Feb

80 John R. Smith 16-Feb

81 Ray Soper 18-Feb

82 Julie Grace 17-Feb

83 Doug Vickers 15-Feb

84 David Shepherd 15-Feb

87 Mike Haseler 12-Feb

88 Dr D.R. Keiller 06-Mar
APPENDIX 4

125

89 Patrick Keane 17-Feb

90 Patrick Keane 18-Feb

91 Dr D.R. Keiller 20-Feb

92 Richard Calhoun 11-Feb

93 Robert Denton 18-Feb

94 Robert Denton 27-Feb

95 John Graham-Cumming 17-Feb

96 D.G. McBeath 22-Feb

97 George Mihailides 28-Feb

98 David Shepherd 11-Feb

99 Patrick Keane 15-Feb

100 Dr D.R. Keiller 23-Feb

101 Dr D.R. Keiller 26-Feb

102 Dr D.R. Keiller 28-Feb

103 Climate Scientists Letter 28-May

104 McIntyre Submission 9-Jun

105 Dr McKitrick supplementary sub 1 13-Apr

106 Dr McKitrick supplementary sub 2 13-Apr

107 Geoffrey Sherrington 4-Mar

108 Professor Tom Wigley 28-Feb

109 Patrick Keane 12-Feb

Submissions Not Published

David Holland 

Conor McMenemie (submission to be treated as confidential) 

The Review reserved the right to withhold publication of submissions if they were 

abusive, potentially defamatory, anonymous, or there were other legal difficulties 

which prevented publication. 
APPENDIX 5

126

APPENDIX 5: PEER REVIEW 

UNDERSTANDING UNCERTAINTY: A BRIEF HISTORY 

OF PEER REVIEW

By Richard Horton, Editor of The Lancet 

Amid the public and scientific furore over alleged events at the University of East 

Anglia‘s Climatic Research Unit (CRU), peer review has emerged as a central issue in 

the dispute. In the Times Higher Education, for example, Andrew Montford, author of 

The Hockey Stick Illusion: Climategate and the Corruption of Science (1), argued that 

events at the CRU had far-reaching implications for the world of scientific publishing 

(2). His charge sheet was extensive – undermining the peer-review process, 

threatening editors who published work contrary to orthodox scientific opinion, 

organising mass resignations from editorial boards, and persuading colleagues to stop 

submitting papers to allegedly offending journals. Montford suggests that ―as many as 

four different journals may have had their normal procedures interfered with‖. He 

continues, 

 “What is an ethical way to deal with a journal?...

 At what point does valid protest elide into something 

 more sinister?...If, as the [CRU] emails suggest, some 

scientists are in fact putting illegitimate pressure on 

journals, either to influence the peer-review process or 

to prevent the release of data, it is easy to see how editors 

may find it difficult to respond.”

Implicit in Montford‘s argument is that peer review is critical to the process of – and 

thereby public trust in – science. Writing in The Guardian, George Monbiot put it this 

way: ―science happens to be [a] closed world with one of the most effective forms of 

self-regulation: the peer review process.‖(3). 

The importance of peer review has been invoked by climate sceptics in other domains 

of the climate debate. Christopher Booker has challenged Dr Rajendra Pachauri, for 

example, for claiming that his Intergovernmental Panel on Climate Change (IPCC) 

included only peer-reviewed research (4). By contrast, Booker reports that a third of 

IPCC sources were newspaper articles, student dissertations, even press releases. 

Again, the suggestion is that the peer-reviewed literature is something special and 

sacred to science. One can make strong and reliable assertions if those statements are 

underpinned by peer-reviewed science. If evidence has not been peer-reviewed, it is 

next to worthless. 

In the context of the CRU, there have been claims that peer reviewers censored 
APPENDIX 5

127

evidence deliberately to exclude those findings from scientific journals. Questions 

have been raised about the line between rejecting a paper on grounds of bad science 

and rejecting it as part of an attempt to suppress contrary opinions. Peer reviewers 

have been accused of letting their politics get in the way of science. Did reviewers‘ 

concerns about the potential adverse policy and political implications of supporting 

perfectly good science at peer review override their scientific judgment? The 

suggestion is that they did. 

The House of Commons Science and Technology Committee report on the CRU (5) 

has emphasised that ―the question of the operation of peer review is going to be a 

critical issue‖ in any evaluation of the CRU‘s work. In their report, in a section 

headed ―Perverting the peer review process‖, MPs set out several specific allegations 

against CRU scientists. First, that they colluded to subvert peer review, slowing or 

blocking the publication of research which disagreed with their own views. Second, 

that climate scientists planned to ―redefine what the peer-review literature is‖ in their 

recommendations to the IPCC (6). Third, that they tried to suppress a paper on 

research fraud. And finally, that CRU staff exerted improper pressure on the editor of 

one journal, an editor who had been open to scientists challenging climate change 

orthodoxy. The House of Commons Committee concluded that there was no evidence 

to suggest that the CRU had tried to subvert the peer review process. They wrote that, 

―Academics should not be criticised for making informal comments on academic 

papers‖.

Much of the concern – and, indeed, confusion – about what took place at the CRU in 

relation to peer review may stem from misunderstandings about what peer review is 

and what it can be expected to do. 

Peer review: firewall or the weakest link? 

For scientific journals, peer review is the confidential evaluation of a submitted 

manuscript by one or more individuals who are experts in an aspect of the work under 

scrutiny. 

Who invented peer review? It‘s hard to be sure, but possibly the prize goes to Ishaq 

bin Ali Al Rahwi (AD 854-931) (7). In his book, Ethics of the Physician, Al Rahwi 

apparently encouraged doctors to keep contemporaneous notes on their patients, later 

to be reviewed by a jury of fellow physicians. But the serious business of journal peer 

review had to wait another 800 years. Henry Oldenburg, editor of Philosophical 

Transactions of the Royal Society, was the first modern editor to adopt peer review in 

the seventeenth century. He used it to famous effect, provoking often fractious, but 

illuminating, debates between scientists across Europe. 

Technology – first the typewriter, then the photocopier, and now the Internet – has 

greatly facilitated peer review. Any scientific journal that lays claim to respectability 
APPENDIX 5

128

must have a robust peer review process. At The Lancet, the process goes like this. A 

research paper is submitted electronically to a secure database and allocated by an 

editor to a colleague. The first or second editor can reject the manuscript at that early 

stage if the paper is judged to be scientifically poor, unsuitable for the journal‘s 

readership, unoriginal, or insufficiently topical. Journals differ here. For The Lancet 

around three-quarters of manuscripts are rejected at this point. 

If a paper survives preliminary editorial review, it is discussed at a pre-review 

meeting to assess its suitability for external peer review. If judged a potential 

candidate for publication, the manuscript is sent to three expert advisors, commonly 

international and representing different methodological dimensions of the research, as 

well as a statistician. There is always the risk of group-think among experts. That is, 

there is an orthodox belief about a particular subject, strongly held, which resists 

alternative perspectives. Editors try to reduce the risk of group-think by sending 

papers to different and widely dispersed reviewers, deliberately seeking or even 

provoking critical reviews (just like Henry Oldenburg). Reviewers are not referees in 

the sense that they can blow a whistle and call time on the paper. We ask reviewers to 

provide written comments for the authors, confidential comments to the editors, and a 

detailed rating for each section of the paper. Those comments are collected, presented, 

and discussed at a once-weekly manuscript meeting attended by all the journal‘s 

editors. 

At this stage, a paper can be rejected or we can open negotiations with authors. If we 

proceed, reviewers‘ questions and concerns are put to the authors, with appropriate 

guidance from editors. The authors will reply by answering each question from 

reviewers, submitting a revised manuscript that attempts to respond to all points 

raised by editors and reviewers alike. The authors may also disagree with or challenge 

reviewers with varying degrees of force. The revised paper is discussed again at a 

manuscript meeting. The options at this stage are to reject, accept, go back to the 

authors with further requests for clarification, or return to reviewers (old or new) for 

additional opinions. We proceed with further revisions of the paper until a final 

reject/accept decision is made. We know that with such a high rejection rate we may 

get it wrong. To limit errors of omission, we have a formal appeals process where 

editors promise to look again at a paper, weigh up the authors‘ arguments, and 

reconsider our decision. 

Once the paper is provisionally accepted, the peer review process is not over. The 

paper is then passed to a scientifically qualified assistant editor who edits the paper‘s 

technical content. Mistakes may still be found at this stage, leading to further editorial 

or expert review, even (though rarely) rejection. A lesson learned from sometimes 

bitter experience is that a paper is not fully accepted until it is published. 

Here are some of the commonest questions asked about the peer review process (8). 
APPENDIX 5

129

Do reviewers make mistakes in their judgments? Of course, and so do editors. 

Sadly, the scientific literature is littered with retractions of papers that once passed the 

test of peer review. Reviewers and editors are disappointingly human. 

Are reviewers objective in their judgments? Pure objectivity is impossible. For some 

subjects, an editor can predict the judgment of the reviewer based on past experience 

with that reviewer. But this misses the point of what an editor is seeking. It is not 

simply the judgment of reject/accept that an editor wants from a reviewer. That 

decision is the responsibility of the editor and the editor alone. What an editor really 

seeks is a powerful critique of the manuscript – testing each assumption, probing 

every method, questioning all results, and sceptically challenging interpretations and 

conclusions. Armed with that critique, the editors decide – and take full responsibility 

for deciding. 

Are reviewers willing to accept new ideas? Certainly, they are, although they might 

question those ideas to destruction. The vast majority of reviewers take their 

responsibility as advisors very seriously indeed. They themselves are often on the 

receiving end of peer review. Most try to be as open as possible to new findings, 

although we encourage them to ask difficult and awkward questions. There are 

occasional exceptions. For example, the world of complementary and alternative 

medicine (CAM) divides the medical community. Orthodox medicine mostly rejects 

papers about reflexology, iridology, and acupuncture treatment that invokes invisible 

pathways (meridians) of qi. CAM is served by a separate class of journals that have 

little overlap with the more mainstream medical literature. In this instance, ideas are 

incommensurable. 

Despite peer review, are authors able to get away with dishonest or dubious 

research? Yes, they are. Peer review does not replicate and so validate research. Peer 

review does not prove that a piece of research is true. The best it can do is say that, on 

the basis of a written account of what was done and some interrogation of the authors, 

the research seems on the face of it to be acceptable for publication. This claim for 

peer review is much softer than often portrayed to the general public. Experience 

shows, for example, that peer review is an extremely unreliable way to detect research 

misconduct. 

Are peer reviewers accountable for what they do? Yes, to the editor. But in a broader 

sense, to the scientific community and to the public as well. To a large extent, the 

trust society places in science depends on the scientific process, including peer review 

and publication, getting it right most of the time. 

Does peer review improve the quality of published research? In our everyday 

practice, we see that it does. And research suggests that it does too (9). Peer review 

improves discussion of the limitations of research. It emphasises uncertainty. It invites 

justification of generalisability. As one study of peer review concluded, ―peer review 
APPENDIX 5

130

is a negotiation between authors and journal about the scope of the knowledge claims 

that will ultimately appear in print‖ (9).

Is there still a need for peer review, given the extraordinary ability of the Internet to 

enable continuous open criticism of research once published (that is, surely a 

thousand readers as reviewers after publication are better than 4 reviewers selected 

by editors before publication)? There is no right answer to this question. Different 

views have been expressed. Certainly, post-publication peer review adds greatly to the 

understanding of a piece of research. But watching pre-publication peer review in 

action - both at the macro level of external expert review and the micro level of 

technical editing - and seeing the extent to which research papers change (mostly for 

the better) after peer review, I think that pre-publication review still has an important 

part to play in science. At its best, pre-publication peer review clarifies, introduces 

uncertainty, insists on placing new work in the context of the totality of available 

evidence, demands a careful explanation of limitations, and prevents flights of 

fanciful over-interpretation. 

Peer review has changed considerably during the past two decades (since I became an 

editor). First, the stakes are higher. Individual and institutional success depends on 

getting papers published in high-impact journals. Citation data are now a standard 

metric for measuring research performance. This trend has increased competition and 

rivalry for places in the best journals. Second, the globalisation of science has 

expanded the geographic range of papers submitted to journals. Research originating 

from China, for example, is now far more common than even five years ago. The 

internationalisation of science has further intensified competition for publication. 

Third, research papers are increasingly multi-disciplinary, requiring a much broader 

range of expertise during peer review. Fourth, science is a stronger part of our public 

culture now than it once was. What scientists used to write only for other scientists is 

today available to – and sometimes read by – non-scientists, policy makers, and the 

media. Fifth, the importance of statistics has grown substantially. Whereas twenty 

years ago The Lancet had no separate statistical peer review process, every paper we 

now publish has been carefully scrutinised by an independent statistical advisor. 

Editors are now far more aware of analytic errors in research. Sixth, to address the 

often conflicting results of individual research studies that are trying to answer the 

same (or a similar) question, a new type of research method has been devised – the 

systematic, as opposed to the narrative, review. Systematic reviews aim to search for 

particular types of study (eg, the randomised trial), then select only the best according 

to pre-specified criteria, and, if possible, to combine those findings in a statistically 

meaningful way (which is called meta-analysis). Examples include the risk of cervical 

cancer among women taking hormonal contraceptives (10) and the effects of a class 

of medicines on heart disease (11). In biomedicine, the Cochrane Collaboration is the 

most mature example of an effort to create a database of systematic reviews on 

treatments. 
APPENDIX 5

131

Finally, editors have had to face an upsurge in the discovery of episodes of research 

misconduct (fabrication, falsification, and plagiarism). The increasing awareness of 

research fraud had led not only to greater vigilance (hopefully not suspicion) among 

editors but also to the birth of institutional mechanisms to set standards and advise on 

research practice (eg, the Committee on Publication Ethics). 

 

Because of the faith journal editors have in peer review, together with the empirical 

evidence they believe exists to support peer review, they take it very seriously indeed 

(12). That said, editors are well aware that peer review is anything but 

uncontroversial. Scientific discoveries that later turn out to be flagrant episodes of 

dishonesty – from Woo-Suk Hwang‘s fabricated claims in Science about cloning 

embryonic stem cells, to Andrew Wakefield‘s falsifications in The Lancet – are not 

uncommon. They raise troubling questions about the robustness of peer review. 

Editors are only too well aware of the limitations of the peer-review system. Authors, 

for example, can be deeply resistant to responding to questions from anonymous 

critics (this fact at least partly drives the argument for fully transparent peer review, 

where reviewers have to disclose their names to authors). The reluctance of some 

authors – and some very famous authors, at that – to take the comments of their peers 

seriously stems from the fact that they believe they have no peers. As one historian of 

peer review put it, somewhat poetically, ―anyone who possessed the MD degree had 

no reason to defer to any colleague as an expert greater than he or she‖ (13).

So what is peer review in today‘s scientific culture? Various views have been more or 

less vividly expressed. Peer review is a ―sacred academic cow‖, according to one 

editor (14). She put it rather well: 

 “the „sacred‟ cow of peer review wanders the meadows 

of scientific publishing because together, scholars and 

editors, believe that it is the best mechanism we have to 

improve the quality of published papers...The one 

component that we cannot control is that of competition 

in the academic world, an issue that continues to circle 

in all disciplines. We all have stories to tell about the 

viciousness of academic politics...although we might 

not be able to eradicate the politics, we can at least 

understand them for what they are...”

Everyone – scientists, the public, policymakers, politicians – would like to believe 

that peer review is a firewall between truth and error (or dishonesty) (15). But as the 

editor of one leading specialist medical journal has rightly pointed out, ―There is no 

question that, when it comes to peer review, the reviewers themselves are the weakest 

(or strongest) links‖ (16). This frustration among editors and scientists that peer 

review cannot always live up to the claims sometimes made for it produces frequent 
APPENDIX 5

132

expressions of dismay. Is peer review a castle built on sand or the bedrock of 

scientific publishing (17)? Is peer review a landmark, landmine, or landfill (18)? Or, 

put bluntly, is peer review simply in crisis? (19). Is it ―a flawed process at the heart of 

science and journals‖ (20)?

Unfortunately, there is evidence of a lack of evidence for peer review‘s efficacy. In 

2002, Tom Jefferson and colleagues published a startling systematic review of all the 

evidence about editorial peer review in biomedical journals. Their exhaustive search 

yielded only a handful of studies. The conclusion? ―Editorial peer review, although 

widely used, is largely untested and its effects are uncertain‖ (21). They went on, 

―Given the widespread use of peer review and its importance, it is surprising that so 

little is known of its effects.‖ Jefferson and his colleagues have confirmed their 

observations more recently (22). Their findings have been replicated by others (23). 

To be fair, there is some evidence that micro peer review – technical editing – can 

improve papers in biomedical journals (24). But, once again, this evidence is not as 

robust as one would either like or have expected. 

Jefferson extended his investigation of peer review by arguing that the objectives of 

the review process were also unclear (25). Without clear objectives, proving the value 

of peer review (or not) would be impossible. After almost 350 years of journal peer 

review, our zeal for and confidence in the peer review process seem inversely 

proportional to our knowledge about what it actually does for science. Those who 

make big claims for peer review need to face up to this disturbing absence of 

evidence. 

Worse still, what evidence is slowly accumulating should perhaps make scientists, 

policymakers, and the public pause. Many who place great weight on the reliability of 

the peer-reviewed scientific literature believe that it reflects the judgment of the 

scientific community about the quality of research. But evidence suggests that 

acceptance of research for publication may well depend on factors other than 

scientific quality alone (26). Furthermore, peer reviewers will disagree greatly in their 

recommendations to editors about a particular research paper. Yet editors seem to be 

significantly influenced by reviewers who, when the quality of their advice is 

measured independently, turn out to be extremely unreliable in their overall 

judgments (27). Editors, some critics could reasonably argue, need to pay less, not 

more, attention to the recommendations of their peer reviewers. 

Scepticism about peer review is healthy. But every editor knows that peer review can 

be an indispensible aid to his or her work. Peer review can rescue science from

embarrassment and error. An extreme example goes some way to showing why. Peter 

Duesberg is a well-known molecular virologist who believes that HIV is not the cause 

of AIDS. In 2009, the journal Medical Hypotheses published a paper by Duesberg 

arguing that the deaths attributed to AIDS in South Africa were false. The editor of 

Medical Hypotheses operated an editorial policy of no external peer review. The 
APPENDIX 5

133

justification was that peer review might suppress creative thinking. In the case of the 

Duesberg paper, the idea that HIV does not cause AIDS was not new. More 

importantly, South Africa is only now reversing its disastrous denialist policies on 

HIV-AIDS. To consider Duesberg‘s old (and discredited) idea at a critical moment for 

the country he was writing about would, most reasonable editors would conclude, 

require some kind of external peer review to assist decision-making. The editor did 

not seek expert reviews. He accepted the paper within a few days of its submission. 

Many scientists in the AIDS community were appalled. They wrote to the publishers 

(Elsevier, also the publishers of The Lancet) to complain. Elsevier removed the paper 

from its online database pending the results of an independent investigation. The 

Lancet was asked to review the paper. We did so and the reviews were uniformly and 

deeply critical. No journal could have conceivably published the Duesberg paper 

based on these reviews. The Duesberg paper remains retracted, excised from the 

scientific literature. Here is an example of what can happen when peer review is 

excluded from a journal‘s processes, and why peer review can bring important 

information to bear on judgments about the suitability of research for publication. 

Thanks to these events, the journal will now implement peer review. The publishers 

are seeking a new editor (28). 

Climate science and peer review

The events surrounding the peer review of certain climate science papers have raised 

important questions about peer review. But some of these questions may be based on 

a misinformed view of the peer review process. Here, I touch on general themes in 

peer review that have emerged during the debate about the role of climate scientists in 

research publication. 

It is common for editors to have multiple, intense, and sometimes sharp interactions 

with authors and reviewers. Publication matters. Authors and reviewers are frequently 

passionate in their intellectual combat over a piece of research. The tone of their 

exchanges and communications with editors can be attacking, accusatory, aggressive, 

and even personal. If a research paper is especially controversial and word of it is 

circulating in a particular scientific community, third-party scientists or critics with an 

interest in the work may get to hear of it and decide to contact the journal. They might 

wish to warn or encourage editors. This kind of intervention is entirely normal. It is 

the task of editors to weigh up the passionate opinions of authors and reviewers, and 

to reflect on the comments (and motivations) of third parties. To an onlooker, these 

debates may appear as if improper pressure is being exerted on an editor. In fact, this 

is the ordinary to and fro of scientific debate going on behind the public screen of 

science. Occasionally, a line might be crossed. We experienced such a border crossing 

recently, where several reviewers and third parties encouraged us to delay publication 

of a paper for non-scientific reasons (29). Defining that line is the crucial task when 

judging the role of CRU scientists. 
APPENDIX 5

134

One issue that is important to solve for the peer review process to work effectively is 

the full disclosure of all financial and relevant non-financial conflicts of interest. If a 

research paper about drug A for disease Y is sent to a reviewer who has shares in a 

company that makes drug B, also for disease Y, there is a potential for the 

introduction of bias into that reviewer‘s advice to the journal – favouring drug B over 

drug A. The editor may still want and value that reviewer‘s advice, but s/he needs to 

know about the reviewer‘s financial conflict to judge the weight s/he gives to the 

review. Non-financial conflicts may be even more important. If a scientist has devoted 

a life‘s work to theory A about disease Y, then clearly s/he might be biased if s/he is 

sent a manuscript that criticises theory A and proposes an alternative and compelling 

theory B for that same disease. Again, the editor would expect the reviewer to declare 

any non-financial academic or intellectual conflicts that might have the potential to 

influence that reviewer‘s critique. In the field of climate research, conclusions about 

the meaning of the science may have been strongly held. The fact that these 

conclusions were strongly held is entirely to be expected and should simply have been 

fully disclosed to editors during the peer review process. 

It would be wrong for editors not to listen to advice about publication even after 

acceptance of a paper. A paper is only fully accepted when it is published. New 

information that informs the decision to publish a provisionally accepted paper before 

publication can be very valuable. The Lancet has rejected papers in this twilight zone 

of peer review. After publication, criticism is common and welcome, even lethal 

criticism. This is the much vaunted self-regulation of science – except that sometimes 

editors and authors are reluctant to act when things go wrong after publication. In 

climate science, disputatious research is unsurprising. It would be extremely 

surprising if that research did not provoke heated exchanges both before and after 

publication. 

Much has been made of whether scientists should or should not take public positions 

on the meaning of their data, especially if those data relate directly to policy or 

practice. The reality is that they do, all the time. Science does not exist in a political 

vacuum. The idea that scientists, including climate scientists, are neutral observers, 

bereft of opinions, is naïve. In biomedical and public health research, scientists are 

often quick to make statements applying their data to the real world. They will often 

do so passionately and be well known for those passionate views. Indeed, the current 

climate of science is such that scientists are encouraged at every stage of their 

research to consider the impact – economic or human – of what they do, and to 

trumpet that impact. Research assessments in the future are likely to include a 

measure of impact when judging the quality of a scientist‘s work. In relation to peer 

review, the scientific, policy, or political positions an author, reviewer, or editor may 

hold could intervene to bias a review in one particular direction. There have been 

many examples of such conflicts in other scientific disciplines – eg, psychology (30) 

and genetic epidemiology (31). These episodes are troubling, but an almost inevitable 

consequence of the way peer review is ordinarily done. 
APPENDIX 5

135

The intersection of politics and science in climate change has arisen at least once 

before. The Skeptical Environmentalist, written by Bjørn Lomborg and published by 

Cambridge University Press, led to huge pressure on the publishers to withdraw the 

book (32). Although the manuscript was reviewed by four experts who all 

recommended publication, the scientific backlash was acute. Letters of protest were 

written to newspapers. One scientist refused to work with Cambridge University Press 

ever again. Lomborg was attacked physically. 

Chris Harrison, in his thoughtful reflections as the editor at Cambridge University 

Press who dealt with Lomborg‘s book, points out that peer review ―offers no 

guarantees of always ensuring the  ̳truth‘‖ (32). But in the case of The Skeptical 

Environmentalist, the concerns were as much political as scientific. The publication of 

this book by a respected scholarly press might play ―to a particular political agenda 

and can be used and abused by vested corporate and political interests.‖ Harrison 

rejected the idea that he should have applied these kinds of value judgment in the 

editorial process. But he notes, ruefully, 

 “While it may not be the responsibility of an editor to

second-guess how a publication will be received in political 

circles, it is clear that politicians and advocacy groups have 

always maintained a keen interest in what is published and 

that they have sought to influence editorial decisions.”

Harrison defended the scholarly publishing industry‘s commitment to pluralism. He 

wrote, 

“Given the scale of interest in the environment it is 

perhaps peculiarly incumbent on the academy and 

general interest intermediaries [Cambridge University 

Press] to host as full and as open a debate as possible...the 

public and the academy can surely only be better served 

by an opportunity to review and debate a wide range 

of perspectives.”

This commitment to pluralism would be the likely view of many scientific editors, 

even when controversy follows. There are interesting parallels between the response 

of the scientific community to Lomborg‘s sceptical book and the reaction of some 

scientists to journal articles on climate change which expressed an opinion contrary to 

their own. One might conclude that these kinds of extreme debate, although difficult, 

are part of the normal fabric of scientific discourse. The question to be answered is: 

where is the line to be drawn between vigorous scientific exchange and improper 

attempts to close down debate (these two positions can be remarkably close to one 

another)? But one should also be conscious of what some observers have described as 
APPENDIX 5

136

the ―chilling‖ effect of political controversy on science. A survey of US National 

Institutes of Health scientists revealed that many engaged in self-censorship after they 

found themselves the subject of political criticism for their work (33). Political 

disagreement over science can shape not only the behaviour of scientists but also the 

future of science itself. As Joanna Kempner has noted, 

 “There is a role for democratic public engagement in 

science. The policy challenge will be to encourage this 

public voice in scientific decision-making, while enabling 

scientists to submit and conduct innovative studies, even 

when they may provoke controversy.” 

Increasingly, commercial, as well as political, interests are also intervening to threaten 

the integrity of peer review (34). 

Peer review and publication can provoke important questions about access to data. 

During the review process, reviewers may seek more information. Except in 

allegations of fraud, it would be highly unusual to provide or request raw data (even 

then, journals expect institutions to take responsibility for investigating the 

authenticity and reliability of original data). But access to data may be sought after 

publication. This is a highly contentious and unresolved issue. For example, Andrew 

Montford argues that, 

 “the more important story in terms of the conduct of 

science in this country concerns the repeated refusals of 

CRU staff to release the data and code underlying their 

global-temperature index...sceptics are universally of 

the opinion that the scientific method requires all research 

material to be released to friend and foe alike...” (2)

In the field of medicine, these issues are currently the subject of much disagreement. 

While many parties might like to see greater sharing of data, this practice remains 

unusual. The Wellcome Trust is taking an especially strong interest in data access. It 

proposes a code of conduct calling for ―maximum public access to data of public 

health importance.‖ The very fact that this proposal is being made illustrates the point 

that routine access to data is not a settled issue or a universal norm in science, as some 

claim. 

The issue of retention of records and exclusion of data in climate research is also a 

matter relevant to the peer review process and the ordinary working of journals. 

Journals do expect records to be kept for limited periods (say, 5 years, although 

journal practices vary). And they are comfortable with the exclusion of data provided 

that those exclusions – and the reasons for exclusion – are fully described, with 

appropriate sensitivity analyses being completed where necessary. 
APPENDIX 5

137

Two additional dimensions of peer review have been raised in the debate over CRU‘s 

work. One relates to confidentiality, the other to uncertainty. Editors send manuscripts 

to reviewers based on a principle of confidentiality. The author expects the editor to 

maintain a covenant of trust between the two parties. The editor will not misuse the 

author‘s work by circulating it outside of the confidential peer review process. The 

editor expects that covenant of trust to be honoured by the peer reviewer. No 

manuscript should be passed to a third party by a reviewer without the permission of 

the editor, usually on the grounds of improving the quality of the critique of the 

manuscript by involving a colleague in the review process. A disclosure to a third 

party without the prior permission of the editor would be a serious violation of the 

peer review process – a breach of confidentiality. It is also of paramount importance 

to report fully in all published scientific papers both quantitative and qualitative 

measures of uncertainty. One of the main benefits of peer review is to focus on areas 

of potential uncertainty and to ensure that those uncertainties are fully acknowledged, 

measured, and reported. 

The future of peer review

Peer review is a human process and so will always contain flaws, produce errors, and 

occasionally mislead. Given that journals are the gatekeepers of scientific publication, 

they have enormous – probably too much – influence over the reputations of 

scientists, research units, and universities. Many measures of academic success 

depend upon journal publication – promotion, tenure, grants, fame, and personal 

wealth. It is not surprising that journals, and the main decision aid used by journals 

(peer review), are the subject of constant tension and occasionally explosive 

controversy. At such moments, it is not only essential to be clear (and modest) about 

what peer review can do, but also to look for opportunities to do better. 

Journal articles are highly stylised reports of research. The linear and logical style of 

the research they report rarely presents a true or accurate picture of how a piece of 

research was done. As the Nobel laureate, Peter Medawar, put it (35) in his essay ―Is 

the scientific paper a fraud?‖ (to which he answered that it was),

 “[the scientific paper] misrepresents the processes

 of thought that accompanied or give rise to the work 

that is described in the paper...The scientific paper in 

its orthodox form does embody a totally mistaken 

conception, even a travesty, of the nature of scientific 

thought.”

Medawar‘s point was that, ―There is no such thing as unprejudiced observations.‖ To 

add insult to injury, research papers may not even fully represent the views of the 
APPENDIX 5

138

authors who completed the work (36), and when faults are found after publication 

those faults may be completely ignored in the subsequent use of that research (37). 

There are actions that the scientific community could take to improve this far from 

happy state of affairs surrounding one its foundational processes. First, there are new 

opportunities and techniques available to search out, identify, and eliminate (or at 

least reduce) unwanted bias in the peer review process (38, 39). Second, all young 

scientists should receive formal training – which they currently do not – in the 

standards and ethics expected in the peer review process (40). It is scandalous that 

peer review is simply not taken as seriously as it should be in the training of scientists. 

The result is that peer review is often idiosyncratic and sometimes unreliable, fuelling 

scientific controversies, such as that over climate science, rather than defusing those 

controversies. Strengthening the training, standards, and expectations around peer 

review would do much to make the quality of peer reviewing part of the formal 

appraisal of a scientist‘s contribution to his or her subject. There is a demand for 

training in peer review (41). And the ethical dimensions of the review process are 

now sufficiently concerning to scientists that they merit training as much as the more 

formal methodological aspects of reviewing (42). Disappointingly, existing training 

packages in peer review deliver little benefit to the quality of the peer review process 

(43-45). 

Third, the peer review process is enormously inefficient. Individual journals will 

undertake peer review and reject manuscripts that will then cycle around other 

journals until either the paper is accepted or the authors are sufficiently exhausted that 

they abandon attempts at publication. In the face of such gross inefficiencies, some 

scientific communities have tried to bring journals together to cooperate and make the 

review process not only more efficient, but also less costly on the time and energy of 

reviewers, authors, and editors (46). Alternatively, there may be intra-journal 

procedures that can be introduced to deliver more efficient peer review (47). 

Fourth, journal editors should adopt more effective methods to resolve disputes 

between authors, reviewers, and readers. Within the journal, an ombudsperson 

operating independently of the editors can be one useful way to resolve intractable 

disagreements about journal processes (48). If a dispute remains impossible to 

resolve, journal editors can take their concerns to the Committee on Publication 

Ethics, a charity that aims to set standards for journal practices, including peer review. 

Journal editors should consider using this facility more often than they currently do –

in some ways, it represents the collective wisdom of a wide range of journal editors, a 

collective wisdom that any scientific editor can draw upon in times of crisis. 

Lastly, peer review should be a subject for research in its own right. Although there is 

a small group of scientists who study peer review (a biomedical peer review congress 

is held every 4 years), that community is extraordinarily fragile when measured 

against the size and importance of the contribution peer review makes to science (49). 
APPENDIX 5

139

Historically, science funding bodies have been reluctant to invest in research on peer 

review. This reluctance is partly responsible for the present vacuum in our knowledge 

about the way scientific knowledge is constructed, reported, and discussed. One 

positive result of the debate over the role of CRU scientists in peer review might be to 

encourage funding bodies – such as the Medical Research Council and the National 

Institute for Health Research – to take the science of peer review far more seriously. 

Journals have inevitable limitations. When a paper with important policy implications 

is considered, editors can ask authors to balance their conclusions by putting the work 

in the context of existing evidence. Or we can commission an editorial that does the 

same. But a journal cannot adjudicate a public debate, and neither can conventional 

peer review. For those occasions when science meets (or clashes with) policy, there 

may be a case for referring the area of controversy to an independent body for a 

public inquiry. In the US, the model used is the Institute of Medicine, which tackles 

controversial aspects of health through its thorough and wide-ranging investigations 

and reports. There may be a case for such a body in the UK. 

The best one might hope for the future of peer review is to be able to foster an 

environment of continuous critique of research papers before and after publication. 

Many writers on peer review have made such a proposal, yet no journal has been able 

to create the motivation or incentives among scientists to engage in permanent peer 

review (50-52). Some observers might worry that extending opportunities for 

criticism will only sustain maverick points-of-view. However, experience suggests 

that the best science would survive such intensified peer review, while the worst 

would find its deserved place at the margins of knowledge. 

This process of weeding out weak research from the scientific literature can be 

accelerated through more formal mechanisms, such as the systematic review. A 

systematic approach to selecting evidence focuses on the quality of scientific methods 

rather than the reputations of scientists and their institutions. This more rigorous 

approach to gathering, appraising, and summing up the totality of available evidence 

has been profoundly valuable to clinical medicine. There may be useful lessons here 

for the IPCC. Climate sceptics and climate scientists, along with their colleagues in 

other scientific disciplines, would likely welcome this greater rigour and scrutiny. It 

would certainly promote quality and strengthen accountability to a more critical 

public (and media) with higher expectations of science. More importantly, intensified 

post as well as pre publication review would put uncertainty – its extent and 

boundaries – at the centre of the peer review and publication process. This new 

emphasis on uncertainty would limit the rhetorical power of the scientific paper (53), 

and offer an opportunity to make continuous but constructive public criticism of 

research a new norm of science. 

Finally, there are more general institutional lessons to be learned from the events that 

took place at the CRU. Peer review cannot eliminate controversy. Leaders of research 
APPENDIX 5

140

organisations – from University Vice-Chancellors to Unit Directors – might consider 

a risk assessment of the research being done under their stewardship. Mapping 

research in this way – who is doing what, where, and how? – would provide the 

institution with signals of potential controversy if the work addressed an issue of 

substantive public, policy, or political interest. This kind of awareness might enable 

scientists – and, indeed, the institution itself – to prepare better for any likely public 

disputes. Vigilance over the research being undertaken at a university could usefully 

anticipate difficulties, promote best practices, protect reputations, and build wider 

public trust and confidence in the research process. These outcomes, together with 

more realistic expectations about peer review and the processes of science, would be 

valuable consequences of the experiences at the University of East Anglia. 

1. Montford A. The Hockey Stick Illusion: Climategate and the corruption of 

science (Stacey International, 2010) 

2 . Montford A. Heated discussions. Times Higher Education March 25, 2010: 

 43-44.

3. Monbiot G. Our narrow, antiquated school system is at the root of the climate 

 email fiasco. The Guardian April 6, 2010: 25. 

4. Booker C. Climategate: a scandal that won‘t go away. The Sunday Telegraph 

 April 18, 2010: 29. 

5. House of Commons Science and Technology Committee. The disclosure of 

 climate data from the Climate Research Unit at the University of East Anglia 

 (Stationery Office, 2010). 

6. This quotation is taken from an email from Phil Jones to Michael Mann. 

7. Spier R. The history of the peer review process. Trends in Biotechnology 

 2002; 20: 357-58.

8. Hernon P, Schwartz C. Peer review revisited. Library and Information Science

Research 2006; 28: 1-3. 

9. Goodman SN, Berlin J, Fletcher SW, Fletcher RH. Manuscript quality before 

and after peer review and editing at Annals of Internal Medicine. Ann Intern 

Med 1994; 121: 11-21. 

10. Smith JS, Green J, de Gonzalez AB, et al. Cervical cancer and use of 

hormonal contraceptives: a systematic review. Lancet 2003; 361: 1159-67.

11. Jun M, Foote C, Lv J, et al. Effects of fibrates on cardiovascular outcomes: a 

systematic review and meta-analysis. Lancet 2010 

12. Green SM, Callaham ML. Current status of peer review at Annals of 

Emergency Medicine. Ann Emerg Med 2006; 48: 304-08.
APPENDIX 5

141

13. Burnham JC. The evolution of editorial peer review. JAMA 1990; 263: 1323-

9. 

14. Fitzpatrick JJ. Peer review: a 2008 report on the sacred academic cow. 

 Applied Nursing Research 2008; 21: 53. 

15. Harms M. Peer review: the firewall of science. Physiotherapy 2006; 92: 193-

94. 

16. DeMaria AN. Peer review: the weakest link. JACC 2010; 55: 1161-62.

17. Berger E. Peer review: a castle built on sand or the bedrock of scientific 

 publishing? Ann Emerg Med 2006; 47: 157-59. 

18. Balistreri WF. Landmark, landmine, or landfill? The role of peer review in 

assessing manuscripts. J Pediatr 2007; 151: 107-08.

19. Mulligan A. Is peer review in crisis? Oral Oncology 2005; 41: 135-41. 

20. Smith R. Peer review: a flawed process at the heart of science and journals. J 

R Soc Med 2006; 99: 178-82.

21. Jefferson T, Alderson P, Wager E, Davidoff F. The effects of editorial peer 

 review. JAMA 2002; 287: 2784-86.

22. Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for 

improving the quality of reports of biomedical studies. Cochrane Database 

Syst Rev 2007; 2: MR16. 

23. Richards D. Little evidence to support the use of editorial peer review to 

ensure quality of published research. Evid Based Dent 2007; 8: 88-89.

24. Wager E, Middleton P. Technical editing of research reports in biomedical 

journals. Cochrane Database Syst Rev 2008; 4: MR2. 

25. Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer 

 review. JAMA 2002; 287: 2786-90.

26. Aarssen LW, Lortie CJ, Budden AE, et al. Does publication in top-tier 

journals affect reviewer behaviour? PLoS ONE 2009; 4: e6283. 

27. Kravitz RL, Franks P, Feldman MD, et al. Editorial peer reviewers‘ 

recommendations at a general medical journal: are they reliable and do editors 

care? PLoS ONE 2010; 5: e10072. 

28. Enserink M. Elsevier to Editor: change controversial journal or resign. Science

2010; 327: 1316. 

29. Horton R. Maternal mortality: surprise, hope, and urgent action. Lancet 2010; 
APPENDIX 5

142

30. McCarty R. Science, politics, and peer review: an editor‘s dilemma. American 

 Psychologist 2002; 57: 198-201.

31. Calnan M, Smith GD, Sterne JA. The publication process itself was the major 

cause of publication bias in genetic epidemiology. J Clin Epidemiol 2006; 59:

1312-18.

32. Harrison C. Peer review, politics, and pluralism. Environmental Science and 

Policy 2004; 7: 357-68. 

33. Kempner J. The chilling effect: how do researchers react to controversy. PLoS 

Medicine 2008; 5: e222. 

34. Curfman GD, Morrissey S, Annas GJ, Drazen JM. Peer review in the balance. 

N Engl J Med 2008; 358: 2276-77.

35. Medawar PB. The Strange Case of the Spotted Mice (Oxford, 1996). 

36. Horton R. The hidden research paper. JAMA 2002; 287: 2775-78.

37. Horton R. Postpublication criticism and the shaping of knowledge. JAMA 

2002; 287: 2843-47.

38. Bornmann L, Mutz R, Daniel H-D. How to detect indications of potential 

sources of bias in peer review. J Informatics 2008; 2: 280-87.

39. Ross JS, Gross CP, Desai MM, et al. Effect of blinded peer review on abstract 

acceptance. JAMA 2006; 295: 1675-80.

40. Walbot V. Are we training pit bulls to review our manuscripts? J Biol 2009; 8: 

24.

41. Snell L, Spencer J. Reviewers‘ perceptions of the peer review process for a 

medical education journal. Med Educ 2005; 39: 90-97.

42. Resnik DB, Gutierrez-Ford C, Peddada S. Perceptions of ethical problems 

with scientific journal peer review. Sci Eng Ethics 2008; 14: 305-10. 

43. Schroter S, Black N, Evans S, et al. Effects of training on quality of peer 

review. BMJ 2004 doi: 10.11136/bmj.38023.700775.AE. 

44. Callaham ML, Tercier J. The relationship of previous training and experience 

of journal peer reviewers to subsequent review quality. PLoS Medicine 2007; 

4: e40. 

45. Schroter S, Black N, Evans S, et al. What errors do peer reviewers detect, and 

does training improve their ability to detect them? J R Soc Med 2008: 101: 

507-14.
APPENDIX 5

143

46. Saper CB, Maunsell JHR. The Neuroscience Peer Review Consortium. Brain

Res 2009; 1272: 1-2.

47. Johnston SC, Lowenstein DH, Ferriero DM, et al. Early editorial manuscript 

screening versus obligate peer review. Ann Neurol 2007; 61: A10-12. 

48. Horton R. The journal ombudsperson: a step toward scientific press oversight. 

JAMA 1998; 280: 298-99. 

49. Linkov F, Lovalekar M, LaPorte R. Scientific journals are ―faith based‖: is 

there science behind peer review? J R Soc Med 2006; 99: 596-98.

50. von Segesser LK. Peer review versus public review – new possibilities of online publishing! Interactive Cardiovascular and Thoracic Surgery 2002; 1:

61-62.

51. Mandviwalla M, Patnayakuni R, Schuff D. Improving the peer review process 

with information technology. Decision Support Systems 2008; 46: 29-40. 

52. Liesegang TJ. Peer review should continue after publication. Am J Ophthalmol

2010; March; 149: 359-60. 

53. Horton R. The rhetoric of research. BMJ 1995; 310: 985-88.

Commentary on „Understanding uncertainty:a brief history of 

peer review’

Elizabeth Wager, Publications Consultant 

Chair of the Committee on Publication Ethics (COPE) 

Richard Horton suggests that  ̳If evidence has not been peer-reviewed, it is next to 

worthless‘. Earlier commentators have gone even further. The evolutionary biologist 

Edward Wilson commented that  ̳a discovery does not exist until it is safely reviewed 

and in print‘.1 

Peer review has been used to select articles for publication since the 17th century. It is 

likely that when Oldenburg was using peer review at London‘s Royal Society, his 

contemporaries at the Royal Society in Edinburgh and across the channel at the 

Académie Royale de Médicine were using similar systems.2 

Interestingly, the inability of peer review to guarantee scientific integrity was noted as 

early as 1785 when the Committee of Papers of the Literary and Philosophical Society 

of Manchester noted that  ̳Responsibility concerning the truth of facts, the soundness 

of reasoning ... [and] the accuracy of calculations is wholly disclaimed; and must rest 

alone, on the knowledge, judgement, or ability of the authors‘.3 

Horton describes the peer review system used at The Lancet. This is typical of major 

medical journals that employ several full-time editors. Such journals reject a 
APPENDIX 5

144

considerable proportion of submissions on the basis of in-house review alone. For 

example, at JAMA (the journal of the American Medical Association) only 36% of 

submissions are sent for external review and the remaining 64% are rejected after 

review by the journal editors alone.4 At journals such as The Lancet and JAMA, 

everything accepted for publication will have been reviewed by experts who are 

independent of the journal, but many articles will be rejected without review by 

external experts. In contrast, journals that do not have large editorial offices or fulltime editors, but whose editors are academics (generally fulfilling this role in addition 

to their regular job), generally send virtually all submissions for external review. 

The Lancet generally sends papers to three reviewers. Other journals may use fewer, 

or occasionally more, reviewers. A survey of 200 journals from a range of disciplines 

found that 73% used two reviewers, 18% used three, 6% used one and 3% used more 

than three.5 

Horton notes that  ̳the scientific literature is littered with retractions of papers that 

once passed the test of peer review‘. The biomedical database Medline (which 

includes over 19 million citations) currently contains nearly 1500 retractions. There 

have also been well-documented cases of journals failing to recognise important 

work. There is even a website devoted to accounts of journals that have rejected work 

that later led to their authors winning the Nobel prize. 

[ttp://www2.uah.es/jmc/nobel/nobel.html#reje]. 

Churchill‘s famous observation about democracy has often been applied to peer 

review, namely that it is the worst system for selecting papers except for all the other 

forms that have been tried. Horton mentions the possibility of alternative systems 

such as post-publication or public review. A few journals have experimented with 

these systems but found them unworkable. When Nature ran a trial of open peer 

review in 2006 (by posting submitted papers on its website), almost half the papers 

received no comments.6 

While formal evidence for the effectiveness of peer review is lacking, there is 

extensive anecdotal evidence from both editors and authors of its benefits. Editors and 

publishers should strive to optimise their peer review systems and to reduce potential 

harm or bias. Guidance is available from several organizations including the 

Committee on Publication Ethics (www.publicationethics.org). 

While the internet has not produced dramatic new ways of assessing articles 

submitted to journals, information technology does offer possibilities which could 

increase the efficiency of disseminating and verifying scientific data. Gene sequence 

data is now routinely entered onto public databases, and such data posting is a 

requirement for publication in traditional journals in some cases. However, before 

such posting could be practical, even for such homogeneous data as gene sequences, 

agreements had to be developed (e.g. MIAME).7

 Such standardization may be 

possible for other types of data yet, as Horton notes, in many disciplines, data sharing 

remains either controversial, or limited by technological, practical or ethical issues.8

Even with increased research and investment it seems unlikely that, despite its 

shortcomings, conventional peer review will be replaced or transformed in the next 

few years. In the meantime, researchers will probably continue to criticise the peer 
APPENDIX 5

145

review system, sometimes with justification (because the system is flawed) and 

sometimes without (because researchers are human, feel strongly about their work and 

do not accept that some rejections are justified). The research community should 

listen to the criticisms of peer review and work towards fixing the remediable faults 

and correcting the inevitable errors. Editors and publishers should continue to work 

hard to prevent and detect misconduct by authors, reviewers and editors and should 

have systems in place for responding appropriately when misconduct occurs. 

However, judgement must be exercised to ensure that resources are wisely spent and 

care should be taken to avoid overzealous responses to individual cases or 

disproportionate reactions to rare problems. 

References 

1. Wilson EO. Consilience: The Unity of Knowledge. Knopf, New York, 1998 

2. Wager E. Peer review in science journals: past, present and future. In Holliman R et 

al (eds) Practising Science Communication in the Information Age. Oxford University 

Press, Oxford, 2009 

3. Kronick DA. Peer review in 18th-century scientific journalism. Journal of the 

American Medical Association 1990;263:1321-2 

4. Fontanarosa P, DeAngelis C. To JAMA peer reviewers and authors – thank you. 

Journal of the American Medical Association 2010;303:780 

5. Current Practice in Peer Review. ALPSP, EASE, ALSSS. www.alpsp.org 

6. Anon Overview: Nature‘s peer revew trial, 2006 

www.nature.com/nature/peerreview/debate/nature05535.html 

7. Brazma A. Minimum information about a microarray experiment (MIAME) --

toward standards for microarray data. Nature Genetics 2001;29:365-71

8. Wager E. Publishing clinical trial results: the future beckons. 

PLoS Clinical Trials 2006;1:e31 

Competing interests statement 

Elizabeth Wager is a freelance publications consultant who provides consulting, 

training, and editorial services mainly to pharmaceutical companies, publishers and 

academic institutions. She chairs the Committee on Publication Ethics (but has written 

this commentary in her individual capacity). She was part of the team that produced 

systematic reviews on the effects of peer review and technical editing cited by 

Richard Horton. She received a fee for producing this commentary. 
APPENDIX 6

146

APPENDIX 6: DATA MINING - ACCESS TO THE 

ORIGINAL CRU E-MAIL ARCHIVE

1. Recognising that the e-mails improperly released into the public domain represent 

only a tiny fraction (less than 0.3%) of the e-mails archived by the key individuals 

in the CRU, the Review team sought to set these in context. The backup server 

(CRUBACK3) had been taken as evidence by the police as part of their own 

investigation and was held by police contracted forensic investigators. A full 

context could only be established by some form of access to the information held 

on this server. In seeking to gain this access a number of legal issues arose, 

notably that: 

the server and its contents were evidence in the continuing police 

investigation; and 

in the opinion of UEA‘s legal advisers, unconstrained access to the contents of 

e-mails on the server by the Review would raise potential privacy and data 

protection issues. 

2. The compromise eventually reached with both the police investigative team and 

the UEA Registrar was for: 

the University to contract an independent computer forensic analyst; 

the police forensic consultants to extract from CRUBACK3 all the e-mails 

from the various archived mailboxes of key UEA staff and to provide these 

under strict security conditions to the independent forensic analyst; 

the independent analyst, respecting the high evidential security requirements 

set out by the police team, to work within secure premises authorized by the 

police; 

the independent analyst to seek to determine the search or selection parameters 

that had extracted the improperly released e-mails from the multiple archives 

and to determine whether this process highlighted any additional material of 

relevance to the Review; and 

any material identified by the the analyst to be redacted by the University, in 

terms of protecting the identity of non-UEA recipients or authors, prior to 

being made available to the Review. 

3. This whole process took an extended period to negotiate and implement. It 

became clear that a full analysis would require considerable further time and 

extensive manual intervention. It would introduce significant delay to the 

publication of the Review‘s report. A decision was reached not to pursue this 

further on grounds of both time and cost against likely results. The Review had 

always regarded the e-mails as pointers to areas for detailed investigation and this 

had been complemented by extensive public requests for submissions and any 

other information in the public domain. A summary report by the independent 

forensic analyst has been placed on the Review website. 
APPENDIX 6

147

4. The Review‘s own analysis of those e-mails already in the public domain is 

highlighted in the following charts, including a  ̳heat map‘ of correspondence 

intensity by month. 

5. The selected e-mails relate largely to controversial issues, although they do not 

appear to have been selected on the basis of a simple word search, as indicated in 

the table below which shows the number of e-mails in which interesting words (or 

a component thereof) occur. Some of the more infamous e-mails contain words 

such as trick or hide and these occur comparatively infrequently:- 

1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

Jan 2 5 2 1 1 1 3 9 32 9 16 26 10

Feb 5 5 3 7 1 2 9 11 48 4 5 4

Mar 1 2 4 4 7 5 11 5 5 21 6 3 8

Apr 1 1 10 2 4 8 6 11 4 9 5 3

May 1 1 12 2 4 1 6 4 14 9 6 24 7

Jun 2 2 2 6 1 5 1 19 6 10 4 6 10 4

Jul 2 2 6 4 6 11 4 30 29 3 5 18

Aug 4 2 1 3 8 4 2 8 7 16 12 4 5 8

Sep 3 6 13 14 6 2 7 4 3 11 4 4 22

Oct 4 4 12 8 8 3 20 8 6 1 10 35

Nov 3 4 2 4 3 1 7 3 3 3 4 6 8

Dec 3 1 5 3 2 2 14 9 16 9

Totals 22 19 46 69 53 47 33 96 73 144 157 75 112 127

Key Word Component Number Key Word Component Number

Briffa 580 error 178

Jones 563 wrong 173

Mann 489 Santer 173

issue 470 fund 150

differ 460 difficult 136

temperature 453 bad 134

good 450 bias 127

review 376 (sk)eptic 124

agree 368 contain 112

IPCC 357 station 106

problem 354 doubt 105

Osborn 347 anomal(y) 101

correct 308 Hulme 101

tree 307 peer 100

miss(ing) 305 avoid 100

author 290 Yamal 100

understand 251 reject 75

concern 248 trick 65

Jansen 212 hockey 60

proxy 207 decline 49

Overpeck 196 hid(e) 48

critic 188 delete 44

argu(ment) 188 mislead 37

Wigley 181 diverge 32
APPENDIX 6

148

6. The periods of greatest intensity generally coincide with repeated selection 

of embedded e-mails having the same core title – with 4 or more core title 

repetitions being viewed as significant in the table below. 

Date Core Title Repeats Subject

Oct-98 climate of the last millennia... 4 Meeting to discuss multi-proxy data 

prompted by Jones 1998 1 

May-99 Straight to the Point 4 Perceived criticism of Briffa 19992

Sep-99 IPCC revisions 7 Making the case for tree ring proxies in 

IPCC TAR Chapter 2

Apr-02 Your letter to Science 5 Criticism of draft letter Mann 20023

Mar-03 Soon & Baliunas 9 Criticism of Soon 20034

Jun-03 Review- confidential 4 Validity of proxy-based temperature 

reconstruction

Oct-03 draft 4 of proposed EOS response to Soon 20034

Feb-04 Stephen McIntyre 4 Requesting Russian tree ring data

Jan-05 [Wg1-ar4-ch06] IPCC last 2000 years 

data

5 Drafting of IPCC chapter

Apr-05 WG1 LA2 meeting - Overlap cluster 

A

5 Coordination between IPCC Lead Authors

Jul-05 MWP figure 5 for IPCC report

Dec-05 HadCRUT2v 4 Missing Southern Pole data

Feb-06 some figures at last! 4 Proxy reconstruction for IPCC AR4

Mar-06 latest draft of 2000-year section text 7 IPCC Chapter 6 in the SOD for the AR4

May-06 Wahl & Ammann paper 4 Use of Wahl 20065

in the SOD for the AR4

Jul-06 Special instructions/timing adjustment 4 Proxy reconstruction for IPCC

Sep-06 No Subject 4 Significance of recent temperatures in IPCC 

report

Jan-07 not so fast 4 J. Eddy figure in IPCC report

Apr-07 FYI 4 Climate Audit attacks on Wang and Jones

Dec-07 sorry to take your time up, but really 

do need a scrub of this 

singer/christy/etc effort

11 Response to Douglas 2007 6

Jan-08 Update on response to Douglass et al. 5 Publication options to rebut Douglas 20076

May-08 EA 21389 - Probabilistic information 

to inform EA decision making on 

climate change impacts - PCC(08)01

4 Tender pack for Environment Agency bid

May-08 JOC-08-0098 - International Journal 

of Climatology

4 Review of Santer 20087

May-08 Our d3* test 4 Validation of climate models

Jan-09 Temperatures in 2009 5 Prognosis for 2009 and UHI effect

Jun-09 2009JD012960 (Editor - Steve 

Ghan):Decision Letter

4 Review of McLean 2009 8

Jul-09 ENSO blamed over warming - paper 

in JGR

14 Response to McLean 2009 8

Sep-09 attacks against Keith 5 Selection of tree-ring data

Oct-09 BBC U-turn on climate 10 Recent 'lack' of warming
APPENDIX 6

149

References 

1. P.D. Jones, K.R. Briffa, T.P. Barnett and S.F.B. Tett, ―High-resolution 

palaeoclimatic records for the last millennium: interpretation, integration and 

comparison with General Circulation Model control-run temperatures‖, The 

Holocene 8, 4 (1998), pp. 455–471

2. KR. Briffa and T.J. Osborn, ―Seeing the wood for the trees‖, Science Vol. 284, 7 

May 1999, pp. 926-927 

3. M.E. Mann and M.K. Hughes, ―Tree-ring chronologies and climate variability‖, 

Science, Volume 296, Issue 5569, 3 May 2002, pp. 848-849,

4. W. Soon, S. Baliunas, ―Proxy climatic and environmental changes of the past 

1000 years‖, Climate Research Volume 23: 89-110, January 2003 

5. E.R. Wahl, D.M. Ritson, C.M. Ammann, ―Reconstructing Past Climate from 

Noisy Data‖, Science 312, 529 (2006) 

6. D.H. Douglass, J.R. Christy, B.D. Pearson and S.F. Singer, ―A comparison of 

tropical temperature trends with model predictions‖, International Journal of 

Climatology, (2007) 

7. B.D. Santer, P.W. Thorne, L. Haimberger, K.E. Taylor, T.M.L. Wigley, J.R. 

Lanzante, S. Solomon, M. Free, P.J. Gleckler, P.D. Jones, T.R. Karl, S.A. Klein, 

C. Mears, D. Nychka, G.A. Schmidt, S.C. Sherwood, and F.J. Wentz, 

―Consistency of modelled and observed temperature trends in the tropical 

troposphere‖, International Journal of Climatology, 28: 1703–1722 (2008) 

8. J.D. McLean, C.R. de Freitas, and R.M. Carter, ―Influence of the Southern 

Oscillation on tropospheric temperature‖, Journal of Geophysical Research, Vol 

14, D14104 (2009) 
APPENDIX 7

150

APPENDIX 7: LAND STATION TEMPERATURE DATA 

1. Contents 

This Appendix supports Chapter 6 of the report addressing land temperature data. It 

contains further information on two issues: 

the Trial Analysis, referred to in Sections 6.3 and 6.4 of Chapter 6, and 

the particular issue of matching station identifiers, referred to in Section 6.5.1 

of Chapter 6. 

2. The Trial Analysis 

2.1 Introduction 

The Review has conducted a trial analysis to demonstrate what an independent 

researcher is able to do, using publicly available land station temperature information, 

should they wish to replicate the CRUTEM analysis. 

We have only carried out each step as far as necessary to demonstrate proof of 

principle and to compare results at a level sufficient to inform our findings in respect 

of allegations made against CRU. It is important to note that we have not carried out 

all of the steps with the necessary degree of scientific rigour needed to form scientific 

conclusions, and we emphasise that no such conclusions should be drawn from this 

work. 

We have performed the following steps: 

1. Obtained raw station temperature data from two repositories as ASCII files. 

2. Written computer code in C++ to read this data into memory in a suitable 

format. 

3. Written computer code to process this data and create gridded global average 

temperature trends. 

4. Compared the results to those of CRU and others in order to inform our 

findings in respect of the allegations made against CRU. 

We have not carried out the full process of homogenising all data. There is no 

fundamental barrier to doing so. It is a time consuming step and requires the 

experience and expertise of a climate research scientist to process each of the raw 

station data entries before averaging. We comment upon the significance of 

homogenisation below. 

2.2 Primary Station Data 

Primary station data was obtained from: 

NCAR:http://www.ncar.ucar.edu/tools/datasets/ 

WWR: http://dss.ucar.edu/datasets/ds570.0/ 

GHCN: http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php. 
APPENDIX 7

151

The GHCN data was freely available. GHCN provided several files including a list of 

stations and locations, a data set of unadjusted data, and a dataset of adjusted data. 

This contained 7280 stations in the unadjusted file and 4771 in the adjusted file. This 

data set contained in some cases several duplicates from the same station. 

For NCAR it was necessary to register (free) first. The NCAR data was a single file 

containing all necessary information and contained 4646 stations. 

The formats of each file were easy to understand. The formats were different from 

each other. 

2.3 Code to Process the Data 

Code was written in C++ to: 

Parse the source files and assemble the information into computer memory. 

Pass over each station and calculate the monthly ―normals‖ defined as the 

average over a set period. In this case we required good data from at least 10 

years in the period 1955-1995. This is larger than was used by CRUTEM in 

order to obtain more valid normals without having to resort to external 

records. 

In the case of GHCN, choose the duplicate with the most valid monthly 

normals. 

Calculate the anomaly for each monthly measurement for each station. The 

anomaly is defined as the monthly temperature minus the normal for that 

month. 

Either choose to use all stations, or only those matched to the CRUTEM3 

station list published in 2009. 

Pass over the data set and assemble average annual anomalies in each cell of a 

5x5 degree grid of latitude and longitude. 

Calculate the overall annual average anomaly by averaging over grid cells. 

Do this for each of GHCN (unadjusted), GHCN (adjusted) and NCAR, and 

plot the annual average anomaly as a function of time. 

Separately read the published CRUTEM3 product and form the same average 

over grid cells and plot this on the same figure for comparison. 

This code was straightforward to write from scratch. The minimal code needed to read 

and process the data sets amounts to only a few hundred executable lines and took 

about 2 days to develop. 

2.4 Results of Global Temperature Trends 

Figure A1 - A4 show the comparison of the time series obtained from the three data 

sets GHCN (unadjusted), GHCN (adjusted) and NCAR. A simple 5 year smoothing is 

applied. Also shown is the result obtained from the CRUTEM3 5x5 gridded product. 
APPENDIX 7

152

The set of input stations used for each line is not identical - the numbers are GHCN 

(unadjusted) [7280], GHCN (adjusted)[4771], NCAR [4646]. Of these, all stations 

which have a valid monthly normal and valid monthly data are included in Fig A1. 

In Fig A2 we have approximately matched the stations in each dataset to those used in 

CRUTEM3. This is described in more detail below. The numbers matched are GHCN 

(unadjusted) [3722], GHCN (adjusted) [2962], NCAR [1957]. CRUTEM itself used 

4138 stations. 

Figure A3 is the same as A2 but showing only the two lines which correspond to 

unadjusted data (GHCN and NCAR). 

Figure A4 is the same as A2 but showing only the two lines which correspond to 

adjusted data (GHCN (adjusted) and CRUTEM). 

Figure A5 reproduces a comparison of different studies taken from the IPCC 4th

Report Chapter 3. 

Figure A1: Temperature anomaly time series created by the Inquiry Team‘s own trial analysis using a 

5x5 grid with 5 year smoothing. Shown are results obtained from GHCN (blue), GHCN-adjusted 

(yellow) and NCAR (green). Also shown is the CRUTEM3 line (black). The Y-axis is degrees x 10. 

The X-axis is year. 
APPENDIX 7

153

Figure A2: Same as Fig A1 but with approximate match of stations to those used in CRUTEM3. Axes 

same as Fig A1. 

Figure A3: Same as Fig A2 but showing only the unadjusted lines (GHCN and NCAR) Axes same as 

Fig A1. 

Figure A4: Same as Fig A2 but showing only the adjusted lines (GHCN (adjusted) and CRUTEM). 

Axes same as Fig A1. 
APPENDIX 7

154

Figure A5: Taken from IPCC 4th Report Chapter 3.

The exercise and comparison of all figures demonstrates that: 

1. Any independent researcher may freely obtain the primary station data. It is 

impossible for any group to withhold data. 

2. It is impossible for any group to tamper improperly with data unless they have 

done so to the GHCN and NCAR (and presumably the NMO) sources 

themselves. 

3. The steps needed to create a temperature trend are straightforward to 

implement. 

4. The computer code necessary is straightforward to write from scratch and 

could easily be done by any competent programmer. 

5. The shape obtained in all cases is very similar: in other words if one does the 

same thing with the same data one gets very similar results. 

6. The result does not depend significantly on the exact list of stations.

7. Adjustments make little difference. 

By performing this simple test one determines easily that the results of the CRUTEM 

analysis follow directly from the published description of the method, and that the 

resultant temperature trend is not significantly different from the other results 

regardless of stations used or adjustments made. The test is therefore sufficient to 

demonstrate that, with respect to the declared method, the CRUTEM analysis does not 

contain either error or adjustments which are responsible for the shape of the resultant 

temperature trend. 
APPENDIX 7

155

A researcher can evidently produce a study which would test the CRUTEM analysis 

quite precisely, without let or hindrance from CRU.

We do not make any statement regarding the correctness of any of these results in 

representing global temperate trends. We do not address any alleged deficiencies 

such as allowance for non climatic effects or the significant drop in station number 

post 1991. We do not address any possible deficiencies of the gridding method. These 

are entirely matters for proper scientific study and debate and lie outside the scope of 

this Review. 

2.5 Discussion of Adjustments (Homogenisation) 

In our trial analysis the GHCN and NCAR lines are without homogenisation 

adjustment. The CRUTEM3 and GHCN (adjusted) lines are both after 

homogenisation adjustments. The adjustments to CRUTEM and GHCN are made in 

different ways by different people. Even so the results are extremely similar. It is 

evident from this comparison alone that neither CRU nor those assembling the GHCN 

(adjusted) dataset have made inappropriate adjustments in order to bias or falsify the 

global temperature trend simply because those adjustments have no significant effect 

on the global average. Nevertheless we consider additional evidence that the 

adjustments made have no significant effect on the global temperature trend. 

In the CRUTEM3 product the homogenisation adjustments are (i) made to only 298 

(10%) of the stations and (ii) are known to be approximately evenly distributed about 

zero. Thus they would not be expected to have a large effect in the global average. 

CRU have submitted detailed evidence of this (p49 and p54 of their submission) 

reproduced here as figure A6 and A7. Figure A6 shows a comparison of CRUTEM3 

with the results obtained by (i) not applying adjustments to the 298 stations and (ii) 

leaving out the 298 adjusted stations. There is no significant difference as expected. 

Figure A7 shows the CRUTEM3 analysis compared to unadjusted results from other 

groups. Again no significant difference is seen. This is fully in accord with the results 

of the trial analysis. 

Comparison of all of these plots demonstrates that adjustments which have been made 

are largely immaterial to the overall shape of the temperature series and are not on 

their own the cause of the rising temperature trend in recent years. 

We have not addressed the opposite question of whether the adjustments made are 

adequate to allow for non climatic effects. This is entirely a matter for proper 

scientific study and debate and outside the scope of this review. 
APPENDIX 7

156

FIGURE A6 CRUTEM3: Removing Adjustments 
APPENDIX 7

157

FIGURE A7 CRUTEM3: Comparison with unadjusted results from other groups 

3. Matching Station Identifiers 

We have used the information supplied by CRU from its website 

http://www.cru.uea.ac.uk/cru/data/landstations/. This lists the 4138 stations used in 

CRUTEM3. We have attempted to match these stations to those appearing the GHCN 

and NCAR data sets. 

This process is not straightforward (at least was not straightforward to us as non 
APPENDIX 7

158

climate scientists). Problems arose through 

Lack of uniqueness of station identification numbers; 

Different identifiers appearing for the same station in some cases; 

Different station names in a few cases (spelling, order of words); 

Locations differing by a few degrees; 

Different conventions for location; 

Ambiguity due to duplicates in GHCN 

We found it impossible to match stations purely from identifiers. Instead we used a 

combination of conjunctions of matching latitude and longitude, the first letters of 

station names, and station identifiers. We were in the end able to match the numbers 

shown in the table. 

Number in CRUTEM3 Matched in GHCN Matched in NCAR

4138 3722 (90%) 1957 (47%)

The number matched in GHCN (90%) is in accord with that claimed by CRU in their 

written responses. We have presumed that the remaining data is available from 

National Meteorological Offices or as part of more recent update messages. 

The Review feels that it has pursued this enough to understand that (i) the matching of 

meta data (lists of stations) across sources is not straightforward and (ii) obtaining the 

final 10% of the primary data would require some additional effort (although anyone 

familiar with the field would presumably have a better knowledge of how to do this). 

This is a secondary issue to this Review, but we make the following observations 

(i) It would benefit the global climate research community if a standardised 

way of defining station metadata and station data could be agreed, 

preferably through a standards body, or perhaps the WMO. As example an 

xml based format which would make the interpretation, use, comparison, 

and exchange of data much more straightforward. 

(ii) Without such standardisation there will remain residual problems in 

issuing unambiguous lists, and assembling primary data from them. We 

feel it would be in the public interest if CRU and other such groups 

developed a process to capture and publish a snapshot of the data used for 

each important publication. 
APPENDIX 7

159
APPENDIX 8

160

APPENDIX 8: SOFTWARE AND DATA STANDARDS 

Software Standards - The available guidance on software development methodologies 

is very extensive ranging from  ̳formal methods‘, where the desired properties of the 

system are expressed in a mathematically formal language, through to the  ̳Unified 

Process‘ developments of this decade. The UK Royal Academy of Engineering has 

published extensively in this area1

. 

Data Management and Archiving Guidance is also available on Records Management 

and archiving from: 

the Joint Information Systems Committee (JISC) of the Higher Education 

Funding Council2

; 

the UK Research Integrity Office (UKRIO)3; and 

the US National Academy of Sciences report ―Ensuring the Integrity, 

Accessibility and Stewardship of Research Data in the Digital Age‖4. We note 

in particular Recommendation 2: “Research institutions should ensure that 

every researcher receives appropriate training in the responsible conduct of 

research, including the proper management of research data in general and 

within the researcher‟s field of specialization. Some research sponsors 

provide support for this training and for the development of training 

programs”. We also note Recommendation 5: “All researchers should make 

research data, methods, and other information integral to their publicly 

reported results publicly accessible in a timely manner to allow verification of 

published findings and to enable other researchers to build on published 

results, except in unusual cases in which there are compelling reasons for not 

releasing data. In these cases, researchers should explain in a publicly 

accessible manner why the data are being withheld from release”.

Information Security - There is again very extensive guidance available of 

information security issues including: 

from a standards perspective ISO/IEC 27002; 

for thought leadership the ―Jericho Forum‖5and the ―Information Security 

Forum‖6

; and 

from a professional perspective the ―Institute of Information Security 

Professionals‖7

 

1

 http://www.raeng.org.uk/news/publications/list/reports/Engineering_values_in_IT.pdf 

 http://www raeng.org.uk/news/publications/list/reports/Complex_IT_Projects.pdf 

2

 http://www.jiscinfonet.ac.uk/partnerships/records-retention-he/managing-research-records 

3

See UKRIO  ̳Code of Practice for Research‘ Section 3.12 ―Collection and Retention of Data‖ at 

http://www.ukrio.org 

4

 http://www.nap.edu/catalog/12615 html 

5

 http://www.opengroup.org/jericho/ 

6

 https://www.securityforum.org/ 

7

 https://www.instisp.org/SSLPage.aspx