1

Report of the International Panel set up by the Uni versity of East Anglia to

examine the research of the Climatic Research Unit.

Introduction

1. The Panel was set up by the University in consultat ion with the Royal Society to assess the integrity of the research published b y the Climatic Research Unit in the light of various external assertions. The Un it is a very small academic entity within the School of Environmental Sciences. It has three full time and one part time academic staff members and about a do zen research associates, PhD students and support staff. The essence of the criticism that the Panel was

asked to address was that climatic data had been di shonestly selected, manipulated and/or presented to arrive at pre-deter mined conclusions that were not compatible with a fair interpretation of t he original data. The members of the Panel are listed in Appendix A at th e end of this report.

2. The Panel was not concerned with the question of wh ether the conclusions of the published research were correct. Rather it was asked to come to a view on the integrity of the Unit’s research and whether as far as could be determined the conclusions represented an honest and scientifi cally justified interpretation of the data. The Panel worked by examining represen tative publications by members of the Unit and subsequently by making two visits to the University and interviewing and questioning members of the Uni t. Not all the panel were present on both occasions but two members were pres ent on both occasions to maintain continuity. About fifteen person/days were spent at the University discussing the Unit’s work.

3. The eleven representative publications that the Pan el considered in detail are listed in Appendix B. The papers cover a period of more than twenty years and were selected on the advice of the Royal Society. A ll had been published in international scientific journals and had been thro ugh a process of peer review. CRU agreed that they were a fair sample of the work of the Unit. The Panel was also free to ask for any other material that it wished and did so. Individuals on the panel asked for and reviewed oth er CRU research materials.

4. The Panel’s work began with a detailed reading of t he published work. Every paper was read by a minimum of three Panel members at least one of whom was familiar with the general area to which the pap er related. At least one of the other two was a generalist with no special clim ate science expertise but with experience of some of the general techniques a nd methods employed in

the work. Most of the members of the Panel read all the publications. The publications provided a platform from which to gain a deeper understanding of the Unit’s research and enabled the Panel to probe particular questions in more

detail.2

5. Broadly the work of the Unit falls into two parts:

• Construction and interpretation of tree ring chrono logies extending over some thousands of years with a view to gaining information about past climates:

• Studies of temperatures over the last few hundred y ears from direct

observations.

Dendroclimatology

1. Tree growth is sensitive to very many factors inclu ding climate. By piecing together growth records from different trees, livin g or dead, it is possible to determine the temporal variation of growth patterns going back many hundreds of years. The dendroclimatological work a t CRU seeks to go beyond this and to extract from the dated growth patterns the local and regional history of temperature variations. The Unit does virtually no primary data acquisition but has used data from published archives and has c ollaborated with people

who have collected data.

2. The main effort of the dendroclimalogists at CRU is in developing ways to

extract climate information from networks of tree r ing data. The data sets are large and are influenced by many factors of which t emperature is only one. This means that the effects of long term temperatur e variations are masked by

other more dominant short term influences and have to be extracted by statistical techniques. The Unit approaches this ta sk with an independent mindset and awareness of the interplay of biologica l and physical processes underlying the signals that they are trying to dete ct.

3. Although inappropriate statistical tools with the p otential for producing misleading results have been used by some other gro ups, presumably by accident rather than design, in the CRU papers that we examined we did not come across any inappropriate usage although the me thods they used may not have been the best for the purpose. It is not clear , however, that better methods would have produced significantly different results . The published work also contains many cautions about the limitations of the data and their interpretation.

4. Chronologies (transposed composites of raw tree dat a) are always work in progress. They are subject to change when additiona l trees are added; new ways of data cleaning may arise (e.g. homogeneity a djustments), new measurement methods are used (e.g. of measuring rin g density), new statistical methods for treating the data may be developed (e.g . new ways of allowing for biological growth trends).

5. This is illustrated by the way CRU check chronologi es against each other; this has led to corrections in chronologies produced by others. CRU is to be commended for continuously updating and reinterpret ing their earlier chronologies.3

6. With very noisy data sets a great deal of judgement has to be used. Decisions have to be made on whether to omit pieces of data t hat appear to be aberrant. These are all matters of experience and judgement. The potential for misleading results arising from selection bias is v ery great in this area. It is regrettable that so few professional statisticians have been involved in this work because it is fundamentally statistical. Under such circumstances there must be an obligation on researchers to document th e judgemental decisions they have made so that the work can in principle be replicated by others.

7. CRU accepts with hindsight that they should have de voted more attention in the past to archiving data and algorithms and recor ding exactly what they did. At the time the work was done, they had no idea tha t these data would assume the importance they have today and that the Unit wo uld have to answer detailed inquiries on earlier work. CRU and, we are told, the tree ring community generally, are now adopting a much more r igorous approach to the archiving of chronologies and computer code. The di fficulty in releasing program code is that to be understood by anyone els e it needs time-consuming work on documentation, and this has not been a top priority.

8. After reading publications and interviewing the sen ior staff of CRU in depth, we are satisfied that the CRU tree-ring work has be en carried out with integrity, and that allegations of deliberate misre presentation and unjustified

selection of data are not valid. In the event CRU scientists were able to give convincing answers to our detailed questions about data choice, data handling and statistical methodology. The Unit freely admits that many data analyses they made in the past are superseded and they would not do things that way today.

9. We have not exhaustively reviewed the external crit icism of the dendroclimatological work, but it seems that some o f these criticisms show a rather selective and uncharitable approach to infor mation made available by CRU. They seem also to reflect a lack of awareness of the ongoing and dynamic nature of chronologies, and of the difficul t circumstances under which university research is sometimes conducted. F unding and labour pressures and the need to publish have meant that p ressing ahead with new work has been at the expense of what was regarded a s non-essential record keeping. From our perspective it seems that the CRU sins were of omission rather than commission. Although we deplore the ton e of much of the criticism

that has been directed at CRU, we believe that this questioning of the methods and data used in dendroclimatology will ultimately have a beneficial effect and improve working practices

Temperatures from Historical Instrumental Records

1. The second main strand of work at CRU has been the collection and collation of instrumental land temperature records from all o ver the world and the construction of regional, hemispherical and global scale temperature records. These records are irregularly distributed in space and time. Modern records come largely from land-based meteorological station s but their geographical distribution is uneven and strongly biased in favou r of the northern hemisphere4 where most of the Earth’s land masses are located. Oceans cover two thirds of the Earth’s surface and away from the main shipping routes coverage is thin. For earlier centuries the record is much sparser. D eriving estimates of past temperatures on a global, hemispheric and regional scale from incomplete data sets is one of the problems faced by the Unit and i n consequence an important current interest is the discovery of useable old te mperature records from a variety of sources.

2. In the latter part of the 20 th century CRU pioneered the methods for taking into account a wide range of local influences that can m ake instrumental records from different locations hard to compare. These met hods were very labour intensive and were somewhat subjective. Much of thi s work was supported by the US Department of Energy and was published with the details of station corrections several times a year. Since the 1980s t he Unit has done no more of this work and have concentrated on the merging and interpretation of data series corrected by others. There have been various analyses of similar publicly available data sets by different internati onal groups. Although there

are some differences in fine detail that reflect th e differences in the analytical methods used, the results are very similar.

3. The Unit has devoted a great deal of effort to unde rstanding how instrumental

observations are best combined to derive the surfac e temperature on a variety of time and space scales. It has become apparent fr om a number of studies that there is elevation of the surface temperature in an d around large cities and work is continuing to understand this fully.

4. Like the work on tree rings this work is strongly d ependent on statistical analysis and our comments are essentially the same. Although there are certainly different ways of handling the data, some of which might be superior, as far as we can judge the methods which CRU has employed are fair and satisfactory. Particular attention was given t o records that seemed anomalous and to establishing whether the anomaly w as an artefact or the result of some natural process. There was also the challenge of dealing with gaps in otherwise high quality data series. In deta iled discussion with the researchers we found them to be objective and dispa ssionate in their view of

the data and their results, and there was no hint o f tailoring results to a particular agenda. Their sole aim was to establish as robust a record of temperatures in recent centuries as possible. All o f the published work was accompanied by detailed descriptions of uncertainti es and accompanied by appropriate caveats. The same was true in face to f ace discussions.

5. We believe that CRU did a public service of great v alue by carrying out much time-consuming meticulous work on temperature recor ds at a time when it was

unfashionable and attracted the interest of a rathe r small section of the scientific community. CRU has been among the leader s in international efforts to determining the overall uncertainty in the deriv ed temperature records and where work is best focussed to improve them.5

6. The Unit has demonstrated that at a global and hemi spheric scale temperature results are surprisingly insensitive to adjustments made to the data and the

number of series included.

7. Recent public discussion of climate change and summ aries and popularizations of the work of CRU and others often contain oversimplifications that omit serious discussion of unc ertainties emphasized by the original authors. For example, CRU publications rep eatedly emphasize the discrepancy between instrumental and tree-based pro xy reconstructions of temperature during the late 20 th century, but presentations of this work by the IPCC and others have sometimes neglected to highlig ht this issue. While we find this regrettable, we could find no such fault with the peer-reviewed papers

we examined

Conclusions

1. We saw no evidence of any deliberate scientific mal practice in any of the work

of the Climatic Research Unit and had it been there we believe that it is likely

that we would have detected it. Rather we found a small group of dedicated if slightly disorganised researchers who were ill-prep ared for being the focus of public attention. As with many small research group s their internal procedures

were rather informal.

2. We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not b een carried out in close collaboration with professional statisticians. Inde ed there would be mutual

benefit if there were closer collaboration and inte raction between CRU and a much wider scientific group outside the relatively small international circle of temperature specialists.

3. It was not the immediate concern of the Panel, but we observed that there were important and unresolved questions that related to the availability of environmental data sets. It was pointed out that si nce UK government adopted a policy that resulted in charging for access to da ta sets collected by government agencies, other countries have followed suit impeding the flow of processed and raw data to and between researchers. This is unfortunate and seems inconsistent with policies of open access to data promoted elsewhere in

government.

4. A host of important unresolved questions also aris es from the application of Freedom of Information legislation in an academic c ontext. We agree with the CRU view that the authority for releasing unpublish ed raw data to third parties should stay with those who collected it.

Submitted to the University 12 April 20106

Addendum to report, 19 April 2010

For the avoidance of misunderstanding in the light of various press stories, it is important to be clear that the neither the panel re port nor the press briefing intended to imply that any research group in the field of clima te change had been deliberately misleading in any of their analyses or intentionall y exaggerated their findings. Rather, the aim was to draw attention to the comple xity of statistics in this field, and the need to use the best possible methods.7

APPENDIX A

PANEL MEMBERSHIP

Chair: Prof Ron Oxburgh FRS (Lord Oxburgh of Liverp ool)

Prof Huw Davies, ETH Zürich Prof Kerry Emanuel , Massachusetts Institute of Technology

Prof Lisa Graumlich , University of Arizona. Prof David Hand FBA, Imperial College, London. Prof Herbert Huppert FRS , University of Cambridge Prof Michael Kelly FRS , University of Cambridge8

APPENDIX B

Peer-reviewed publications for assessment

.

1. Brohan, P., Kennedy, J., Harris, I., Tett, S.F.B. a nd Jones, P.D., 2006: Uncertainty estimates in regional and global observed temperatu re changes: a new dataset from

1850. J. Geophys. Res. 111 , D12106.

2. Briffa, K. R., F. H. Schweingruber, P. D. Jones, T. J. Osborn, S. G. Shiyatov, and E. A. Vaganov. 1998a. Reduced sensitivity of recent tr ee-growth to temperature at high

northern latitudes. Nature 391 :678-682.

3. Briffa, K. R., F. H. Schweingruber, P. D. Jones, T. J. Osborn, I. C. Harris, S. G. Shiyatov, E. A. Vaganov, and H. Grudd, 1998b. Trees tell of past climates: but are they speaking less clearly today? Philosophical Transactions of the Royal Society of London Series B – Biological Sciences 353, 65-73.

4. Briffa, K. R. 2000. Annual climate variability in t he Holocene: interpreting the message of ancient trees. Quaternary Science Reviews 19 , 87-105.

5. Briffa, K.R., Osborn, T.J., Schweingruber, F.H., Ha rris, I.C., Jones, P.D., Shiyatov, S.G. and Vaganov, E.A., 2001: Low-frequency temper ature variations from a northern tree-ring density network. J. Geophys. Res. 106 , 2929-2941.

6. Briffa, K. R., V. V. Shishov, T. M. Melvin, E. A. V aganov, H. Grudd, R. M.

Hantemirov, M. Eronen, and M. M. Naurzbaev. 2008. T rends in recent temperature and radial tree growth spanning 2000 years across n orthwest Eurasia. Philosophical Transactions of the Royal Society B-Biological Scie nces 363 , 2271-2284.

7. Jones, P.D. and Moberg, A., 2003: Hemispheric and l arge-scale surface air temperature variations: An extensive revision and a n update to 2001. J. Climate 16

, 206-223.

8. Jones, P.D., Raper, S.C.B., Bradley, R.S., Diaz, H. F., Kelly, P.M. and Wigley, T.M.L., 1986a: Northern Hemisphere surface air tem perature variations: 1851-1984. Journal of Climate and Applied Meteorology 25 , 161-179.

9. Jones, P.D., Raper, S.C.B. and Wigley, T.M.L., 1986 b: Southern Hemisphere surface air temperature variations: 1851-1984. Journal of Climate and Applied Meteorology

25 , 1213-1230.

10. Jones, P.D., Groisman, P.Ya., Coughlan, M., Plummer , N., Wang, W-C. and Karl,

T.R., 1990: Assessment of urbanization effects in t ime series of surface air temperature over land. Nature 347, 169-172.

11. Jones, P.D., Lister, D.H. and Li, Q., 2008: Urbaniz ation effects in large-scale temperature records, with an emphasis on China. Jou rnal of Geophysical Research,

113, D16122.

Supporting documentation

Briffa and Melvin (2009) which is online at http://www.cru.uea.ac.uk/cru/people/briffa/yamal200 9/9

TR017 – Bradley, R.S., Kelly, P.M., Jones , P.D., Goodess, C.M. and Diaz, H.F., 1985: A Climatic Data Bank for Northern Hemisphere Land A reas, 1851-1980, U.S. Dept. of Energy, Carbon Dioxide Research Division, Technical Report TRO17 , 335 pp.

TR022 – Jones, P.D., Raper, S.C.B., Sante r, B.D., Cherry, B.S.G., Goodess, C.M., Kelly, P.M., Wigley, T.M.L., Bradley, R.S. and Diaz , H.F., 1985: A Grid Point Surface Air Temperature Data Set for the Northern H emisphere, U.S. Dept. of Energy, Carbon Dioxide Research Division, Technical Report TRO22 , 251 pp.

TR027 – Jones, P.D., Raper, S.C.B., Cherr y, B.S.G., Goodess, C.M. and Wigley, T.M.L., 1986: A Grid Point Surface Air Temperature Data Set for the Southern Hemisphere, 1851-1984, U.S. Dept. of Energy, Carbon Dioxide Research Division, Technical Report TR027 , 73 pp.