Reply from Gordon B. Drummond and Brian D. M. Tom

We thank Dr Hopkins and his co-authors for their trenchant criticism of the articles that have been published in this series. They take us to task for a number of errors and omissions, many of which we would refute, although we do agree that in one case (our statement about 95% confidence intervals) we were wrong. We wrote: The 95% confidence values for the mean indicate that we'd be 95% likely to get another estimate of the mean within this range, using another set of data samples from the same population. This was incorrect. When samples are taken from a population, each sample will yield a different estimate of the confidence interval of the mean. If we were to take sample after sample from a population, and use each sample to calculate a 95% confidence interval for the mean value, in the long run 95% of these confidence intervals would include the actual mean of the entire population. In many other respects, we appear to agree. We suggest that their final judgment of our objectives should be more positive as we develop the themes further. We are far from the first to be aware that data presentation and analysis need to be improved. Dr Hopkins has himself written of the difficulty in using ‘innovative and controversial practices’ and cited previous attempts to promote better practice, including several of his own, although we read a note from the editor of one of his articles that he himself has been subject to strong disagreement (Hopkins et al. 2009). Why have previous attempts to improve data presentation and statistical rigour failed? There are probably several reasons, perhaps the most important of which is the difficulty of writing in a way that is both easily understood and also exact. We note that the authors assert, in their covering letter: ‘The authors’ use of non-technical language is also no excuse for muddying the waters over important statistical concepts. It is possible to communicate these concepts clearly and accurately to non-statisticians’. Here, we disagree profoundly. The authors (and reviewers) of the present articles have all commented on how hard it has been to express these statistical concepts clearly and simply. The articles are intended primarily to be easy to read, and aim to successively tackle single topics. Each topic is developed in an intuitive way, using a simple, sometimes fanciful, but we hope memorable example so that a non-expert can understand the logic (or in the case of the null hypothesis, appreciate the awkward logic) of the conventional statistical approaches that are used by the vast majority of biological scientists when they attempt to analyse their data. Like it or not, P values still predominate in most papers. Hopkins notes that it may take ‘decades for their suggestions to become mainstream’ (Hopkins et al. 2009), and we agree completely that effecting improvement will require protracted effort. To support our contention, that getting these awkward concepts expressed simply is far from easy, we would cite the preface to a useful handbook by Lang & Secic (1997). After quoting Mark Twain, who wrote ‘My books are water; those of the great geniuses are wine – everybody drinks water’ the authors observe: ‘Both fine wines and biostatistics are characterized by complexities and subtleties that are truly appreciated only by the relatively few people who devote the time to master them. To these readers, we extend our apologies; this book was not written for you. Rather, it was written for a much larger group of readers: those who thirst for a basic understanding of statistics, but who do not aspire to appreciate the nuances. This is a book about reporting and interpreting statistical presentations, not about understanding theories of probabilities or mathematical concepts. This is a book for water drinkers. It is exceedingly difficult to explain many statistical concepts in terms that are both technically accurate and easily understood by those with only a cursory knowledge of the topic. Thus, if our explanations do not include some of the finer points of a topic or if they have bypassed some distinction of meaning, it is because we believe that such fine points and distinctions would detract from an explanation otherwise adequate for most readers. ’ This has been our intention as well, and we are sorry if we have upset some wine drinkers. We suspect that going through the points raised in his critical letter, one by one, would not only be complex for the reader but would also descend into ‘yes it is, no, it isn't’. For example, it matters little that the term dynamite plot has been less used than the term bar graph, when what we are considering is in fact the former. Most of the critique is that we have omitted to consider some aspects of the topics we raise. We have studiously avoided excess detail, because we consider simplicity is the key. For example, it may well be important how to deal with outliers, but as they are not the central issue being considered, we miss out their consideration. These articles have been written to be short and easily assimilated, a coffee-break read. We have had a lot of positive feedback that they are useful for our colleagues, who have been using them for teaching. Although Hopkins’ guideline article (Hopkins et al. 2009) is comprehensive, it takes about 50 minutes to read, and has a complex inter-related structure with two very large tables, no figures, and few if any examples: not coffee-break reading, or teaching material. Our final defence is ‘horses for courses’. We are aiming at a different readership. We would not wish to mislead, and intend to correct any overt errors, but we are taking a gradual approach. We would be interested to hear from other readers.