Statistical Perspectives: All Together NOT

To the Editor, An editorial under the title Statistics: all together now, one step at a time appeared in this and other physiological journals earlier this year [4] to announce a forthcoming series of 14 perspective articles on statistics. Two of the articles have now been published in six journals (6; 7) and the third has started to appear [5]. The purpose of this letter is to identify what we consider to be substantial errors and omissions. In the first article, Show the data, don’t conceal them [7], we agree with only the following two assertions. (For clarity, quoted assertions throughout this letter are shown in italics and are sometimes paraphrased for conciseness.) • Inspect dot plots of the original observations. Yes, although such plots need not appear in the publication unless there is something unusual about the data. • ‘Dynamite-plunger plots’ are inappropriate for repeated measurements. Yes. The term bar graph, which exceeds plunger plot by a factor of 23,000 in a Google search, could have been introduced here. The term line graph is also worth noting for a plot in which means of repeated measurements are connected with line segments. Next, several assertions we consider to be misleading. • Showing means and error bars would conceal the fact that a variable had a log–normal distribution. This assertion may be true for the standard error of the mean but not the standard deviation. A standard deviation that in magnitude approaches or exceeds the mean is a good indication that the variable needs log transformation, as is a positive relationship between the SD and the mean when there are several groups [2]. For this and other reasons, the error or error bar should always be a standard deviation. • Error bars illustrating the 95% confidence interval (CI) should be included. Use of confidence intervals for effect statistics should be obligatory, but it is not appropriate to use confidence intervals in dot or scatter plots or with means of original variables. The level of confidence need not be 95%. Sterne and Davey Smith [13] suggested a default of 90%. • To define reference ranges a very large sample of normal values is needed. The use of the word ‘normal’ here is confusing. A random sample from the population would be better. Also, depending on the distribution, a large or even modest sample size will suffice. Now the assertions that are wrong. • Dynamite-plunger plots are never an appropriate way to plot the data. We agree that bar graphs are inappropriate to display means of repeated measurements, but they are fine to convey magnitudes of differences in means in different groups, provided the error bars are standard deviations. • When data are skewed, the mean is not always the best choice of summary. An alternative and perhaps preferable index of scatter would be the 95% confidence limits. Then the 95% CI is chosen as the reference range. Confidence limits or intervals are neither a measure of scatter nor a reference range. The authors presumably intended to refer to the use of quantiles. The usual quantiles for skewed data are the quartiles (lower quartile, median and upper quartile), often represented as box and whisker plots. The authors instead have suggested the 2.5th and 97.5th percentiles, which would be appropriate only to define the reference range for a clinical measure. There is then a confusing discussion about observations that fall outside the 95% reference range being ‘abnormal’. It is unclear to us what readers are supposed to take from this section. Presumably the abnormality relates to flagging an individual’s value in a clinical setting; flagging of outliers in a research setting would require a reference range much wider than 95%. • The 95% confidence values for the mean indicate that we’d be 95% likely to get another estimate of the mean within this range, using another set of data samples from the same population. Here we have the classic misconception about the meaning of the confidence interval. In fact there is only an 85% chance that another sample mean will fall within the 95% confidence interval defined by the first sample (the exact value depends a little on the degrees of freedom). The 95% confidence interval is a confidence interval for the true (large-sample) mean, not the sample mean. Finally the omissions. These issues may be scheduled for future articles, but in our view they should have been at least mentioned in the first article. • There is only one mention of logarithms, when data skewed to the right are defined parenthetically and inaccurately as lognormal. Why is there no mention of log transformation or use of logarithmic axes, and of the related issues of uniformity of effects and errors? The DOI:10.1111/j.1549-8719.2011. 00144.x