The Practice of Statistics

When presented with a book titled Statistical Rules of Thumb, a statistician’s natural reaction mightbe towrite down a few of their own favorite rules of thumb. Are they included in this book? Does the discussion highlight the central idea for a nonstatistician? Are there suf cient caveats to be sure the rules of thumb are not mistaken for undeniable truths? What other rules and ideas does the author present that might help us explain our  eld to our students, our colleagues and our consulting clients? I thought of a handful of statistical rules of thumb that I emphasize in my teaching for undergraduate and master’s-level business students. I wrote them down before I began reading, and as I worked through the book, I paid particular attention to the author’s treatment of these big ideas. I am happy to report that Statistical Rules of Thumb addressed all of my rules in a very satisfactory way. This book makes great reading, and it will be useful for our colleagues or clients who use statistics in their research. It would make a great gift for a student about to begin a graduate program in a  eld that will require data analysis or modeling. It will be handy for statistics instructors who need to remind themselves to back up from time to time and look at the big picture. The applications and perspectives owe much to van Belle’s career in public health and environmental studies, but the book is still broadly applicable. There are eight chapters, and each contains about a dozen rules. Each rule is presented with a brief introduction, followed by a statement of the rule and an illustration via an example. Next, we see a basis for the rule presented as an informal argument for the rule’s plausibility, and  nally some discussion and extensions. It is this last section that usually contains the requisite warnings of when the rule applies. A companion Web site, www.vanbelle. org, gives updates and errata, as well as a “rule of the month” with additional discussion. This intriguing site promises in the future to providea chance for readers to comment on (and perhaps add to) the rules of thumb cataloged in the book. The book begins with basic ideas of inference (Chapter 1) and thoughtsabout sample size (Chapter 2). The ideas here mesh nicely with my own list. One very basic rule on my list is “for data with no extreme outliers, the standard deviation is about a quarter of the range. ” I  nd this to be a very handy device for beginners who have a hard time getting a feel for what standard deviation means, and it generalizes easily to the idea of standard error in univariate inference or regression. In Rule 1.16, “Estimate the Standard Error with Sample Range/Sample Size,” the author presents a slightly more sophisticated version of this rule for standard error. The point in either case is that practitioners can have an estimate of the amount of variation in a dataset just from knowing the nature of the data. I do have one minor complaint about Chapter 1: the justi cations of some of the rules refer to statistical concepts, often themselves rules of thumb, that are well outside the scope of a typical  rst course in statistics. These might intimidate an otherwise interested reader into not following through the book. An example of this problem is Rule 1.15, which makes a case advocating use of the bootstrap by contrasting it with the delta method. Little background on the delta method is provided, and I fear the reference will be mysterious for most of the target audience. Chapter 3 on covariation contains good advice for anyone working to develop statistical models. The book keeps its focus on correlation, simple linear regression, and agreement measures, but the lessons apply to model building in a variety of  elds. My most general rule of thumb for this material is: “check assumptions carefully and interpret results cautiously. ” Statistical Rules of Thumb breaks this advice down into several discrete chunks that are dif cult to argue with. The book does a particularly good job of emphasizing the differences between correlation (a symmetric measure of covariation) and regression (an asymmetric measure). One dif culty here is that some of the best ideas that pertain to regression are expressed more generally in Chapter 1. For example, Rule 1.4 on checking assumptions of independence, equal variance, and normality, and Rule 1.12 on properties preserved under transformation, both seem better suited for the chapter on regression. A reader lookingfor references to regression would not be led to these ideas by either the chapter titles or the index. Chapters 4 and 5 are more specialized in the direction of the author’s expertise in epidemiology and environmental studies. Even within these disciplines, many of the tenets discussed provide good general advice. Rules 4.1, “start with the Poisson to model incidence or prevalence,” and 5.1, “think lognormal,” refer speci cally to model-building conventions in their respective disciplines. The discussion of these rules is especially thoughtfuland makes the point that an underlying scienti c model forms the backboneof a successful statistical analysis. An interesting intellectual exercise (perhaps a future section of the author’s web site) might be to combine the rules in this book into a shorter set of meta-rules. Much of Chapters 4 and 5 could be summed up under the general direction, “if you choose not to use the standard tools for analysis, explain carefully the properties of your data that make the alternative approach more reasonable. ” The one meta-rule that perhaps does not get enough attention here would be, “know what to focus on, and what to ignore, in the output from your statistical software. ” Chapters 6 and 7 present an overview of the statistical research process, beginning with the design of an experiment, and moving on to reducing the data and presenting the results graphically. Chapter 8 deals with the role of a statistician as a consultant, and as such, it is the only part of the book directed at least as much toward the statistical expert as the novice. The rules here are closer to opinions, such as Rule 8.8, “any statistical package will do. ” As such, these latter chapters may generate more arguments than the technical suggestions in the earlier chapters. Those arguments will not come from me, however, as I found I strongly agreed with the author in most cases, such as Rule 7.4, “never use a pie chart. ” In cases where I am more ambivalent, I at least respect the well-written basis presented; for example, Rule 6.4, “higher order interactions rarely occur. ” Although I generally agree with the author’s contention that the chapters are mostly independent and can be read in any order, there are several places in StatisticalRules of Thumbwhere I wish the order of topicswas slightlydifferent. “Start by plotting the data” was my  rst rule of thumb, but that idea appears here quite late, in Rules 6.7 and 7.3. This is a thought-provokingbook, and through the Web site, the author has thoughtfullyprovided a mechanism for readers to provide their suggestions and reactions. I strongly encourage statisticians and researchers to have Statistical Rules of Thumb handy, and to reply to Professor van Belle’s invitation to respond with their favorite rules of thumb.