Design and Analysis of Cluster Randomization Trials in Health Research

sample statistics are treated as though they are parameters. The various sample correlations can differ substantially, but the reader is given little help in deciding which to use in a particular instance. Standard errors are not given and this de cit reduces the usefulness of the statistics for later meta analyses. Incidentally, the Task Force did not condemn correlation-based measures, but it did express a preference for nonstandardized measures of effect size (e.g., ordinary regression coef cients) whenever the units of measurement are meaningful. Counternull intervals are promoted as substitutes for con dence intervals (p. 13). The endpoints of counternull intervals are the effect sizes under the null and alternative hypotheses for which the likelihood functions are equal. Counternull intervals are interpreted as con dence intervals, but otherwise are similar to the likelihood intervals described by Royall (1997, chaps. 4 and 6). In R, the intervals are motivated by appealing to p-values rather than likelihood. Counternull intervals are useful for giving meaning to the likelihood of the null hypothesis, but they fall short as substitutes for con dence intervals. In particular, they suffer from the severe limitation that the “con dence coef cient” is not chosen by the investigator. Instead, the coef cient is data dependent and is equal to 1ƒ the two-sided p value that corresponds to the contrast. Accordingly, the con dence coef cient could be absurdly small or large (e.g., 1 ƒ 5  1010 , p. 98). If interval estimates of standardized effect size measures are desired, then a more sensible approach is to construct con dence intervals having  xed con dence coef cients (e.g., Steiger and Fouladi 1997). Throughout the book, the BESD is used to display correlation-based effect sizes computed in ANOVA or frequency table settings. The BESD is constructed by  xing the margins of a 2 2 frequency table (groups  outcomes) to make the column sums and the row sums homogeneous. The cell frequencies are obtained by equating the estimated ê coef cient to r (the correlationbased effect size measure) and solving for the cell frequencies. The result is a 2 2 table with success proportion equal to 12 41 r5 in the two columns and equal to 12 on the average. Thompson and Schumacker (1997) argued that, in the two-sample comparison of means setting, the BESD is appropriate only when the response variable can be meaningfully dichotomized to produce a 50% success rate and that, in general, the BESD can seriously distort effect size measures. These criticisms were not addressed in R . Instead, the BESD was employed in a routine, uncritical manner. On page 26, for example, the BESD is applied to data from a prospective study of the effects of aspirin on risk of heart attack. Fitting the conventional multiplicative (i.e., log linear) model for these data yields an estimated odds ratio of 0.55 and an associated 95% con dence interval of 0.43–0.69. The BESD for these data consists of a 2  2 table in which average heart attack risk is 50% (48.3% in the aspirin group and 51.7% in the placebo group) and the odds ratio is 0.87. This application of the BESD is both unnecessary and inappropriate. The odds ratio itself is a meaningful measure of the multiplicative effect of exposure to aspirin; that is, the odds of heart attack in the aspirin group were only 55% as large as the odds in the placebo group. The BESD fails to preserve multiplicative effect size and instead preserves an irrelevant characteristic of the two-way table, namely the estimated ê coef cient. This application of the BESD illustrates the danger of performing data analysis without the bene t of a statistical model. The BESD is especially obtuse when applied to nonpairwise contrasts among the levels of an ANOVA factor. In this situation, the columns of the 2  2 table do not correspond to actual factor levels, but instead represent hypothetical groups at interpolated levels of the factor. One might be able to make sense of this display if the factor is quantitative, but it is doubtful that any clarity has been achieved. There is a need for a book that helps behavioral scientists to use and to understand contrasts and associated effect size measures. The authors (especially Rosenthal and Rosnow) deserve commendation for their persistence in this endeavor. Hopefully, they will succeed next time.