R practice: Meta-analysis (4)Minato Nakazawa
7 March 2011
1 Meta-analysis to integrate odds ratios by Peto’s method
Last week we have seen the example of Peto’s method as the study which Yusuf et al. (1985) applied the Peto’s design tosecondary preventive effect of beta blocker against heartattack.
Table 1.3 shows the result of meta-analysis of 15 RCTs about the preventive effect (reduction of mortality) of long-term
administration of beta-blocker by Peto’s method. In R, it is supported by metafor package.
¶³
Table 1.3
studyidreynoldswilhelmsson
ahlmarkmulticentrebaber
rehnqvist
norwegiantaylorhansteenbhatjulianaustralianswedish
mangercats
eis
rehnqvist2

bbdeath
3
7
5
102
28
4
98
60
25
138
64
45
957
25

bbno
38
114
69
1533
355
59
945
632
278
1916
873
263
291
858
154

cntldeath
3
14
11
127
27
6
152
48
37
188
52
47
16
45
31

cntlno
39
116
93
1520
365
52
939
471
282
1921
583
266
293
883
147

µ

´

Using the data entered this as tab-delimited text file yusuf1985.txt, fitting the random effects model and integrated by
Peto’s method and checking heterogeneity can be done by the following script (it was done last week).
¶³
x <- read.delim("yusuf1985.txt")
library(metafor)
res <- rma(measure="PETO",ai=bbdeath,n1i=bbno,ci=cntldeath,n2i=cntlno,data=x)
forest(res,atransf=exp,slab=paste(x$studyid),xlim=log(c(0.01,50)),at=log(c(1:10/10,2:10)))summary(res)exp(res$b)µ´
We can get forest plot for random effects model (integrated odds ratio is shown as 0.78 at the bottom of the plot).
summary(res) gives the result of heterogeneity test and the model fitting test. Heterogeneity test is to test the null-hypothesisthat the result of studies are homogeneous. Here we get p-value of 0.2927 so that we cannot reject the null-hypothesis.
exp(res$b) shows the estimated odds ratio by Peto’s method. And it should be noted that the random effects model fitting
is significant (p-value is 0.0006).

1¶³> summary(res)Random-Effects Model (k = 15; tau^2 estimator: REML)
logLik-4.3403

Deviance
8.6806AIC
12.6806

BIC
13.9587tau^2 (estimate of total amount of heterogeneity): 0.0173 (SE = 0.0238)
tau (sqrt of the estimate of total heterogeneity): 0.1315
I^2 (% of total variability due to heterogeneity): 26.22%
H^2 (total variability / within-study variance):
1.36
Test for Heterogeneity:
Q(df = 14) = 16.3466, p-val = 0.2927
Model Results:
estimate-0.2422

se0.0704--Signif. codes:

zval-3.4424pval
0.0006ci.lb-0.3801

ci.ub-0.1043***

0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1> exp(res$b)[,1]
intrcpt 0.7849175µ´

2 Sensitivity Analysis
Here I cite the explanation from Egger et al. (1997). This paper basically uses the data by Yusuf et al. (1985) but added 2
new studies (details are not available) so that the analysis may slightly differ from our practice.
Opinions will often diverge on the correct method for performing a particular meta-analysis. The robustness of the findings todifferent assumptions should therefore always be examined in a thorough sensitivity analysis. This is illustrated in [figure 4] (seebelow) for the meta-analysis of β blockade after myocardinal infarction. Firstly, the overall effect was calculated by different
statistical methods, by using both a fixed and a random effects model. The [figure 4] shows that the overall estimates are virtually
identical and that confidence intervals are only slightly wider with the random effects model. This is explained by the relatively
small amount of variation between trials in this meta-analysis.

In our practice using R, the following script generates the result of fitting fixed effects model and its forest plot.

2

¶

³

res.fixed <- rma(measure="PETO",ai=bbdeath,n1i=bbno,ci=cntldeath,n2i=cntlno,data=x,method="FE")forest(res.fixed,atransf=exp,slab=paste(x$studyid),xlim=log(c(0.01,50)),at=log(c(1:10/10,2:10)))summary(res.fixed)
exp(res.fixed$b)µThe forest plot with the case of previous one (random effects model) are shown below.

´Besides the effect of model, the effects of trial quality, trial size, and the exclusion of trials stopped early should be examined
as sensitivity analysis. It can be done by using subset() function to limit trials to meet with specified conditions andcomparing them. I will cite Eggar et al.’s explanation again.
Secondly, methodological quality was assessed in terms of how patients were allocated to active treatment or control groups, howoutcome was assessed, and how the data were analysed. The maximum credit of nine points was given if patient allocation wastruly random, if assessment of vital status was independent of treatment group, and if data from all patients initially included wereanalysed according to the intention to treat principle (Note: we cannot practice this because of lack of such detailed information).
[Figure 4] shows that the three low quality studies (≤ 7 points) showed more benefit than the high quality trials. Exclusion ofthese three studies, however, leaves the overall effect and the confidence intervals practically unchanged.
Thirdly, significant results are more likely to get published than non-significant findings, and this can distort the findings ofmeta-analysis. The presense of such publication bias can be identified by stratifying the analysis by study size — smaller effects
can be significant in larger studies. If publication bias is present, it is expected that, of published studies, the largest ones willreport the smallest effects. [Figure 4] shows that this is indeed the case, with the smallest trials (50 or fewer deaths) showing thelargest effect. However, exclusion of the smallest studies has little effect on the overall estimate.
Finally, two studies were stopped earlier than anticipated on the grounds of the results from interim analyses. Estimates oftreatment effects from trials that were stopped early are liable to be biased away from the null value. Bias may thus be introduced
in a meta-analysis that include such trials. Exclusion of these trials, however, affects the overall estimate only marginally.
The sensitivity analysis thus shows that the results from this meta-analysis are robust to the choice of the statistical method andto the exclusion of trials of poorer quality or of studies stopped early. It also suggests that publication bias is unlikely to havedistorted its findings.

In our practice, the effect of sample size (its meaning implys publication bias) can be tested by the following script.

3¶³

x$death <- x$bbdeath+x$cntldeath
A <- subset(x,death<=50)
B <- subset(x,((death>50)&(death<=100)))
C <- subset(x,death>100)
resA <- rma(measure="PETO",ai=bbdeath,n1i=bbno,ci=cntldeath,n2i=cntlno,data=A)
resB <- rma(measure="PETO",ai=bbdeath,n1i=bbno,ci=cntldeath,n2i=cntlno,data=B)
resC <- rma(measure="PETO",ai=bbdeath,n1i=bbno,ci=cntldeath,n2i=cntlno,data=C)
pdf("sizeeffect.pdf",width=4,height=8)
layout(1:3)
forest(resA,atransf=exp,slab=paste(A$studyid),xlim=log(c(0.01,50)),at=log(c(1:10/10,2:10)))
forest(resB,atransf=exp,slab=paste(B$studyid),xlim=log(c(0.01,50)),at=log(c(1:10/10,2:10)))
forest(resC,atransf=exp,slab=paste(C$studyid),xlim=log(c(0.01,50)),at=log(c(1:10/10,2:10)))dev.off()µ´

3 Reference• Egger M, Smith GD, Phillips AN (1997) Meta-analysis: Principles and procedures. BMJ , 315: 1533-.
[http://www.bmj.com/content/315/7121/1533]

4