# Bayesian Equivalence Testing and Meta-Analysis in Two-Arm Trials with Binary Data

Authors: Cynthia Kpekpena; Saman Muthukumarana
Journal: Computational and Mathematical Methods in Medicine
 (2018)

---

## Abstract

We consider a Bayesian approach for assessing hypotheses of equivalence in two-arm trials with binary Data. We discuss the development of likelihood, the prior, and the posterior distributions of parameters of interest. We then examine the suitability of a normal approximation to the posterior distribution obtained via a Taylor series expansion. The Bayesian inference is carried out using Markov Chain Monte Carlo (MCMC) methods. We illustrate the methods using actual data arising from two-arm clinical trials on preventing mortality after myocardial infarction.


---

## Body

## 1. Introduction

Consider a clinical trial where a pharmaceutical company wants to test a new drug against a currently existing drug. Sometimes in these studies, the clinical trial end point may be the success or failure of the treatment. A binary outcome is an outcome whose unit can take on only two possible states “0” and “1.” This success/failure response variable could be heart disease (Yes/No), patient condition (Good/Critical), how often patient feel depressed (Never/Often), and so on. The natural distribution for modeling these types of binary data is the binomial distribution given by(1)fx;p=nxpx1−pn−xforx=0,1,…,n,p∈0,1.The mean and variance for the binomial random variable areEX=np and varX=np1−p, respectively. In (1), it is assumed that there are only two outcomes (denoted “success” or “failure”) and a fixed number of trials (n). The trials are independent with a constant probability of success.The main objective of this type of clinical trial is to determine whether there is a significant difference between active treatment (new drug) and reference treatment (current drug). Tests of significance have generally been argued not to be enough. That is, if thep value for a test of significance leads to the nonrejection of the null hypothesis, it is not a proof that the null hypothesis holds. The clinician may want to test a null hypothesis of equivalence against an alternative hypothesis that states that there is a sufficient difference between the two drugs.Equivalence testing is widely used when a choice is to be made between a drug (or a treatment) and an alternative. The term equivalence in the statistical sense is used to mean a weak pattern displayed by the data under study regarding the underlying population distribution. Equivalence tests are designed to show the nonexistence of a relevant difference between two treatments. It is known that Fisher’s one-sided exact test is the same as the test for equivalence in the frequentist approach [1]. This testing procedure is similar to the classical two-sided test procedure but involves an equivalence zone determined by a margin known as equivalence margin (δ).The equivalence margin (δ), which represents a margin of clinical indifference, is usually estimated from previous studies and as such is also based primarily on clinical criteria as well as statistical principle. This margin is influenced by statistical principle but largely dependent on the interest of the experimenter and research questions clinicians wish to answer. As such, the statistical method employed together with the design of the study must be in such a manner that the margin of difference is not too restrictive to capture the bounds of the research question. For a test of equivalence of two binomial proportions, the equivalence margin is discussed in [2].The frequentist approach to equivalence testing is the two one-sided test (TOST) procedure. By the TOST, equivalence is established at theα significance level if a 1−2α×100% confidence interval for the difference in treatment means μi−μj is contained within the interval −δ,δ where δ is the equivalence margin.The motivation for this paper is based on the fact that for a given disease, there is likely to be many other substitute drugs or new drugs that can be used to treat the patients. But these drugs may not all be at the same cost; some may possibly have adverse side effects, and the method of application could be complex for others. On grounds of these information, we do equivalence testing to see if two different drugs can be regarded as equivalent in terms of their treatment effect. There are a variety of different approaches to this problem as indicated by some recent literature. See Wellek [1], Albert [2], Gamalo et al. [3], Rahardja and Zhao [4] and Zaslavsky [5] for comprehensive details on recent developments. We remark that Gamalo et al. [3] consider a Bayesian approach to proportions along with noninferiority trials. In this paper, we consider a Bayesian approach focusing on equivalence tests. We also construct a simple normal approximation and provide a mechanism for missing data analysis as well.The remaining sections of this article are organized as follows: In Section2, Bayesian inferential procedure for binary data is discussed. Section 3 presents a normal approximation to the posterior distribution obtained via a Taylor series expansion. We then examine the suitability of this normal approximation. We discuss a Gibbs sampling mechanism for estimating missing data in Section 4. In Section 5, we analyze a published dataset by Carlin [6] and Yusuf et al. [7]. This dataset consists with 22 treatment-control trials to prevent mortality after myocardial infarction. We conclude with a discussion of the approach in Section 5.


## 2. Bayesian Inferential Procedure

LetXt be the number of individuals with positive exposure out of a total of nt patients in treatment group with proportion Pt. Accordingly, let Xc denote the number of individuals with positive exposure out of a total nc in the control group with proportion Pc. Then,(2)Xt∼Binnt,Pt,Xc∼Binnc,Pc.The priors on the parameters,Pt and Pc are given by(3)Pt∼Betaα,β,Pc∼Betaε,η.Then the posterior distributions ofPt and Pc are given by(4)πPt∣Xt∝Betaxt+α,nt+β−xt,πPc∣Xc∝Betaxc+ε,nc+η−xc.For Bayesian inference about treatment effect, a test is required to determine whether the posterior probability of treatment proportionsPt and Pc lies within the bounds of the equivalence margin or not. There is therefore, the need to sample from the posterior distribution of Pt−Pc. The marginal posteriors of Pt and Pc are Beta distributions, and therefore πPt−Pc∣Xt,Xc is not in an analytically tractable form. So, P1t,P2t,…,Pnt are generated from πPt∣Xt and independently P1c,P2c,…,Pnc generated from πPc∣Xc because Pt and Pc are independent. Then, P1t−P1c,P2t−P2c,…,Pnt−Pnc can be treated as a random sample from πPt−Pc∣Xt,Xc.


## 3. Normal Approximation to the Beta Posterior Distribution

Note that the posterior distributions ofPt and Pc are Beta distributions. By following Kpekpena [8], a normal approximation to posteriors can be obtained using a Taylor series expansion of the Beta distribution. By applying a Taylor series expansion with first three terms, it can be shown that πPt∣Xt≈Nμ,σ, where(5)μ=1−α−xt2−α−nt−β,σ=1−2−α−nt−β3/1−xt−α1−nt−β+xt1/2.Similarly, the approximation ofπPc∣Xc can also be obtained. The details of this construction are given in the Appendix. We provide some approximations based on this development in Table 1 and Figures 1 and 2. It is clear that the approximation starts to work well for the values of the posterior parameters from x+α=10 and n+β−x=10. However, the approximation is not suitable when Beta posterior parameters are less than 10.Table 1
Normal approximation to the Beta distribution.
Exact distribution
Approximation
Beta
2
,
1
N
1
,
∞
Beta
1
,
2
N
0
,
∞
Beta
10
,
10
N
0.5000
,
8.4853
Beta
5
,
1
N
1
,
∞
Beta
1
,
5
N
0
,
∞
Beta
2
,
2
N
0.5000
,
2.8284
Beta
3
,
3
N
0.5000
,
4.0
Beta
2
,
4
N
0.2500
,
4.6188
Beta
4
,
4
N
0.5000
,
4.8990
Beta
5
,
5
N
1
,
5.6569
Beta
30
,
20
N
0.6042
,
14.1673
Beta
20
,
30
N
0.3958
,
14.1673
Beta
50
,
20
N
0.7206
,
18.3776
Beta
20
,
50
N
0.2794
,
18.3776Figure 1
The normal approximations of (a)Beta2,2, (b) Beta3,3, (c) Beta2,4, and (d) Beta4,4.
(a)
(b)
(c)
(d)Figure 2
The Normal approximations (a)Beta5,5, (b) Beta10,10, (c) Beta30,20, and (d) Beta20,30.
(a)
(b)
(c)
(d)


## 4. Estimating Missing Data in Arms

Missing data are easily handled in Bayesian inference by treating them as another set of parameters. We estimate the missing values conditioning on the observed data. For example, letX1,…,Xn be a binary random sample from BerP in an arm and suppose that Xm is missing. Note that P represents Pt in treatment arm and Pc in control group. Let P∼Betaα,β and Y=∑i≠mnXi. Then, the likelihood of the observed data is(6)LXobs∣P=n−1yPy1−Pn−1−y.The posterior ofP based on the complete data X=Y,Xm is(7)πP∣X∝Py+xm1−Pn−y−xm1Bα,βPα−11−Pβ−1.The full conditionals ofP and Xm are(8)πP∣y,xm∼Betay+xm+α,n−y−xm+β,πxm∣y,P∼BerP.It is easy to generate from these full conditionals inR, so P and xm can be estimated using Gibbs sampling.


## 5. Data Analysis

We apply our approach on data analyzed in [7, 9]. The data includes 22 trials of beta-blockers to prevent mortality after myocardial infarction. For each of the 22 trials, a test of equivalence is done to ascertain whether the treatment proportion is equivalent to the control proportion. This example is based on the Statistical inferential procedure for binary data discussed in Sections 2 and 3. For each arm, the number of patients who had myocardial infarction out of a total nt is considered to be the number of successes in nt binomial trials. Similarly, the number of cases in the control group is treated as a binomial outcome independent of the treatment group. The equivalence margin δ is chosen to be as small as possible such that if the absolute value of the difference in the control and treatment proportions is less than δ, and we can say that the two proportions are equivalent. For demonstration purpose, we assume a practically meaningful equivalence margin δ=0.01. We use noninformative priors Beta2,1 for the parameters, Pt and Pc. The hypothesis for a test of equivalence of study number i and its control group is as follows:(9)H0:Pti−Pci≥δ,H1:Pti−Pci<δ.We perform the equivalence test in (9) using the Bayes factor [9]. Table 2 gives the results of the equivalence tests. The first column Di is the ith study label. Columns 2 and 3 are the treatment proportion (xt/nt) and control proportion (xc/nc), respectively. Columns 4 (PH1∣X) and 6 (PAH1∣X) are the posterior probabilities that H1:Pti−Pci<δ is true under the Beta posterior distributions and under the normal approximation to the Beta posterior, respectively. Column 5 (B) is the Bayes factor for exact posterior, and BA in column 7 is the Bayes Factor based on the normal approximation. For study 1, the Bayes factor for the exact posterior is 7.3822 whereas that of the normal approximation is 7.5466. Both Bayes factors are above 1 which implies that H0 is more likely to be true, and H0 is the hypothesis that the treatment proportion is not equivalent to the control proportion. We remark that classical hypothesis tests give one hypothesis a preferred status and only consider evidence against it which is not the case in Bayesian tests. Results also indicate that the approximation and exact computation lead to the same conclusion in each study indicating the suitability of the approximation.Table 2
Posterior probabilities and Bayes factor.
D
i
P
t
i
P
c
i
P
H
1
∣
X
B
P
A
H
1
∣
X
B
A
D
1
0.0789
0.0769
0.1193
7.3822
0.1170
7.5466
D
2
0.0614
0.1207
0.0725
12.7931
0.0669
13.9506
D
3
0.0725
0.1183
0.1184
7.4459
0.1132
7.8283
D
4
0.0665
0.0836
0.2343
3.2680
0.2294
3.3583
D
5
0.0789
0.0740
0.3718
1.6896
0.3710
1.6957
D
6
0.0678
0.1154
0.0998
9.02004
0.0962
9.3964
D
7
0.1037
0.1618
0.0015
665.6667
0.0010
971.6452
D
8
0.0949
0.1019
0.3861
1.5900
0.3854
1.5949
D
9
0.0899
0.1312
0.0787
11.7064
0.0784
11.7538
D
10
0.0720
0.0979
0.0432
22.1482
0.0395
24.3175
D
11
0.0733
0.0892
0.2950
2.3898
0.2958
2.3802
D
12
0.1711
0.1767
0.2317
3.3159
0.2341
3.2722
D
13
0.0309
0.0546
0.1951
4.1256
0.1900
4.2626
D
14
0.0664
0.0510
0.3030
2.3003
0.3022
2.3089
D
15
0.1623
0.2109
0.0989
9.1112
0.0994
9.0554
D
16
0.1594
0.1784
0.1871
4.3447
0.1886
4.3010
D
17
0.1116
0.0984
0.2193
3.5560
0.2214
3.5147
D
18
0.0530
0.0390
0.2673
2.7411
0.2649
2.7754
D
19
0.0345
0.0224
0.3175
2.1496
0.3293
2.0372
D
20
0.1531
0.1835
0.1584
5.3131
0.1552
5.4439
D
21
0.0691
0.1181
0.0331
29.2110
0.0311
31.1380
D
22
0.0324
0.0579
0.0932
9.7296
0.0867
10.536We now consider a missing data analysis in an arm. As an example, suppose an observation was missing in the treatment group under study 1. We estimate this missing value using Gibbs sampling derived in Section4. The posterior distributions of parameters P and xm are given in Figure 3 based on 20000 MCMC simulations. According to Figure 3, it is likely that xm is 0. The trace plot in Figure 4 shows that mixing is good enough, and there are no large spikes in the autocorrelation plot after lag 0. This is an indication of convergence of the Markov Chain.Figure 3
The distribution ofxm shows it is more likely to be 0.
(a)
(b)Figure 4
There is no discernible pattern in the trace plot and no large spikes in the autocorrelation plot.
(a)
(b)We also consider a meta-analysis of the binary data in two-arm trials in order to assess the between-study variations. Letyi be the estimate of the true effect size μi corresponding to the ith study. Then, the random effects model is given as(10)yi=μi+eiwhereei∼N0,σi2.As developed in Muthukumarana and Tiwari [10], we consider a hierarchical Dirichlet process formulation for μi as follows:(11)μi∣F∼F,F∼DPM,H,H∼Nμ,τ2,μ∼Nμ0,dτ2,1τ2∼Ga,b,where M, μ0, and d are known.We analyze the dataset published in Nissen and Wolski [11] to assess the between-study heterogeneity. In this dataset, there are 42 trials including 15565 diabetes patients who were put on rosiglitazone (treatment group) and 12282 diabetes patients assigned to medication that does not contain rosiglitazone (control group). Note that the interest is on myocardial infarction and death from rosiglitazone as a treatment for diabetes. We use the odds ratio as the treatment effect. The parameters of the model are estimated by Gibbs sampling algorithm implemented in R. The estimates of the model parameters (μi) are given in Table 3.Table 3
The estimates of posterior treatments and standard deviations.
Parameter
Estimate
Standard deviation
τ
2
0.74
0.2794073
μ
0.71
0.4142608
μ
1
−0.73
0.8151172
μ
2
−0.63
0.7358434
μ
3
1.2
0.4914719
μ
4
−1.1
0.6541732
μ
5
−0.74
0.8379231
μ
6
−0.77
0.7702218
μ
7
−0.70
0.8191522
μ
8
−0.61
0.7714286
μ
9
−0.74
0.7989194
μ
10
−0.75
0.8194161
μ
11
−0.77
0.7766973
μ
12
−0.74
0.8008516
μ
13
−0.67
0.7848413
μ
14
−0.73
0.8148389
μ
15
−0.65
0.7427869
μ
16
−0.72
0.6874473
μ
17
−0.67
0.6670385
μ
18
−0.74
0.8121317
μ
19
−0.69
0.7916802
μ
20
−0.74
0.8151257
μ
21
−0.78
0.7748052
μ
22
−0.78
0.7449046
μ
23
−0.71
0.8175963
μ
24
−0.75
0.8185433
μ
25
−1.6
0.4702446
μ
26
−0.57
0.5950003
μ
27
−0.74
0.8104195
μ
28
−0.74
0.8192542
μ
29
−0.74
0.8091605
μ
30
−0.73
0.8372302
μ
31
−0.58
0.6008941
μ
32
−0.72
0.8096048
μ
33
−0.76
0.8134446
μ
34
−0.69
0.6464147
μ
35
−0.72
0.8196591
μ
36
−0.75
0.8109735
μ
37
−0.71
0.8172117
μ
38
−0.72
0.8127885
μ
39
−0.74
0.8024024
μ
40
−0.73
0.8114458
μ
41
−0.189
0.5258747
μ
42
0.01
0.3224310We also conducted a simulation study to assess the validity of the approach. Each study was simulated by means of a binomial random variable in which the number of cases in the treatment group and the control group are generated as independent binomial random variables. We generate twenty binomial successes using the rbinom random generator. We assumen=200 in each case and fix the p at 0.7. This setting is similar to administering a treatment in twenty hospitals with 200 patients in each hospital. Fixing p at 0.7 generates number of cases that do not vary so much from each other. This is confirmed in the non-significance of the chi-square test for heterogeneity. Another set of twenty “number of cases” is generated from the binomial distribution, but this time we induce heterogeneity. This is done by varying the success probability of each trial. For instance rbinom(1, 200, 0.86), rbinom(1, 200, 0.10), rbinom(1, 200, 0.55), ….Note that our interest is in comparing the posterior treatment means of the heterogeneous studies with the studies that are not heterogeneous. Table4 compares the posterior treatment means of 20 studies with heterogeneity to the treatment means of 20 other studies in which there is no heterogeneity. Column 1 is the posterior treatment means of the nonheterogeneous (μi) studies whereas μi∗ in column 2 is the posterior treatments of the heterogeneous studies. Treatment means in column 1 (μi) are mostly 0.68 or just slightly below or above it. If the responses are similar, the treatment effects are supposed to be an estimate of a common treatment mean. On the other hand, all the treatment means in column 2 (μi∗) differ from each other significantly indicating that the model can assess the between-study variations.Table 4
Estimates of treatment means for twenty studies with 200 observations within each study.
Study
μ
i
μ
i
∗
1
0.68
0.092
2
0.67
0.39
3
0.67
0.80
4
0.68
0.69
5
0.68
0.76
6
0.68
−2.8
7
0.68
−1.4
8
0.68
0.35
9
0.68
−0.11
10
0.70
0.53
11
0.67
0.69
12
0.70
−0.054
13
0.68
0.72
14
0.67
0.39
15
0.68
0.41
16
0.69
0.79
17
0.67
0.81
18
0.69
−0.94
19
0.68
−1.6
20
0.69
0.81


## 6. Discussion

We have considered a Bayesian analysis of binary data in testing hypotheses of equivalence. The tests of hypotheses of equivalence are popular in clinical trials, and our approach is relatively simple and easy to perform. Bayesian formulation was considered for testing hypothesis of equivalence, and we observed that the normal approximation to the Beta posterior can be used for moderately large sample sizes. We also presented a mechanism for estimating missing data in arms. This is useful in situations where data are partially missing in some arms. We also considered a meta-analytic approach for assessing between-study variations.There are two directions we would like to pursue along the methods discussed in this article. We are interested in enhancing the method to accommodate extra covariates into the model in the presence of multiple outcomes in an arm. The incorporation of covariates makes the Bayes factor inappropriate, and we would like to examine the other model selection criterion in place of the Bayes factor.



---

*Source: 8134132-2018-08-08.xml*