Counterpoint: A tale of two meta-analyses revisited

Dear Sir, We appreciate the interest of Tzellos and colleagues in our commentary regarding meta-analyses of major adverse cardiovascular events (MACEs) occurring in trials of briakinumab and ustekinumab, and we would like to emphasize several points.1 Our aim was to demonstrate that meta-analyses of rare events using only study-level data are inherently limited, and thus the results of these studies should be interpreted with caution. One of the major issues with meta-analyses of rare events is that they are susceptible to minor changes in the statistical techniques.2,3 There has been substantial debate about which statistical methodology is best in performing meta-analyses of rare events, detailed in our initial commentary, which is a good indicator that they all have shortcomings that need to be considered when interpreting their results. Furthermore, a major limitation of meta-analyses using only event data, such as the studies by Tzellos et al. and Ryan et al., is an inability to adjust for unequal follow-up time in the treatment and placebo groups during the randomized-controlled phase. We have shown in a previously published meta-analysis of rare events that adjusting for dropouts can, in fact, influence the results. While examining the risk of infection and malignancy with the use of anti-TNF agents in patients with psoriatic disease, we demonstrated a statistically significant increased risk of overall infection when using only event data (Odds Ratio (OR) = 1.18, 95% Confidence Interval (CI): 1.05-1.33), but not when conducting a separate meta-analysis of rates with adjustment for follow-up time (Incidence Rate Ratio (IRR) = 1.01, 95% CI: 0.92- 1.11).4 The study by Gordon et al., one of the nine individual randomized-controlled trials (RCTs) included in both meta-analyses by Tzellos et al. and Ryan et al., which found 5 MACEs (out of the total 10 MACES reported across the 9 pooled trials), had a 95.7% dropout rate in the placebo group versus 24.1% in the treatment group over the 12-week placebo-controlled portion of the trial (most of which were due to lack of efficacy).5 Although this is the most extreme example, all of the remaining 8 trials included in the meta-analyses had more dropouts, and thus less follow-up time, in the placebo compared to treatment groups, with a mean dropout rate of 11.7% among the placebo groups compared to 4.3% in the treatment groups.6-13 It is a rational conclusion, and not simply speculation, that this may have biased the results in a meta-analysis that did not adjust for the differing follow-up time in the treatment versus placebo groups. Some of these shortcomings, such as adjusting for dropouts, can be overcome by performing a time-to-event analysis. Despite the limitations of the approaches used by Tzellos and Ryan, the results of these meta-analyses suggest a safety signal, which is simply a hypothesis that warrants further investigation. The question of whether anti-IL-12/23 inhibitors increase risk of MACEs cannot be adequately addressed using only published data from randomized-controlled trials. Access to individual patient-level data to perform time-to-event meta-analyses is key to yielding more accurate results. Moreover, the cardiovascular safety signal observed in these meta-analyses may be false owing to sources of statistical error (see original commentary for details). The cardiovascular safety signal observed in pre-approval studies has not been observed in post-approval studies. For example, an increased risk of MACEs has not been found in long-term safety studies of ustekinumab with 5 years of follow-up, with the overall rate of MACEs in ustekinumab-treated patients (0.44⁄100 patient-years (PY)) being comparable to those reported with the use of anti-TNF agents in psoriasis (range 0.36–0.84⁄100 PY).14-16 Moreover, neither the Federal Drug Administration (FDA) nor European Medicines Agency (EMA) has issued warnings about cardiovascular risk associated with ustekinumab. More rigorous research evaluating the impact of psoriasis treatments on cardiovascular risk is urgently needed.