Multiple testing correction in a meta-analysis of all adverse events recorded in large long-term randomised trials of statin therapy

Background: Randomised trials have shown that statin therapy reduces the risk of major vascular events (ie, heart attacks, strokes and coronary revascularisation procedures) without any increase in the risk of nonvascular causes of death or of site-specific cancer, but does produce a small increase in the risk of muscle pain or weakness, diabetes and, possibly, haemorrhagic stroke. Although statins are widely prescribed, there are concerns that they might have a range of other side effects. This DPhil addresses these concerns through an individual-participant-data meta-analysis of all recorded adverse events (AEs) in all large, long-term, randomised, double-blind trials of statin therapy, taking into consideration the substantial challenges related to multiple hypotheses testing (MHT).
Methods: Double-blind randomised trials of statin therapy with at least 1,000 participants and a scheduled treatment duration of at least 2 years were included. Individual participant data on all AEs reported in 19 trials of statin vs placebo (123,940 randomised participants) and 4 trials of a more intensive versus a less intensive statin regimen (30,724 randomised participants) were analysed. A literature review identified potentially relevant MHT methods and simulation studies were done to assess their expected performance (ie, control of false positive [FP] and false negative results). Adverse event data were organised and coded according to a common medical dictionary based upon the Medical Dictionary for Regulatory Activities (MedDRA). Under the selected strategy, inverse-variance-weighted meta-analyses of the effects on all AEs were performed using time-to-event analyses for the first occurrence of each outcome among participants randomly assigned into each trial.
Results: The literature review identified 6,569 eligible papers, of which 337 were included for full text review. Five MHT methods (Holm, Hochberg, Hommel, Benjamini & Hochberg and Mehrotra & Adewale) were selected as the most appropriate based on their statistical control of type I and II error rates. Simulation analyses identified the Mehrotra & Adewale method, which controls the false discovery rate (FDR), as the most suitable as it resulted in a low expected number of FP results whilst maintaining reasonable statistical power to detect any real effects of statin therapy. Blinded (ie, using a ‘shuffled’ treatment allocation) meta-analyses of the trials of statin vs placebo were first done, which confirmed that no correction for MHT resulted in 179 (4.4%) FP findings while the Mehrotra & Adewale MHT method led to zero FPs. Unblinded analyses (ie, using the actual treatment allocation) of all non-lipid-related AEs then confirmed the already known beneficial effects of statins on major vascular events. They also showed that statin therapy was FDR-significantly associated with a reduced risk of having an arteriogram or an angiogram procedure. Subsequent analyses which ignored the already known beneficial or harmful effects of statin therapy, very rare outcomes and irrelevant overlap in tests, identified two new benefits of statin therapy: reductions in the risk of peripheral embolism and thrombosis (361 [0.6%] vs 451 [0.7%], rate ratio [RR] 0.72, 95% confidence interval [CI]: 0.61−0.83, p=0.0012), and carotid endarterectomy (47 [<0.1%] vs 83 [0.1%], RR 0.57, 95% CI: 0.40−0.80, p= 0.0013). However, statins were also FDR-significantly associated with an increased risk of hepatobiliary investigations (1,349 [2.2%] vs 1,025 [1.7%], RR 1.32, 95% CI: 1.22–1.43, p<0.0001) and the extremely rare outcome parosmia (18 vs 2, p=0.00036). The meta-analyses of the 4 trials of more vs less statin revealed that more intensive statin therapy was associated with an FDR-significantly reduced risk of coronary artery disorders (3,518 [22.9%] vs 3,779 [24.6%], RR 0.92, 95% CI: 0.88−0.96, p= 0.00025), general system disorders (5,098 [33.1%] vs 5,682 [37.1%],RR 0.87, 95% CI: 0.84−0.91, p<0.0001), any surgical and medical procedures (4,067 [26.4%] vs 4,333 [28.3%],RR 0.93, 95% CI: 0.89−0.97, p= 0.00059) and the rare AE of renal therapeutic procedures (30 [0.2%] vs 57 [0.4%], RR 0.54, 95% CI: 0.35−0.82, p= 0.0038). More intensive statin regimens were also FDR-significantly associated with an increased risk of hepatobiliary investigations (502 [3.3%] vs 253 [1.6%], RR 1.95, 95% CI: 1.69–2.25, p<0.0001) and high blood creatine phosphokinase (223 [1.4%] vs 146 [1.0%], RR 1.51, 95% CI: 1.24−1.86, p<0.0001).
Conclusion: These findings highlight the importance of using appropriate statistical methods to control for multiple testing. The results confirmed the already known benefits of statin therapy while identifying some new potential benefits and harms. However, the nature and frequency of the newly detected harms confirm that the benefits of statin therapy greatly outweigh the potential harms.

Advisor: Emberson, J
Reith, C
Baigent, C