Rejoinder to commentaries on ‘Multivariate meta‐analysis: Potential and promise’

We would like to thank our discussants for their contributions and we are grateful for the opportunity to respond. We attempted to provide a balanced account of the advantages and limitations of multivariate meta-analysis but are aware that, as the authors of such an article, we are almost inevitably enthusiastic about the multivariate methods. Larry Hedges’ enthusiasm seems to be at least as great as ours, however, and he emphasizes and extends the type of advantages that we describe in Section 5.2. 2. Larry’s comments also make it clear that multivariate meta-analysis has applications outside medicine, and indeed one of the earliest applications of the approach was in the field of education [1]. Another advantage of the multivariate methods that we consider especially important is the ‘borrowing of strength’ that we refer to in our article and we thank David Cox for his three (extreme) cases that clarify the issues involved, echoing analytic examples elsewhere [2, 3]. We also recognize the potential limitations of multivariate meta-analysis. Martin Bland’s commentary carefully describes thought processes that are very similar to ours when assessing the arguments for and against the multivariate methods. Both Martin and David refer to our third example, which we deliberately included in the hope of provoking a response, and we agree with David that this example is quite frightening when you consider the small number of studies and the number of parameters being estimated. We hope that this will encourage the medical statistical community to be very careful if, and when, they choose to adopt the multivariate methods. We agree with Antonio Gasparrini and Ben Armstrong that reduced models for the between-study variance structure are likely to be needed for such examples with limited amounts of data, and the best way to proceed in such instances is an important area for future research. We emphasized four main areas of application. Roger Harbord draws further attention to diagnostic test meta-analysis, and Antonio and Ben focus on multiple parameter models in observational studies. Both of these commentaries describe more sophisticated methods than we chose for our focus, and they explain how these may be and have been applied. We find these accounts encouraging as they highlight the use and application of multivariate meta-analysis in practice. Although none of our examples involve clinical trial data, we agree with Martin that this is another important application. We also agree with Martin’s point that multivariate methods are yet to become popular when analysing individual studies, but they are valuable in multivariate meta-analysis using individual patient data (IPD). For example, IPD multivariate meta-analysis has been considered in the context of longitudinal data, where treatment effects at multiple time-points are of interest [4]; indeed, longitudinal data are perhaps one example where individual studies do already use multivariate analysis methods, to account for the correlation between repeated patient measurements. We are not sure that we would encourage the routine use of multivariate methods either within studies or in meta-analysis, preferring instead a selective approach that asks whether there is anything substantial to gain from these methods: but the gains from multivariate methods may be greater in meta-analysis, especially in situations where some studies do not provide all outcomes. Roger discusses the kinds of Bayesian methods that we also described in Section 4.2. 2 and notes that these may be especially useful when data are sparse. We agree, but also add a note of caution because we anticipate that inferences will be especially sensitive to the prior specification in such instances. However, some of the arguments made by us and our discussants seem to involve an implicit prior belief that the outcomes we have considered should be positively correlated between studies. Bayesian methods provide a means to directly incorporate this belief. Roger also discusses the visual display of