Abstract: From the 9330 reports found by our searches, 39 studies were eligible for this review with effect estimate and measure of precision extracted for 190 comparisons of IPD-MA and AD-MA. We classified the quality of studies as ‘no important flaws’ (29 (74%) studies) or ‘possibly important flaws’ (10 (26%) studies). A median of 4 (interquartile range (IQR): 2 to 6) comparisons were made per study, with 6 (IQR 4 to 11) trials and 1225 (542 to 2641) participants in IPD-MAs and 7 (4 to 11) and 1225 (705 to 2541) for the AD-MAs. One hundred and forty-four (76%) comparisons were made on the main treatment effect meta-analysis and 46 (24%) made using results from analyses to explore treatment effect modifiers. There is agreement in statistical significance between the IPD-MA and AD-MA for 152 (80%) comparisons, 23 of which disagreed in direction of effect. There is disagreement in statistical significance for 38 (20%) comparisons with an excess proportion of IPD-MA detecting a statistically significant result that was not confirmed with AD-MA (28 (15%)), compared with 10 (5%) comparisons with a statistically significant AD-MA that was not confirmed by IPD-MA. This pattern of disagreement is consistent for the 144 main effect analyses but not for the 46 comparisons of treatment effect modifier analyses. Conclusions from some IPD-MA and AD-MA differed even when based on identical trials, participants (but not necessarily identical follow-up) and treatment effect measures. The average difference between IPD-MA and AD-MA in z scores, ratio effect estimates and standard errors is small but limits of agreement are wide and include important differences in both directions. Discrepancies between IPD-MA and AD-MA do not appear to increase as the differences between trials and participants increase. IPD offers the potential to explore additional, more thorough, and potentially more appropriate analyses compared to those possible with AD. But in many cases, similar results and conclusions can be drawn from IPD-MA and AD-MA. Therefore, before embarking on a resource-intensive IPD-MA, an AD-MA should initially be explored and researchers should carefully consider the potential added benefits of IPD.

Plain-language summary: Meta-analysis is a statistical technique to combine results from separate research studies. A meta-analysis can be performed using summary data published in a study report, referred to as aggregate data (AD), or using data collected on each individual participant in the study, referred to as individual participant data (IPD). A meta-analysis of individual participant data (IPD-MA) can take longer and be more expensive than a meta-analysis of aggregate data (AD-MA), but the IPD-MA can be more reliable and can answer much more detailed questions than an AD-MA. We searched for studies, published up to 7 January 2016, that compared results of IPD-MA with AD-MA. We found that four times out of five, similar conclusions can be drawn, but in one out of five cases the two different types of meta-analyses gave different results and conclusions. As we could not reliably identify when an IPD-MA and AD-MA will differ most using these studies, we recommend that an AD-MA should be done first before doing an IPD-MA. If there are shortcomings with the AD-MA, researchers should then consider the possible benefits of IPD whilst remembering the extra work involved.