DOI: 10.3886/icpsr39489
title: Integrating Causal Inference, Evidence Synthesis, and Research Prioritization Methods [Methods Study], United States, 2013-2018
title: Archival Version

description: Comparative effectiveness research compares two or more treatments to see which one works better for certain patients. For example, research can see if medicines or stents work better for people with heart problems. Such research may include:

Observational studies. A research team studies what happens when patients and their clinicians choose the treatments. Traits, such as age or health, may affect patients' treatment choices. These traits may also affect patients' responses to treatments. It may be hard for the team to tell if a patient's traits, the treatment, or a mix of the two affected how well the treatment worked.
 Clinical trials. The team assigns patients to a treatment by chance. Traits may affect a patient's ability to join a clinical trial.

In this study, the team tested ways to improve understanding of which treatment works better. First, the team compared different methods that account for things, such as patients' traits, that could affect results of observational studies. In the second part of the study, the team worked on ways to use all available data with a method called meta-analysis. This method combines data from both study types.
description: To advance statistical methodologies for comparative effectiveness research by (1) comparing direct and indirect risk adjustments to observational data for causal inference and (2) developing network meta-analytic methods to integrate data from randomized controlled trials (RCTs) and observational studies in the first aim for evidence synthesis.
description: Observational studies and RCTs can compare treatment benefits and harms. However, these alternative comparative effectiveness research study designs may yield conflicting conclusions due to differences in eligibility criteria and other confounding factors. This empirical study compared and developed improved statistical methods for comparative effectiveness research.
Researchers obtained patient data from the Duke Databank for Cardiovascular Disease for three coronary artery disease treatments. First, researchers studied methods to adjust for treatment selection bias with missing data in observational survival data to see if the methods altered causal inferences about relative treatment benefits. To control for confounders in individual-level registry data, researchers compared five regression methodologies (Cox proportional hazards model and two propensity score matching methods: optimal full propensity matching and inverse probability of treatment weighting, each with two methods for combining imputations). Researchers investigated whether these methods led to different comparative effectiveness estimates of the three treatments and whether patient eligibility for trial participation could partly explain such differential effects.
Next, researchers developed a comprehensive evidence synthesis method called cumulative network meta-analysis that integrated RCT evidence with study- and individual-level registry and observational data. This approach continuously summarized and updated evidence as new studies emerged, offering improvements in power to detect treatment effects and to generalize inferences.
A stakeholder panel met periodically to review results and provide feedback on methodological approaches. The panel included physicians, frequentist and Bayesian statisticians, database researchers, journal editors, and members of the PCORI Methodology Committee.
description: Datasets: 
DS1: Integrating Causal Inference, Evidence Synthesis, and Research Prioritization Methods [Methods Study], United States, 2013-2018