DOI: 10.3886/icpsr39479
title: Modeling Strategies for Observational Comparative Effectiveness Research: What Works Best When? [Methods Study], 2013-2018
title: Archival Version

description: Comparative effectiveness research compares two or more treatments to see which one works better for which patients. In some studies, researchers assign patients by chance to several treatments or to have or not have a treatment. But approaches that assign patients by chance are not always suitable. For example, assigning patients to a new treatment may not be good medical care.
For this reason, researchers sometimes do studies using data collected when patients and their doctors choose the treatments. Data from such studies are observational data. When using observational data for research, it can be hard to know if the effect of a treatment is because of the treatment or other factors, such as patients' age or health history. In these cases, researchers use statistical methods to understand the effect of the treatment. Depending on the study's focus and design, some methods work better than others.
In this study, the research team developed guidance for researchers to help them choose methods for their study.
 To access the methods and software, please visit the DECODE CER Tool website.
description: To develop guidance for researchers for selecting and applying appropriate statistical methods to analyze observational data in comparative effectiveness research (CER). The study aims were the following:

 systematically review existing literature and seek stakeholder input to identify studies from the literature on methods for causal inference in
the setting of observational data used for CER; 
 analyze simulated data to further characterize statistical properties of common propensity score-based methods; 
 develop and disseminate a Decision Tool for Causal Inference and Observational Data Analysis Methods in Comparative Effectiveness Research (DECODE CER) to guide clinical researchers and statisticians in using PS-based methods and instrumental variables (IVs).
description: Clinical researchers analyzing observational data face challenges in trying to determine which statistical analysis options are appropriate for supporting causal inferences, given their specific questions and data. Options include propensity score matching; weighting and subclassification; doubly robust methods; instrumental variables; and methods for handling data with treatment regimens that could change over time, or time-varying treatments. Studies indicate that the method used may affect the findings. For results that are consistent with the goals of a study, it is important to align an appropriate method to the study question and data.
To guide researchers, the research team conducted a systematic review of the literature, identifying important considerations in selecting and applying statistical methods for causal inference with observational data. In addition, the team conducted simulation studies to investigate the statistical properties of propensity-score-based approaches. Results from the systematic review and simulations guided the development of a decision guide to aid researchers in choosing appropriate methods for CER.
In the systematic review, the research team identified studies of causal inference methods specific to the use of observational data in CER. The review focused on studies with a single binary treatment and a single binary outcome. The team searched the PubMed, EMASE, PsycINFO, and Current Index to Statistics databases for simulation studies or theoretical findings that assessed control of bias and precision in analyzing observational data. Simulation studies compared five different propensity-score-based methods.
Researchers with expertise in statistics, social epidemiology, health policy, outcomes effectiveness research, pharmacy, and physical therapy provided input on literature search strategies, analytic issues, and the decision guide.
description: Datasets: 
DS1: Modeling Strategies for Observational Comparative Effectiveness Research: What Works Best When? [Methods Study], 2013-2018