Health Information Technology: An Updated Systematic Review With a Focus on Meaningful Use

In the United States, adoption of health information technology (IT) has been accelerated by the meaningful use incentive program, which provides financial incentives to individual health care providers and organizations that demonstrate that they use certified health IT to meet a set of several use criteria specified by the Centers for Medicare & Medicaid Services (1, 2). This increase in use has been accompanied by a concomitant increase in the number of published evaluations of health IT. Because of the rapidly expanding evidence base, the Office of the National Coordinator requested a systematic update of the literature (3). The objective of this review is to update previous reviews (46) and examine recent evidence that relates health IT functionalities prescribed in meaningful use regulations to health care quality (including process, health, and patient and provider satisfaction outcomes), safety (including medication safety and other manifestations of patient safety), and efficiency (including costs, utilization, timeliness, and time burden of health care). Methods Although we did not develop a formal protocol for this update, we based it on the search strategy, inclusion and exclusion criteria, data collection, and synthesis methods from previous reviews on health IT (46). Data Sources and Search Strategy A 5-person technical expert panel, which included academic, health care delivery, and policy experts in health IT, guided the review process. Literature searches were based on the strategy initially used by Chaudhry and colleagues (4) and updated by Goldzweig (5) and Buntin (6) and their respective colleagues. This strategy uses broad-based search terms for the English-language literature indexed in PubMed. Our initial search covered the period of January 2010 to November 2011. We used a computer-aided screening method (7) to update that search to November 2012, and then updated searches again to August 2013 (Tables 1 and 2 of the Supplement). Our expert panel reviewed the search results and suggested additional articles that may have been missed. Supplement. Tables Study Selection Two expert reviewers used a Web-based system, DistillerSR (8), to independently select studies. Following the methods of our previous reviews, we considered hypothesis-testing studies of health IT effects and descriptive quantitative studies for inclusion. We classified articles as hypothesis-testing if the investigators compared data between groups or across periods and used statistical tests to assess differences. Hypothesis-testing studies were further classified by study design (such as randomized, controlled trials [RCTs]). To be included, a study needed to evaluate a health IT functionality encompassed by the meaningful use regulations. The meaningful use requirements specify 25 criteria for health IT functionality and use (such as Use CPOE for medication orders), of which providers must meet a portion to receive incentive payments (Tables 3 and 4 of the Supplement). Data Synthesis and Analysis Using a structured form, we abstracted information about the following: study design; clinical setting, health care conditions, and aspects of care assessed; research sites; health IT type (commercial or homegrown); meaningful use functionality evaluated; and context and implementation details. We adapted criteria developed to assess health IT applications in patient safety to classify articles according to reported context (4 domains) and implementation details (7 specific components) (9). We used a modified version of the outcome result classification framework (positive, mixed-positive, neutral, or negative), originally used by Buntin and colleagues (6), to assess outcomes. Our adaptations made the classification framework more conservative than the original framework, thus increasing the likelihood that an article's findings would be classified as mixed-positive, neutral, or negative (Table 5 of the Supplement). The functionality evaluated, context domains, implementation components, and article outcomes were classified by dual-review, and conflicts were resolved by consensus. Role of the Funding Source This project was done under contract to the Office of the National Coordinator. Representatives of the Office of the National Coordinator were briefed on study findings and reviewed a draft manuscript but were not involved in the analysis or decision to submit the manuscript for publication. Results We deemed 2482 of 12 678 titles identified in the searches potentially relevant. We excluded 2023 of these after abstract review and another 223 after full-text review. The 236 articles selected for review encompassed 278 outcomes because some articles addressed several aspects of care and outcomes (Figure). The full list of the included studies can be found in the reference list in the Supplement. Quality outcomes (n= 170) were evaluated more than safety (n= 46) and efficiency (n= 62) outcomes combined, and care processes (n= 103) outcomes were more than twice as common as health outcomes (n= 47). Studies with a simple prepost design were most common (31%), followed by RCTs (25%) and time-series studies (11%). More than one half (53%) evaluated commercial health IT products, although approximately one quarter did not report whether the products that were evaluated were commercial or homegrown. Figure. Evidence search and selection. Meaningful Use Functionalities Most studies addressed clinical decision support (CDS)(n= 85 [36%]); computerized provider order entry (CPOE) (n= 49 [21%]); or multifunctional health IT interventions (n= 47 [20%]), which evaluated broad IT interventions, such as electronic health records (EHRs), that encompassed many of the functionalities required under meaningful use. Twelve of the 25 functionalities in the meaningful use regulations, such as capacity to track vital signs or maintain medication allergy lists, were not specifically evaluated in any studies. These features, however, were likely critical to the functionality of IT interventions (such as CDS) that were evaluated in many studies. Nevertheless, for some aspects of meaningful use, such as implement systems to protect privacy and security, no eligible studies were found. Quality Outcomes Overall, 147 articles assessed the effect of health IT on 170 quality-related outcomes (Tables 6 and 7 of the Supplement). More than one half of these studies assessed CDS alerts and reminders; the most commonly studied outcomes were medication management, screening and preventive care, and process quality for diabetes and venous thromboembolism. More than three quarters of studies of alerts and reminders, including large quasi-experimental studies and RCTs, reported positive effects. Notable among these included a 1-year prospective, cluster randomized trial in 12 primary care pediatric practices that reported that EHR-embedded CDS substantially improved the use of asthma control medications, spirometry, and the maintenance of care plans (10); a study of nearly 20 000 surgical patients that found that CDS was associated with a 30% increase in adherence to infection prevention guidelines (11); a controlled before-and-after study compared 360 primary care physicians in New York, New York, with a matched set of control physicians and reported that EHR-sensitive measures of process quality (such as eye examinations and urine testing for patients with diabetes) improved substantially only in practices that received high levels of technical assistance in implementing their EHRs (12); and a before-and-after study showing that a CDS intervention in a commercial health IT system was associated with a significant increase in venous thromboembolism prophylaxis and a substantial decline in the rate of venous thromboembolism among nearly 40 000 patients admitted to a single academic medical center (13). Eighteen percent of studies reported no statistically significant improvements in quality or even negative effects. An illustrative example of these studies reported that on the basis of a nationally representative sample of physician office visits, EHRs were associated with poor depression care among patients diagnosed with multiple chronic conditions. The authors of this study hypothesized that EHR workflows gave precedence to the treatment of physical conditions, and therefore psychosocial problems were left unaddressed (14). Safety Outcomes Forty-six studies investigated the effects of health IT functionalities on patient safety outcomes, focusing exclusively on medication safety (Tables 8 and 9 of the Supplement). Approximately 78% of these studies reported at least some positive effects. Of note, benefits were found for a wide range of medication safety outcomes in various care settings. Automated dose calculation features within CPOE systems were found to have significant relative reductions in errors in medication dosage ranging from 37% to 80% (1517), and authors of 1 study reported that innovative features, such as incorporation of an order verification screen with a patient picture, was associated with complete elimination of incorrect patient orders (18). However, a few studies reported that, in some cases, health IT did not have the desired effect on medication safety, and alert fatigue and incongruent workflows were described as barriers to successful use of these systems. Efficiency Outcomes We identified 58 articles that assessed the effect of health IT on 62 efficiency-related outcomes (Tables 10 and 11 of the Supplement). Cost effects ranged from a 75% decrease to a 69% increase in the targeted costs. A few of the studies clustered in the range of 6% to 12% increases in the targeted costs. Understanding the relationship between health IT and health care utilization is complicated by external factors, such as the payment environment in which the health care providers operate. We identified some large studies that reported that patient and provider acces