JMIR MEDICAL INFORMATICS

Hersh et al

Original Paper

Outcomes From Health Information Exchange: Systematic Review
and Future Research Needs
William R Hersh1, MD; Annette M Totten1, PhD; Karen B Eden1, PhD; Beth Devine2, MBA, PhD, PharmD; Paul
Gorman1, MD; Steven Z Kassakian1, MD; Susan S Woods3, MD; Monica Daeges1, BS; Miranda Pappas1, MA; Marian
S McDonagh1, PharmD
1

Pacific Northwest Evidence-Based Practice Center, Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University,
Portland, OR, United States
2

Centers for Comparative and Health System Effectiveness, University of Washington, Seattle, WA, United States

3

Veteran's Affairs Maine Healthcare System, Augusta, ME, United States

Corresponding Author:
William R Hersh, MD
Pacific Northwest Evidence-Based Practice Center
Department of Medical Informatics & Clinical Epidemiology
Oregon Health & Science University
3181 SW Sam Jackson Park Rd.
BICC
Portland, OR,
United States
Phone: 1 503 494 4563
Fax: 1 503 494 4551
Email: hersh@ohsu.edu

Abstract
Background: Health information exchange (HIE), the electronic sharing of clinical information across the boundaries of healthcare organizations, has been promoted to improve the efficiency, cost-effectiveness, quality, and safety of health care delivery.
Objective: To systematically review the available research on HIE outcomes and analyze future research needs.
Methods: Data sources included citations from selected databases from January 1990 to February 2015. We included
English-language studies of HIE in clinical or public health settings in any country. Data were extracted using dual review withadjudication of disagreements.
Results: We identified 34 studies on outcomes of HIE. No studies reported on clinical outcomes (eg, mortality and morbidity)
or identified harms. Low-quality evidence generally finds that HIE reduces duplicative laboratory and radiology testing, emergency
department costs, hospital admissions (less so for readmissions), and improves public health reporting, ambulatory quality ofcare, and disability claims processing. Most clinicians attributed positive changes in care coordination, communication, andknowledge about patients to HIE.
Conclusions: Although the evidence supports benefits of HIE in reducing the use of specific resources and improving the qualityof care, the full impact of HIE on clinical outcomes and potential harms are inadequately studied. Future studies must addresscomprehensive questions, use more rigorous designs, and employ a standard for describing types of HIE.
Trial Registration:
PROSPERO Registry No CRD42014013285; http://www.crd.york.ac.uk/PROSPERO/
display_record.asp?ID=CRD42014013285 (Archived by WebCite at http://www.webcitation.org/6dZhqDM8t).
(JMIR Med Inform 2015;3(4):e39) doi: 10.2196/medinform.5215
KEYWORDS
diagnostic tests; health information exchange; outcome assessment (health care); patient readmission; routine; systematic review

http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 1
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Introduction
In recent years, there has been substantial growth in the adoptionof the electronic health record (EHR) in ambulatory and hospitalsettings across the United States, fueled largely by incentive
funding provided by the Health Information Technology for
Economic and Clinical Health (HITECH) Act. Following
HITECH, 94% of nonfederal hospitals [1], 78% of
hospital-based physicians [2], 84% of emergency departments(EDs), and 73% of hospital outpatient departments in the UnitedStates have adopted EHRs [3]. The motivation to increase theadoption of EHRs is grounded in evidence that healthinformation technology (HIT) can improve the quality, safety,
efficiency, and satisfaction with care, as has been reported in aseries of systematic reviews [4-7].
One key challenge to effective use of HIT, however, is that mostpatients in the United States, especially those with multiple
conditions, receive care across a number of settings [8,9]. To
enable data to follow patients wherever they receive care,attention has recently focused on health information exchange(HIE), defined as the reliable and interoperable electronic
sharing of clinical information among physicians, nurses,
pharmacists, other health care providers, and patients across theboundaries of health care institutions, health data repositories,
laboratories, public health agencies, and other entities that are
not within a single organization or among affiliated providers[10].
The Office of the National Coordinator for Health Information
Technology (ONC) has defined the following forms of HIE
[11]:
1.
2.
3.

Directed exchange: Sending and receiving secure
information electronically between care providers.
Query-based exchange: Provider-initiated requests for
information on a patient from other providers.
Consumer-mediated exchange: Patients aggregating andcontrolling the use of their health information among careproviders.

ONC also uses the words “push” to describe directed exchange
and “pull” to describe query-based exchange [12]. ONC
leadership has also advocated that HIE be thought of as a verb
and not as a noun, with more focus on the action of exchange
and what is achieved with the information than on thetechnological and organizational structures required [13]. This
is not meant to imply that the structures are not necessary, rather
it is designed to shift the focus when evaluating HIE fromdocumenting what has been created to the impact HIE has on
health and health care.
The HITECH Act recognized that EHR adoption alone wasinsufficient to realize the full promise of HIT, allocating US$563 million for states or state-designated entities to establish
HIE capability among health care providers and hospitals [11].
As a result of HITECH funding, HIE adoption has grown in aparallel though somewhat smaller manner. By 2014, 76% of
US hospitals had engaged in some form of HIE [14]. An annual
survey of organizations engaged in HIE found 135 in the UnitedStates in 2014 [15].
http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

Hersh et al
Evaluating the effectiveness of HIE (and HIT generally) has
been challenging [16]. HIE systems are intermediate toimproving care delivery, allowing clinicians and others
improved access to patient data to inform decisions, andfacilitate appropriate use of testing and treatment. HIE is not
specific to any health issue or diagnosis. HIE implementations
have often been supported by one-time start-up funding, without
long-term support to sustain the programs long enough forevaluation.
There are 3 previously published systematic reviews that focus
exclusively on HIE [17-19]. One of these reviews was conducteda half-decade ago [17], another focused only on US-based andclinical-only (ie, not public health) activities [18], and a thirdassessed mainly the associations between study characteristics
and the frequency of positive outcomes [19]. We expanded uponthese reviews to not only perform a systematic review of HIE
but also determine needs for future research that reflect ourassessment of the benefits and limitations of HIE.

Methods
Key questions guiding this review were developed by the review
team with input from a group of stakeholders and the Agency
for Healthcare Research and Quality (AHRQ). A standard
protocol was developed using input from key informants and atechnical expert panel, registered in PROSPERO [20], andposted on an AHRQ public website. A technical report further
describes the methods and includes search strategies andadditional information [21]. A research librarian conducted
electronic database searches identifying relevant articles
published between January 1990 and February 2015 in
MEDLINE (Ovid), PsycINFO, CINAHL, and the CochraneLibrary databases. Searches were peer reviewed by anotherlibrarian and supplemented by references identified fromadditional sources, including reference lists, table of contents
of journals not indexed in databases searched, gray literaturesources, and experts. English-language studies of HIE thatreported on clinical, economic, population, and intermediate(eg, patient or provider perceptions, availability or accuracy of
data, or time saved) outcomes were included. We included
comparative studies of effectiveness, and other designs for morequalitative outcomes. We excluded studies that investigatedbenefits of HIE other than in clinical or public health settings(eg, to enhance clinical research). Two investigators
independently evaluated each study to determine inclusion
eligibility. Disagreement was resolved by consensus with a third
investigator making the final decision as needed.
Details of included studies were extracted by one investigatorand reviewed for accuracy and completeness by a second
investigator. Two investigators independently assessed risk ofbias for all effectiveness studies. Differences were resolved bydiscussion and consensus and reviewed by the team ofinvestigators. Individual studies were rated as “low,”“moderate,” or “high” risk of bias. Investigators then assessed
the strength of the body of evidence. Both the risk of bias andstrength of evidence ratings were conducted using the criteria
and procedures described in the AHRQ Methods Guide for
Effectiveness and Comparative Effectiveness Reviews [22].
JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 2
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Hersh et al

The strength of evidence consisted of the following 4 major
categories: high, moderate, low, or insufficient, based on themethodological limitations of studies; consistency across studies;precision of estimates; and directness of effect. Ratings werereviewed by a second investigator, and disagreements wereresolved by consensus or involvement of a third investigator if
necessary. Data could not be combined in a quantitative
meta-analysis because of heterogeneity in the interventions, theoutcomes measured, and the way data were reported. Therefore,
we combined studies qualitatively based on the similarity ofthe type of HIE, the implementation of the HIE, outcomesmeasured, and results reported. Where studies were not similarin these areas, we provided the results of the individual studies
without grouping them.

specific outcome (Table 1). The next most common design was
survey, which was usually focused on perception ofeffectiveness and perceived outcomes: 2 studies were RCTs—1
RCT assessed a particular directed information exchange (2published papers, 1 on clinical outcomes, and 1 on perceptions)and the other evaluated a clinical decision support intervention
using data from an HIE implementation. Two studies usedcross-sectional analyses of large databases to compare healthcare organizations having access to HIE with those without
access. Two other studies used a case series methodology, oneof which involved asking clinicians if HIE access avoided
undesirable resource use, and then calculating the costs saved
and the other that retrospectively analyzed data to determine
duplicative testing averted.

Results

The identified studies were performed mostly in the UnitedStates, but we identified 8 studies from 5 other countries. Of
the 26 studies in the United States, 2 assessed multiple HIE
implementations across the entire United States, 1 assessed
multiple HIE implementations in 2 states (California andFlorida), and the remaining 23 studies were conducted in 13
states. Most studies used retrospective designs, usually with anapproach examining the association of HIE use with 1 or moreclinical variables. All of these studies focused on the direct
effect of HIE, usually reporting reduction in resource use or
costs, without determining its larger impact (eg, overall total or
proportion of spending in an ED vs the total dollars that HIE
appeared to save). None of the studies analyzed individual
episodes of care to determine clinical appropriateness of possible
changes brought about by HIE use.

Of the 5211 potentially relevant citations identified in ourliterature searches, 849 articles were selected for full-text review
and 34 studies were ultimately deemed to address outcomes.
Study characteristics, results, and risk of bias assessments arepresented in Multimedia Appendices 1 and 2. Of the studies
included in this report, 2 were randomized controlled trials(RCTs) described in 3 papers and 32 were observational andsurvey studies. Most were conducted in the United States,
although 8 were from Europe, Canada, Israel, and South Korea.
These studies reported clinical or public health process,economic, or population outcomes; however, none of the studies
explicitly stated that they assessed for harms of HIE or reported
any negative unintended consequences. The majority wereassessed to be of low risk of bias (ie, good internal validity) but
also contained mostly retrospective observational evidence.
Of 34 studies, 26 reported clinical, economic, or population
outcomes (see Multimedia Appendix 1), whereas the other 8were found to report on perceptions of outcomes (seeMultimedia Appendix 2). None of the studies evaluated primaryclinical outcomes from HIE (eg, mortality and morbidity) nor
explicitly measured or reported harms. We list the study designs
and geographic locations in Table 1.

The prospective studies also had limitations. The 2 RCTs
(reported in 3 papers) were focused on highly specific uses of
HIE, namely, directed exchange of ED reports in one andpharmacotherapy clinical decision support in another. Of note,however, was that neither study showed benefit of HIE. The
other prospective study was a case series that was limited byits methodology relying on physician self-reports of resources
not utilized when HIE was used, with no follow-up or validationof their decisions, or analysis of more holistic views of clinicaloutcomes or costs.

The most common study design for assessing outcomes was
retrospective cohort, typically with HIE use associated with a

http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 3
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Hersh et al

Table 1. Study designs and locations.
Study designs and locations

References

Retrospective cohort (18)[23-40]Survey (8)[41-48]

Randomized controlled trial (2 reported in 3
papers)[49-51]

Cross sectional (2)[52,53]

Case series (2)[54,55]Austria (1)[47]Canada (2)[49,51]Finland (2)[23,46]Israel (2)[29,56]South Korea (1)[48]

All of United States (2)[41,53]

California and Florida[52]

Colorado (1)[24]Indiana (3)[35,36,44]

Louisiana (1)[34]Massachusetts (1)[45]Minnesota (1)[55]North Carolina (1)[50]

New York (6)[32,33,37,40,42,43]

Oklahoma (1)[38]South Carolina (1)[54]Tennessee (3)[25,27,28]

Texas (1)[31]

Virginia (1)[39]

Wisconsin (2)[26,30]Designs (number)

Location (number)

Most of these studies had reasonable but not strong internal
validity. As the intervention (HIE) was only one of manypotential influences on clinical outcome (ie, many more factors
go into clinical outcomes than the decision to consult an HIE
on a patient), there was possible confounding. Because no
confounders were explicitly identified and incorporated into
the analyses, most studies with appropriate retrospective
methods were rated as having low or moderate risk of bias.
Because of the type of study designs used, reporting limitations,and the lack of ability to combine results, the strength of thisbody of evidence was rated as low, meaning that future studies
have the potential to alter these findings in magnitude or
direction. In addition, the number of studies and their locationsin the United States represent a small fraction of functioning
HIE systems. A larger number are reported to be operational,
sustainable, or innovating according to the eHealth Initiative
Annual Data Exchange Survey, which reported a total of 84
such HIE implementations in 2013 [57] and 106 in 2014 [15].
http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

In other words, while a substantial number of HIE
implementations exist in the United States, only a small number
have been subject to evaluation. This low number of studies
relative to HIE efforts also makes it difficult to generalize about
what aspects of HIE, such as location, type, and setting, areassociated with the results reported in research.

Improving Resource Use
Most of the studies of HIE effectiveness focused on resourceuse. We categorized these as follows (Table 2): laboratory
testing, radiology testing, hospital admissions, hospital
readmissions, referrals and consultations, ED costs, public heathreporting, quality of care, and other aspects of HIE. Although
the risk of bias in most studies was low to moderate, theresulting evidence from them was mostly of low strength due
to retrospective designs. This low-strength evidence mostly
favored the value of HIE in reducing resource use and costs,
especially in the ED, but used a very narrow cost perspective
and did not account for how HIE was used and its impact on
JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 4
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
the overall care of the patient beyond the immediate setting

Hersh et al
where it was used.

Table 2. Study results by categories.
Category (number)

Results

Laboratory testing (6)A total of 6 studies showed benefit for health information exchange (HIE) in reducing overall testing, although estimatesof impact on cost were mixed [23-26,54,55]: 4 studies took place in the emergency department (ED) setting, all showing
some amount of reduced testing and cost savings [25,26,54,55], whereas 2 studies were conducted in ambulatory settings,
with one showing an increase [23] and the other showing a reduction in the increased overall rate of testing [24].

Radiology testing (9)A total of 7 studies carried out in the ED setting showing reduced testing [25-28,52,54,55]; 2 studies were conducted inambulatory settings, with one showing a decrease [23] and the other showing no change in the rate of testing [24].

Hospital admissions (8) A total of 2 studies found a reduction in hospital admissions and lower costs [25,54]; 3 other studies also measured somebenefit for HIE use in reducing hospital admissions [29,32,56], although 3 additional studies found no such reduction[30,31,49].
Hospital readmissions(2)

Whereas 1 study showed benefit for HIE in reducing hospital readmissions [33], the other did not [53].

Referrals and consulta- A total of 2 studies assessed HIE for reducing referrals and/or consultations, with conflicting results [23,54].
tions (2)
ED costs (2)A total of 2 studies found reduced overall ED costs per patient when HIE was available [25,26]. Neither study reported
overall ED expenditures, making it unknown what proportion of overall ED spending was impacted by HIE.

Public heath reporting(3)

A total of 3 studies assessed HIE in public health settings, all of which were conducted in the United States and reported
improved automated laboratory reporting [36], improved completeness of reporting for notifiable diseases [35], and improved
identification of HIV patients for follow-up care [34].

Quality of care in ambu- A total of 2 retrospective studies found HIE associated with improved quality of care [37,38], whereas a randomized controlled
latory settings (3)trial focused on medication reconciliation found increased ability to detect medication adherence problems, the results did
not show improvement in adherence after it was identified and addressed by providers [50].
Other aspects of HIE(3)

A total of 3 studies assessed other aspects of HIE, including reduction in time for processing of Social Security Disability
claims [39], increased ability to identify frequent ED users [40], and associated HIE implementation with improved patientsatisfaction scores in hospitals [41].

Perceptions
A number of studies evaluated clinician or patient perceptionsof outcomes of HIE (see Multimedia Appendix 2), with allreporting perceptions that HIE leads to some benefit including
improved outcomes. Clinician perceptions of the value of HIE,
where studied, were generally positive. However, how such
perceptions translate into improved care is unknown. This bodyof evidence was considered low strength.

Factors Associated With Outcomes
To determine whether effectiveness of HIE varied by study
type, health care setting, location, or HIE type, we examined
whether HIE was found to have some beneficial effect or not
across characteristics. As presented in Table 1, thepreponderance of studies reported that HIE use for different
functions, in various settings, and of varying types produced
mostly positive outcomes. Although the number of positive
versus negative studies was not an indicator of the overall
direction of the evidence, we did note that for each “negative”
study, there was at least one “positive” one. For type of HIE,
there was no clear pattern of findings to suggest that one type
was clearly better than another, even indirectly. The 2 RCTs
reported no benefit for their selected outcomes from HIEintervention [49,50], although a perceptions study from one ofthem reported impressions of improved patient outcomes andmanagement [51]. These were in contrast to the observational
study designs where almost all found beneficial effects of HIE.
For the HIE setting, only ambulatory and ED had enough studies
http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

to evaluate patterns, with outpatient settings less likely to findbeneficial results compared with studies in ED settings. The
sparseness of studies across geographic settings did not allow
for identification of patterns, although across most studies in
the United States, the findings were positive.

Discussion
A collection of low-quality evidence supports the value of HIE
for reducing duplicative laboratory and radiology test ordering,
lowering ED costs, reducing hospital admissions (less so forreadmissions), improving public health reporting, increasing
ambulatory quality of care, and improving disability claims
processing. The evidence is low quality because of theretrospective nature of the studies and the limited questions thatthey ask. It is unlikely that additional studies of the kindincluded in this review will advance the field and strengthen
our understanding when HIE can reduce laboratory and imaging
tests associated with episodes of care without broadening theirscope and using more rigorous designs. Although the
preponderance of evidence reports positive effects of HIE in
reducing resource use and improving quality of care, it is entirely
possible that focused studies with stronger study designs andmore comprehensive assessment of utilization or clinicaloutcomes might reach a different conclusion.
We found no studies explicitly addressing patient-specificclinical outcomes such as morbidity, mortality, or functional
status, and therefore the body of evidence is insufficient todetermine whether HIE has an impact on patient outcomes. We
JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 5
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
also did not identify any studies that used systematic andcomprehensive economic analysis. Although some of the studies
we included projected or estimated cost savings based on
measured changes in utilization or perceptions of clinicians,
there were no studies that explicitly measured costs and assessed
economic impact in a comprehensive fashion. It is fair to say,then, that there was insufficient evidence to reach conclusions
on the economic impact of HIE.

Applicability
How likely are the effects reported in this review to be observed
when applied under diverse conditions in health systems,
hospitals, and clinics in the United States? The greatest
confidence in the applicability of these findings comes fromthe breadth of settings—geographic, organizational, andtechnical—from which they are derived. By contrast, there arelimitations to the applicability of the findings (beyond
limitations to the internal validity already mentioned) having
to do with these main concerns: (1) concentration of evidence
from a relatively small number of HIE systems; (2) use ofinternally developed and refined health IT systems comparedwith local instances of commercial systems; and (3) the
exceptionally broad variety of systems, contexts, and purposesof HIE reported in the studies included in this review.
First, the concern that the bulk of the evidence about health IT
impact arises out of a relatively small number of centers has
been raised before [4]. These centers have been referred to as“health IT leaders,” which are typically large academic medical
centers with internally developed health IT systems,
implemented incrementally, and refined over a long period. The
nature of the health IT systems is in each case unique (being
locally developed), and more importantly it is difficult toseparate the effects of the health IT from the confounding
influences of the health system itself. However, whether findings
from these systems can be generalized to the very different
context of health system and hospital implementations ofcommercially developed systems over shorter periods with lessinternal development and implementation infrastructure has
been called into question [4]. This “health IT leader” effectappears to be reduced in more recent updates to the 2006
systematic review by Chaudhry et al [4] but the issue remains
important [5,7]. In this review of HIE, the concentration ofevidence phenomenon is also present, with large numbers of
published studies emanating from relatively few areas, this time
regional implementation programs rather than academic healthcenters, such as Texas, New York, and the MidSouth e-HealthAlliance.
Second, separate from the “health IT leader” concern, which
has to do with the organizational capacity, resources, andmission of these centers, is the issue of internally developed
systems compared with commercially developed systems.
Although few of the studies we included described whether
their software used was commercial or locally developed, theoverall model of health IT purchase and installation of nonhealth
IT leaders are usually quite different from that of the incremental
internal development, implementation, and refinement that are
seen in systems such as the Department of Veterans Affairs or
the aforementioned “health IT leader” systems. Related to this
http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

Hersh et alconcern is a finding from other aspects of health IT [58], namely,
clinical decision support, where systems evaluated by theirdevelopers tend to achieve more positive outcomes from theirevaluation than external evaluators. This phenomenon must be
assessed with HIE as well.
Third, and most important, in terms of limiting the applicabilityof these findings about HIE to real-world use is the exceptionallywide variety of systems, purposes, and contexts of use. To
predict whether specific implementations of HIE in specifichealth care contexts will have favorable impacts on specificdesired outcomes is not possible from this review and in mostcases would not be possible from comparison with individual
studies because (1) it is unlikely that studies with low risk ofbias have been published for most such specific questions, and(2) in almost all cases these are complex interventions that areincompletely specified, with insufficient detail to draw strong
meaningful inferences [59].

Limitations of the Evidence Base
The significant limitations of the evidence base, that is, theindividual studies included in this review, have been raised inprevious systematic reviews of health IT [4,5,7] and of HIE[18]. There are four primary concerns about the limitations of
the available evidence on the impact of HIE (and health IT in
general): (1) suitability of study design; (2) execution of thestudies; (3) complexity of the interventions with implications
for interpretation and for generalizability; and (4) changes inthe technology or policy governing its use.
First, the evidence in this area addresses a wide variety ofquestions covering diverse domains beyond medical sciencefrom computer science, human factors, sociology, organizationand management, and other disciplines. This broad array ofquestions calls for an equally diverse range of study designs.
Studies of usability and use require usability engineering
methods, studies of individual behavior call for methods fromanthropology and behavioral sciences, studies of organizationalchange warrant methods drawn from management and systems
science, whereas studies of population effects call for the
methods of epidemiologists. A significant limitation of thisliterature, with its breadth of research questions, is the limited
toolbox often drawn upon to answer them.
The second limitation is in execution of the studies. Even whenstrong study designs are used, their execution may be lacking,whether in sampling strategies, measurement methods, or
analytic approaches. The unit of analysis problem is but one
example. Interventions carried out at the level of the healthsystem, hospital, or clinic may be analyzed at the level of thepatient or episode, without controlling for variation at these
multiple levels. Incomplete measurement is another: forexample, where ED test ordering is measured in isolation,
ignoring the possibility that the same test might later be orderedin another setting such as urgent care, primary care, or in
hospital.
The third limitation has to do with the complexity ofinterventions, where the HIE or other health IT system itself isnecessarily only part of a more complex intervention. The
complexity of interventions to change the behavior of clinicians
JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 6
(page number not for citation purposes)

JMIR MEDICAL INFORMATICSor others in the health systems studied requires more thoroughspecification, to both adjust for confounders and make senseout of how to apply interventions elsewhere. Others havedocumented the inadequacy of specification of the details ofcomplex interventions and called for a more systematic andthorough reporting [59,60].
Finally, the literature does not comprehensively cover changes
in technology or policies governing its use. For example,
whereas most studies come from the locally developed systemsof HIE leaders as noted earlier, there has been a more recent
growth in the commercial marketplace for HIE. In addition, thewidespread adoption of EHRs under the HITECH Act in the
US means that a more diverse array of health care organizations
will be participating in HIE implementations. As an example
of policy changes governing HIE development, as noted inTable 1, most studies have been of query-based systems whereas
more recent meaningful use criteria for incentive funding callfor implementation of directed exchange.

Future Research Needs
Given the limited conclusions that can be reached after reviewof the large volume of published literature on HIE, what are theimplications for future research? Recognizing that HIE, likehealth IT in general, will almost certainly undergo increasingly
widespread implementation in the future, the first aim ofresearchers should be to shift the emphasis from whether HIE
systems should be implemented to specifically how they shouldbe implemented. The question to be answered is not “Does HIE
have positive effects?” but rather “How can HIE be implemented
in order to result in the greatest benefit for patients, clinicians,
and health systems with the least cost and harm?”
A second aim of research on HIE should be to develop greater
focus and clarity about the level at which interventions areoperating and the types and levels at which outcomes aremeasured. The outcomes of interest and the factors influencing
them may be quite different at different levels of analysis, from
specific systems or functionalities of HIE to individual patients,
providers, or episodes of care; to health care units such as theED, primary care practice, or hospital ward; to institutions suchas hospitals; to aggregates such as health systems; or to broader
regional multiorganization entities or regions. Combining or
confusing these levels of intervention and levels of analysis
only increases the challenges for those who conduct the research
and for those who wish to interpret and apply it.
To help achieve an improved focus and clarity, a more formal
analytic framework and a more descriptive taxonomy areneeded. An example of such a framework that could be usefully
applied in this area is Rasmussen’s sociotechnical hierarchy,
which specifies the multiple levels of a complex sociotechnicalsystem that must be considered together to understand systembehavior change [61]. Examples of its application include
Vicente’s analysis of the forces acting at multiple levels to
reduce hazards arising from patient-controlled analgesia devices[62] and Leveson’s Systems—Theoretic Accident Modelingand Processes model for understanding system performanceand safety [63].

http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

Hersh et al
Similarly, a formal taxonomy for implementation of complex
interventions has been proposed that would enable more
complete and useful specification of interventions to allow better
analysis, interpretation, and application [59,64]. This taxonomy
should be extended specific to HIE to include clinical, technical,
and organizational details of the HIE implementation as outlined
by Vest [65]. The clinical taxonomy should focus not only on
patient outcomes, but also on issues such as health disparities
related to HIE and health system issues that may improve orundermine use of HIE. The technical taxonomy should include
aspects of system architecture, messaging and terminology
standards, and other details. It should also address the financial
aspects of implementations, such as whether locally developedor commercial software is used and whether the HIE
organization is public or private. The HIE research community
should consider a standardized reporting instrument for HIE
evaluation comparable to the Consolidated Standards ofReporting Trials statement for RCTs [66].
The third step researchers can take to improve the evidence basefor implementation of HIE is to broaden the methodologic
toolbox applied to these questions. As indicated earlier, thestudy approach and architecture must be suited to the question
being asked, employing methods from usability engineering,
behavioral sciences, systems engineering, and organizational
sciences, depending on the question being addressed. These
would include methods used in engineering and quality
improvement, as well as in the study of complex adaptive
systems.
What types of studies should be performed? RCTs are
impractical for technologies with wide-ranging purposes like
HIE. Yet, retrospective studies associating HIE versus nonuse
for outcomes such as test ordering and hospital admissions arevery limited in conclusions that can be drawn. Research is also
challenging because many of the important clinical outcomesthat could be positively affected by HIE have many otherpotential contributing and confounding factors relating to thepatient, his or her clinicians, the quality of care delivered, theEHR, other health IT used, the nature of the health care delivery
system, and the regulatory environment. Given the growing
evidence based on robust evaluations in other areas of healthIT, as noted in systematic reviews [7], methodological insights
can be gleaned from other topic areas.
Future studies should be prospective, carried out in mature HIE
settings, specify a priori what patients and/or use cases are likely
to benefit from HIE, and compare appropriate outcomes for theuse or nonuse of HIE. The prospective collection of data fromdiverse settings where HIE is used, classified by the taxonomy
advocated earlier, could allow for prospective cohort studies
that could identify aspects of HIE associated with beneficial
outcomes. This will likely require an effort comparable in scope
to national data collection efforts, such as the Patient-Centered
Outcomes Research Institute Clinical Data Research Network
initiative [67]. Ideally, such an undertaking could be synergisticwith these other large-scale efforts.
Evaluation should be a requirement for all HIE implementations,certainly those funded by grants or other external funding. The
challenge of evaluating health IT projects, especially in
JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 7
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
community settings, is well-known [16], but all funders must
demand this requirement to grow the evidence base. By thesame token, funders must provide adequate resources for such
evaluations. In addition, evaluations should be performed byresearchers external to the project to reduce potential bias fromsystem developers evaluating their own implementations [58].

Conclusions
The full impact of HIE on clinical outcomes and potential harms
is insufficiently studied, although evidence provides some

Hersh et al
support for benefit in reducing use of some specific resources
and achieving improvements in quality of care measures. To
advance our understanding of HIE, future studies need to addresscomprehensive questions, use more rigorous designs, and bepart of a coordinated, systematic approach to studying HIE.
Going forward, HIE will become a more integrated part of healthcare delivery, and its evaluation needs to be focused on
maximizing the improvements that HIE usage brings to overallclinical care.

Acknowledgments
The authors gratefully acknowledge the following individuals for their contributions to this project: Andrew Hamilton, MLS,MS, for conducting literature searches and Spencer Dandy, BS, for assistance with preparing this manuscript (both are locatedat the Oregon Health & Science University); and Jon White, MD, Task Order Officer at the Agency for Healthcare Research andQuality (AHRQ). The primary funding source was The Agency for Healthcare Research and Quality (Contract Number
290-2012-00014-I, Task Order 11), Rockville, Maryland. SZK was also supported by a grant from the National Library ofMedicine Training (Grant No T15LM007088). The findings and conclusions in this document are those of the authors, who areresponsible for its content, and do not necessarily represent the views of AHRQ or the National Library of Medicine. No statementin this report should be construed as an official position of AHRQ or the US Department of Health and Human Services.

Conflicts of Interest
None declared.

Multimedia Appendix 1
Studies of health information exchange included for assessing outcomes.
[PDF File (Adobe PDF File), 115KB-Multimedia Appendix 1]

Multimedia Appendix 2
Patient and clinician survey perceptions of health information exchange.
[PDF File (Adobe PDF File), 19KB-Multimedia Appendix 2]

References
1.

2.

3.

4.

5.

6.

7.
8.

Dustin C, Meghan G, Furukawa M. ONC Data Brief: Number 16. 2014. Adoption of electronic health record systems
among US non-federal acute care hospitals: 2008-2013 URL: https://www.healthit.gov/sites/default/files/oncdatabrief16.
pdf [accessed 2015-10-07] [WebCite Cache ID 6c6lYtrUT]
Hsiao C, Hing E. NCHS Data Brief Number 143. 2014. Use and characteristics of electronic health record systems among
office-based physician practices: United States, 2001-2013 URL: http://www.cdc.gov/nchs/data/databriefs/db143.htm[accessed 2015-10-07] [WebCite Cache ID 6c6lpDgb3]
Eric J, Esther H. NCHS Data Brief Number 187. 2015. Progress with electronic health record adoption among emergency
and outpatient departments: United States, 2006-2011 URL: http://www.cdc.gov/nchs/data/databriefs/db187.htm [accessed
2015-10-07] [WebCite Cache ID 6c6lwBTBp]
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: Impact of health information
technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006 May 16;144(10):742-752. [Medline:
16702590]
Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: New trends
from the literature. Health Aff (Millwood) 2009;28(2):w282-w293 [FREE Full text] [doi: 10.1377/hlthaff.28.2.w282][Medline: 19174390]
Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: A review of the recent
literature shows predominantly positive results. Health Aff (Millwood) 2011 Mar;30(3):464-471 [FREE Full text] [doi:10.1377/hlthaff.2011.0178] [Medline: 21383365]
Jones SS, Rudin RS, Perry T, Shekelle PG. Health information technology: An updated systematic review with a focus on
meaningful use. Ann Intern Med 2014 Jan 7;160(1):48-54. [doi: 10.7326/M13-1531] [Medline: 24573664]
Bourgeois FC, Olson KL, Mandl KD. Patients treated at multiple acute health care facilities: Quantifying information
fragmentation. Arch Intern Med 2010 Dec 13;170(22):1989-1995. [doi: 10.1001/archinternmed.2010.439] [Medline:
21149756]

http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 8
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
9.
10.
11.

12.

13.

14.

15.

16.

17.
18.
19.

20.

21.
22.
23.

24.

25.

26.

27.

28.

29.

Finnell JT, Overhage JM, Grannis S. All health care is not local: An evaluation of the distribution of emergency department
care delivered in Indiana. AMIA Annu Symp Proc 2011;2011:409-416 [FREE Full text] [Medline: 22195094]
What is HIE (health information exchange)?. 2012. URL: https://www.healthit.gov/providers-professionals/
health-information-exchange/what-hie [accessed 2015-10-07] [WebCite Cache ID 6c6m5qCuS]
Williams C, Mostashari F, Mertz K, Hogin E, Atwal P. From the Office of the National Coordinator: The strategy foradvancing the exchange of health information. Health Aff (Millwood) 2012 Mar;31(3):527-536 [FREE Full text] [doi:10.1377/hlthaff.2011.1314] [Medline: 22392663]
Dullabh P, Ubri P, Loganathan S, Latterner M. Evaluation of the State Health Information Exchange Cooperative Agreement
Program: State Approaches to Enabling HIE: Typology Brief. Washington, DC: Office of the National Coordinator for
Health Information Technology; 2014. URL: https://www.healthit.gov/sites/default/files/statehietypologybrief.pdf [accessed
2015-10-07] [WebCite Cache ID 6c6obU0FC]
Healthcare Information and Management Systems Society. Healthcare IT News. 2015. URL: http://www.
healthcareitnews.com/news/hie-verb-onc-wants-move-quickly-data-exchange [accessed 2015-10-07] [WebCite Cache ID
6c6mPt5aH]
Report N. Health Information Exchange Among US Non-Federal Acute Care Hospitals: 2008-2014 (ONC Data Brief
Number 24). Washington, DC: Office of the National Coordinator for Health Information Technology (ONC); 2015. URL:
https://www.healthit.gov/sites/default/files/data-brief/ONC_DataBrief24_HIE_Final.pdf [accessed 2015-10-07] [WebCite
Cache ID 6c6okXYub]
eHealth I. Post HITECH: The Landscape of Health Information Exchange. 2014 Sep 10. URL: http://visit.medicity.com/
rs/aetnainc/images/2014%20eHI%20Data%20Exchange%20Survey%20Key%20Findings.pdf [accessed 2015-10-07][WebCite Cache ID 6c6mVWlpt]
Kern LM, Ancker JS, Abramson E, Patel V, Dhopeshwarkar RV, Kaushal R. Evaluating health information technology incommunity-based settings: Lessons learned. J Am Med Inform Assoc 2011;18(6):749-753 [FREE Full text] [doi:10.1136/amiajnl-2011-000249] [Medline: 21807649]
Hincapie A, Warholak T. The impact of health information exchange on health outcomes. Appl Clin Inform 2011;2(4):499-507[FREE Full text] [doi: 10.4338/ACI-2011-05-R-0027] [Medline: 23616891]
Rudin RS, Motala A, Goldzweig CL, Shekelle PG. Usage and effect of health information exchange: A systematic review.
Ann Intern Med 2014 Dec 2;161(11):803-811. [doi: 10.7326/M14-0877] [Medline: 25437408]
Rahurkar S, Vest JR, Menachemi N. Despite the spread of health information exchange, there is little evidence of its impact
on cost, use, and quality of care. Health Aff (Millwood) 2015 Mar;34(3):477-483. [doi: 10.1377/hlthaff.2014.0729] [Medline:
25732499]
Hersh W, Totten A, Gorman P, Eden K, Devine B, Woods S, et al. Health Information Exchange PROSPERO
2014:Crd42014013285. 2014. URL: http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014013285[accessed 2015-10-07] [WebCite Cache ID 6c6meV9wz]
Hersh W, Totten A, Eden K, Devine B, Gorman P, Kassakian S, et al. Health Information Exchange. Rockville, MD:
Agency for Healthcare Research and Quality; 2015.
Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Reviews:
AHRQ Publication Number 10(14)-EHC062-EF. Rockville, MD: Agency for Healthcare Research and Quality; 2014.
Mäenpää T, Asikainen P, Gissler M, Siponen K, Maass M, Saranto K, et al. Outcomes assessment of the regional healthinformation exchange: A five-year follow-up study. Methods Inf Med 2011;50(4):308-318. [doi: 10.3414/ME10-01-0048][Medline: 21336419]
Ross SE, Radcliff TA, Leblanc WG, Dickinson LM, Libby AM, Nease DE. Effects of health information exchange adoption
on ambulatory testing rates. J Am Med Inform Assoc 2013;20(6):1137-1142 [FREE Full text] [doi:10.1136/amiajnl-2012-001608] [Medline: 23698257]
Frisse ME, Johnson KB, Nian H, Davison CL, Gadd CS, Unertl KM, et al. The financial impact of health informationexchange on emergency department care. J Am Med Inform Assoc 2012;19(3):328-333 [FREE Full text] [doi:10.1136/amiajnl-2011-000394] [Medline: 22058169]
Tzeel A, Lawnicki V, Pemble KR. The business case for payer support of a community-based health information exchange:
A humana pilot evaluating its effectiveness in cost control for plan members seeking emergency department care. Am
Health Drug Benefits 2011 Jul;4(4):207-216 [FREE Full text] [Medline: 25126351]
Bailey JE, Wan JY, Mabry LM, Landy SH, Pope RA, Waters TM, et al. Does health information exchange reduce unnecessary
neuroimaging and improve quality of headache care in the emergency department? J Gen Intern Med 2013 Feb;28(2):176-183[FREE Full text] [doi: 10.1007/s11606-012-2092-7] [Medline: 22648609]
Bailey JE, Pope RA, Elliott EC, Wan JY, Waters TM, Frisse ME. Health information exchange reduces repeated diagnostic
imaging for back pain. Ann Emerg Med 2013 Jul;62(1):16-24. [doi: 10.1016/j.annemergmed.2013.01.006] [Medline:
23465552]
Ben-Assuli O, Shabtai I, Leshno M. The impact of EHR and HIE on reducing avoidable admissions: Controlling maindifferential diagnoses. BMC Med Inform Decis Mak 2013;13:49 [FREE Full text] [doi: 10.1186/1472-6947-13-49] [Medline:
23594488]

http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

Hersh et al

JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 9
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
30.
31.
32.

33.

34.

35.
36.

37.
38.
39.

40.

41.
42.
43.
44.
45.
46.

47.
48.
49.

50.

51.

52.
53.

Tzeel A, Lawnicki V, Pemble KR. "Hidden" value: How indirect benefits of health information exchange further promotesustainability. Am Health Drug Benefits 2012 Sep;5(6):333-341 [FREE Full text] [Medline: 24991331]
Vest JR. Health information exchange and healthcare utilization. J Med Syst 2009 Jun;33(3):223-231. [Medline: 19408456]
Vest JR, Kern LM, Campion TR, Silver MD, Kaushal R. Association between use of a health information exchange systemand hospital admissions. Appl Clin Inform 2014;5(1):219-231 [FREE Full text] [doi: 10.4338/ACI-2013-10-RA-0083][Medline: 24734135]
Vest JR, Kern LM, Silver MD, Kaushal R. The potential for community-based health information exchange systems to
reduce hospital readmissions. J Am Med Inform Assoc 2015 Mar;22(2):435-442. [doi: 10.1136/amiajnl-2014-002760][Medline: 25100447]
Magnus M, Herwehe J, Gruber D, Wilbright W, Shepard E, Abrams A, et al. Improved HIV-related outcomes associatedwith implementation of a novel public health information exchange. Int J Med Inform 2012 Oct;81(10):e30-e38. [doi:10.1016/j.ijmedinf.2012.06.005] [Medline: 22883431]
Dixon BE, McGowan JJ, Grannis SJ. Electronic laboratory data quality and the value of a health information exchange tosupport public health reporting processes. AMIA Annu Symp Proc 2011;2011:322-330 [FREE Full text] [Medline: 22195084]
Overhage JM, Grannis S, McDonald CJ. A comparison of the completeness and timeliness of automated electronic laboratory
reporting and spontaneous reporting of notifiable conditions. Am J Public Health 2008 Feb;98(2):344-350. [doi:10.2105/AJPH.2006.092700] [Medline: 18172157]
Kern LM, Barrón Y, Dhopeshwarkar RV, Kaushal R. Health information exchange and ambulatory quality of care. Appl
Clin Inform 2012;3(2):197-209 [FREE Full text] [doi: 10.4338/ACI-2012-02-RA-0005] [Medline: 23646072]
Nagykaldi ZJ, Yeaman B, Jones M, Mold JW, Scheid DC. HIE-i-health information exchange with intelligence. J Ambul
Care Manage 2014;37(1):20-31. [doi: 10.1097/JAC.0000000000000002] [Medline: 24309392]
Feldman SS, Horan TA. Collaboration in electronic medical evidence development: A case study of the Social Security
Administration's MEGAHIT System. Int J Med Inform 2011 Aug;80(8):e127-e140. [doi: 10.1016/j.ijmedinf.2011.01.012][Medline: 21333588]
Shapiro JS, Johnson SA, Angiollilo J, Fleischman W, Onyile A, Kuperman G. Health information exchange improves
identification of frequent emergency department users. Health Aff (Millwood) 2013 Dec;32(12):2193-2198. [doi:10.1377/hlthaff.2013.0167] [Medline: 24301405]
Vest JR, Miller TR. The association between health information exchange and measures of patient satisfaction. Appl Clin
Inform 2011;2(4):447-459 [FREE Full text] [doi: 10.4338/ACI-2011-06-RA-0040] [Medline: 23616887]
Altman R, Shapiro JS, Moore T, Kuperman GJ. Notifications of hospital events to outpatient clinicians using healthinformation exchange: A post-implementation survey. Inform Prim Care 2012;20(4):249-255. [Medline: 23890336]
Campion TR, Ancker JS, Edwards AM, Patel VN, Kaushal R. Push and pull: Physician usage of and satisfaction with healthinformation exchange. AMIA Annu Symp Proc 2012;2012:77-84 [FREE Full text] [Medline: 23304275]
Chang KC, Overhage JM, Hui SL, Were MC. Enhancing laboratory report contents to improve outpatient management of
test results. J Am Med Inform Assoc 2010;17(1):99-103 [FREE Full text] [doi: 10.1197/jamia.M3391] [Medline: 20064809]
Kaushal R, Dhopeshwarkar R, Gottlieb L, Jordan H. User experiences with pharmacy benefit manager data at the point ofcare. J Eval Clin Pract 2010 Dec;16(6):1076-1080. [doi: 10.1111/j.1365-2753.2009.01255.x] [Medline: 20666888]
Maass MC, Asikainen P, Mäenpää T, Wanne O, Suominen T. Usefulness of a regional health care information system inprimary care: A case study. Comput Methods Programs Biomed 2008 Aug;91(2):175-181. [doi: 10.1016/j.cmpb.2008.04.005][Medline: 18514363]
Machan C, Ammenwerth E, Schabetsberger T. Evaluation of the electronic transmission of medical findings from hospitals
to practitioners by triangulation. Methods Inf Med 2006;45(2):225-233. [doi: 10.1267/METH06020225] [Medline: 16538293]
Park H, Lee S, Kim Y, Heo E, Lee J, Park JH, et al. Patients' perceptions of a health information exchange: A pilot programin South Korea. Int J Med Inform 2013 Feb;82(2):98-107. [doi: 10.1016/j.ijmedinf.2012.05.001] [Medline: 22658777]
Lang E, Afilalo M, Vandal AC, Boivin J, Xue X, Colacone A, et al. Impact of an electronic link between the emergency
department and family physicians: A randomized controlled trial. CMAJ 2006 Jan 31;174(3):313-318 [FREE Full text][doi: 10.1503/cmaj.050698] [Medline: 16399880]
Willis JM, Edwards R, Anstrom KJ, Johnson FS, Del Fiol G, Kawamoto K, et al. Decision support for evidence-based
pharmacotherapy detects adherence problems but does not impact medication use. Stud Health Technol Inform
2013;183:116-125. [Medline: 23388267]
Afilalo M, Lang E, Léger R, Xue X, Colacone A, Soucy N, et al. Impact of a standardized communication system oncontinuity of care between family physicians and the emergency department. CJEM 2007 Mar;9(2):79-86. [Medline:
17391577]
Lammers EJ, Adler-Milstein J, Kocher KE. Does health information exchange reduce redundant imaging? Evidence fromemergency departments. Med Care 2014 Mar;52(3):227-234. [doi: 10.1097/MLR.0000000000000067] [Medline: 24374414]
Jones SS, Friedberg MW, Schneider EC. Health information exchange, health information technology use, and hospital
readmission rates. AMIA Annu Symp Proc 2011;2011:644-653 [FREE Full text] [Medline: 22195120]

http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

Hersh et al

JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 10(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
54.

55.

56.
57.

58.

59.
60.

61.
62.
63.

64.

65.

66.

67.

Hersh et al

Carr CM, Gilman CS, Krywko DM, Moore HE, Walker BJ, Saef SH. Observational study and estimate of cost savings
from use of a health information exchange in an academic emergency department. J Emerg Med 2014 Feb;46(2):250-256.
[doi: 10.1016/j.jemermed.2013.05.068] [Medline: 24071033]
Winden TJ, Boland LL, Frey NG, Satterlee PA, Hokanson JS. Care everywhere, a point-to-point HIE tool: Utilization andimpact on patient care in the ED. Appl Clin Inform 2014;5(2):388-401 [FREE Full text] [doi: 10.4338/ACI-2013-12-RA-0100][Medline: 25024756]
Ben-Assuli O, Shabtai I, Leshno M. Using electronic health record systems to optimize admission decisions: The Creatininecase study. Health Informatics J 2015 Mar;21(1):73-88. [doi: 10.1177/1460458213503646] [Medline: 24692078]
eHealth Initiative. Result from Survey on Health Data Exchange 2013. The Challenge to Connect. 2013. URL: https://ehi-rails-app.s3.amazonaws.com/uploads/article/file/3/eHIResultsFromSurveyonHealthDataExchange2013.pdf [accessed
2015-06-17] [WebCite Cache ID 6c6n4pAts]
Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Van Spall Harriette G C, et al. Features ofeffective computerised clinical decision support systems: Meta-regression of 162 randomised trials. BMJ 2013;346:f657[FREE Full text] [Medline: 23412440]
Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting complex behaviour change interventions: The
need for a scientific method. Implement Sci 2009;4:40 [FREE Full text] [doi: 10.1186/1748-5908-4-40] [Medline: 19607700]
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing andevaluating complex interventions: The new Medical Research Council guidance. BMJ 2008;337:a1655 [FREE Full text][Medline: 18824488]
Rasmussen J. Risk management in a dynamic society: A modelling problem. Saf Sci 1997 Nov;27(2-3):183-213. [doi:10.1016/S0925-7535(97)00052-0]
Vicente KJ. What does it take? A case study of radical change toward patient safety. Jt Comm J Qual Saf 2003
Nov;29(11):598-609. [Medline: 14619352]
Leveson N. Engineering a safer world. Systems thinking applied to safety. Cambridge: MIT Press; 2012. URL: https://mitpress.mit.edu/sites/default/files/titles/free_download/9780262016629_Engineering_a_Safer_World.pdf[WebCite Cache
ID 6c6ovJCqs]
Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy(v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior changeinterventions. Ann Behav Med 2013 Aug;46(1):81-95. [doi: 10.1007/s12160-013-9486-6] [Medline: 23512568]
Vest JR. More than just a question of technology: Factors related to hospitals' adoption and implementation of healthinformation exchange. Int J Med Inform 2010 Dec;79(12):797-806. [doi: 10.1016/j.ijmedinf.2010.09.003] [Medline:
20889370]
Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomized
trials. Ann Intern Med 2010 Jun 1;152(11):726-732. [doi: 10.7326/0003-4819-152-11-201006010-00232] [Medline:
20335313]
Fleurence RL, Curtis LH, Califf RM, Platt R, Selby JV, Brown JS. Launching PCORnet, a national patient-centered clinicalresearch network. J Am Med Inform Assoc 2014;21(4):578-582 [FREE Full text] [doi: 10.1136/amiajnl-2014-002747][Medline: 24821743]

Abbreviations
AHRQ: Agency for Healthcare Research and Quality
ED: emergency department
EHR: electronic health record
HIE: health information exchange
HIT: health information technology
HITECH: Health Information Technology for Economic and Clinical Health
IT: information technology
ONC: Office of the National Coordinator for Health Information Technology
RCT: randomized controlled trial

http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 11(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Hersh et al

Edited by G Eysenbach; submitted 08.10.15; peer-reviewed by R Elmore, M Solomon; comments to author 06.11.15; revised version
received 10.11.15; accepted 11.11.15; published 15.12.15
Please cite as:
Hersh WR, Totten AM, Eden KB, Devine B, Gorman P, Kassakian SZ, Woods SS, Daeges M, Pappas M, McDonagh MS
Outcomes From Health Information Exchange: Systematic Review and Future Research Needs
JMIR Med Inform 2015;3(4):e39
URL: http://medinform.jmir.org/2015/4/e39/
doi: 10.2196/medinform.5215
PMID: 26678413©William R Hersh, Annette M Totten, Karen B Eden, Beth Devine, Paul Gorman, Steven Z Kassakian, Susan S Woods, Monica
Daeges, Miranda Pappas, Marian S McDonagh. Originally published in JMIR Medical Informatics (http://medinform.jmir.org),
15.12.2015. This is an open-access article distributed under the terms of the Creative Commons Attribution License(http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium,provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information,a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.

http://medinform.jmir.org/2015/4/e39/

XSL• FO
RenderX

JMIR Med Inform 2015 | vol. 3 | iss. 4 | e39 | p. 12(page number not for citation purposes)