JMIR MEDICAL INFORMATICS

Reis et al

Original Paper

Is There Evidence of Cost Benefits of Electronic Medical Records,
Standards, or Interoperability in Hospital Information Systems?
Overview of Systematic Reviews
Zilma Silveira Nogueira Reis1*, MD, PhD; Thais Abreu Maia2*, MBA, MSc; Milena Soriano Marcolino3*, MD, PhD;
Francisco Becerra-Posada4*, MD, DrPH; David Novillo-Ortiz4*, MLIS, PhD; Antonio Luiz Pinho Ribeiro5*, MD, PhD
1Informatics Center in Health, Obstetrics and Gynecology Department, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

2State Health Secretariat of Minas Gerais, Planning Advisory, Belo Horizonte, Minas Gerais, Brazil

3

Faculty of Medicine, Internal Medicine Department, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

4

Pan American Health Organization, Washington, DC, DC, United States

5

Medical School, Telehealth Center, Hospital das Clínicas, Internal Medicine Department, Universidade Federal de Minas Gerais, Belo Horizonte,
Minas Gerais, Brazil*

all authors contributed equally

Corresponding Author:
Zilma Silveira Nogueira Reis, MD, PhD
Informatics Center in Health
Obstetrics and Gynecology Department
Universidade Federal de Minas Gerais
Av. Prof. Alfredo Balena, 190
Belo Horizonte, Minas Gerais, 30130100
BrazilPhone: 55 3134099648
Email: zilma.medicina@gmail.com

Abstract
Background: Electronic health (eHealth) interventions may improve the quality of care by providing timely, accessible
information about one patient or an entire population. Electronic patient care information forms the nucleus of computerized
health information systems. However, interoperability among systems depends on the adoption of information standards.
Additionally, investing in technology systems requires cost-effectiveness studies to ensure the sustainability of processes forstakeholders.
Objective: The objective of this study was to assess cost-effectiveness of the use of electronically available inpatient datasystems, health information exchange, or standards to support interoperability among systems.
Methods: An overview of systematic reviews was conducted, assessing the MEDLINE, Cochrane Library, LILACS, and IEEE
Library databases to identify relevant studies published through February 2016. The search was supplemented by citations fromthe selected papers. The primary outcome sought the cost-effectiveness, and the secondary outcome was the impact on qualityof care. Independent reviewers selected studies, and disagreement was resolved by consensus. The quality of the included studies
was evaluated using a measurement tool to assess systematic reviews (AMSTAR).
Results: The primary search identified 286 papers, and two papers were manually included. A total of 211 were systematic
reviews. From the 20 studies that were selected after screening the title and abstract, 14 were deemed ineligible, and six met theinclusion criteria. The interventions did not show a measurable effect on cost-effectiveness. Despite the limited number of studies,
the heterogeneity of electronic systems reported, and the types of intervention in hospital routines, it was possible to identify
some preliminary benefits in quality of care. Hospital information systems, along with information sharing, had the potential to
improve clinical practice by reducing staff errors or incidents, improving automated harm detection, monitoring infections more
effectively, and enhancing the continuity of care during physician handoffs.
Conclusions: This review identified some benefits in the quality of care but did not provide evidence that the implementationof eHealth interventions had a measurable impact on cost-effectiveness in hospital settings. However, further evidence is needed
to infer the impact of standards adoption or interoperability in cost benefits of health care; this in turn requires further research.

http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 1
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Reis et al(JMIR Med Inform 2017;5(3):e26) doi: 10.2196/medinform.7400
KEYWORDS
electronic medical records; standards; medical information exchange; health information exchange; cost; benefits and costs

Introduction

Methods

Information technology (IT) applied to health care, or electronic
health (eHealth) [1], ostensibly offers numerous benefits to thequality of health information, particularly in its recording,retrieval, and use. Patients can benefit directly from safe andaccessible electronic clinical information for better decision
making [2]. However, demographics and patient data are highly
fragmented and distributed across multiple unintegrated systems[3]. Comprehensive and consistent health care, leading toeffective use of services, requires the computerization of healthdata for more efficient communication. To achieve this,standardized information channels are needed to make syntacticinteroperability possible among electronic records systems.
Semantic interoperability is necessary to guarantee theconsistency of information, as health information models require
adopting standards to support communication [2]. Even if thestandardization of electronic health records (EHRs) in eHealthsystems is accomplished, health data sharing will continue tobe a global challenge. Few publications exist concerning theimpact of medical records and interoperability among healthsystems in cost and benefits of patient care.

Basic Concepts and International Standards

Improvements in health and economic indicators are relevant
metrics to justify IT investments. Indeed, planning and investingin IT is necessary for the efficient use of information that not
only advances health care but also holds financial, social,
cultural, and ethical benefits. Comparative cost-effectivenessstudies guide agencies and institutions in choosing the best
option for desired clinical outcomes and costs, which is the key
to ensuring the sustainability of government health systems andtheir welfare programs [3,4].

It is also important to emphasize that interoperability is usuallydivided into (1) syntactic interoperability: the capability of two
or more systems to communicate and exchange data throughspecified data formats and communication protocols, and (2)semantic interoperability: the ability for data shared by systems
to be understood at the level of fully defined domain concepts[8].

This review analyzes systematic reviews addressing the costbenefit and effectiveness of electronic medical records (EMR),
standards adoption, or interoperability to discuss the benefits,
drawbacks, and lessons learned from the implementation ofactions related to eHealth and serves as a reference forgovernment representatives and stakeholders. The assessmentof the involvement of government and private health institutionsin the implementation and maintenance of eHealth interventions
that were tested and valuated worldwide is also of interest. The
study was directed by 2 questions: What evidence exists
regarding the impact of computerizing applications, standards,
health information exchange, or interoperability to support thequality of care or patient outcomes in hospital settings? Whatcritical cost-benefit evidence is published to provide a clear
understanding of the value of eHealth implementations?

On eHealth
The terms used here to describe eHealth technologies areavailable in Multimedia Appendix 1. Despite different meanings,
some papers use the terms electronic medical record (EMR)
and electronic health record (EHR) synonymously. A relevant
dissimilarity exists between health information exchange forsystems integration and interoperability. The former refers toorganizational framework for the dissemination of electronic
health care information or clinical data across health-related
institutions and systems to enhance patient care [5]. The latter
relates to the the ability or capability of two or more systems
to exchange information and use the exchanged information,
which may support a longitudinal record widely available acrossinstitutions and over life spans [6]. Additionally, in a more
specific context, “interoperability means the ability of healthinformation systems to work together within and acrossorganizational boundaries in order to advance the effective
delivery of health care for individuals and communities” [7].

Worldwide coordinated efforts resulted in the development ofstandards to define an EHR as one or more repositories ofactionable information by computers. The European Committee
for Standardization (CEN), health level seven (HL7),
International Organization for Standardization (ISO), andopenEHR Foundation are nonprofit organizations dedicated toproviding frameworks and standards. Terminologies, EHR
specifications, and information models are proposed by theseinternational standards organizations that support the exchange,
integration, interoperability, and retrieval of electronic healthinformation [6].
To better represent the meaning of standards in the primary
selected systematic reviews, we adopted the generic definition
for the term as: “A document adopted by consensus by arecognized entity, that provides rules, guidelines and/or features
for common use, in order to obtain an optimal level of
performance in a given context…” [9].

On Economic Analysis
Economic analysis supports health care policy andorganizational decision making. However, it encounters somedifficulties with eHealth systems, which are as follows:
constantly changing technology, inconsistent study design tomanage inadequate sample sizes, the inappropriateness of
http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 2
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
conventional techniques of economic evaluation, and the
problem of placing value on health and nonhealth outcomes[10]. Consequently, five methods have been used to calculate
the cost-effectiveness of traditional and eHealth interventions:cost-minimization
analysis,cost-benefit
analysis,cost-effectiveness analysis, cost-utility analysis, andcost-consequence analysis [11].

Data Sources and Search Strategy
This review of systematic reviews has been conducted inaccordance with the preferred reporting items for systematic
reviews and meta-analyses (PRISMA) statement [12] and therecommended methodological considerations when using
existing systematic review as described by Whitlock et al [13].
On February 22, 2016, electronic searches were conducted on
the MEDLINE, Cochrane Library, LILACS, and IEEE Library
databases. To identify the EHR concept, standards forinteroperability, and health information and its cost benefits,
the search strategy was:((“Electronic health records”[MeSH Terms] OR “Health
Information Exchange”[MeSH Terms] OR (“Health Information
Management”[MeSH Terms] OR (“Medical Informatics”[MeSH Terms]) AND (“Interoperability” OR “Standard ofInformation”)) AND (“Cost-Benefit Analysis”[MeSh] OR“Evaluation Studies”[Publication Type] OR “Program
Evaluation”[MeSh] OR impact or effectiveness)
The search was limited by language of publication (English,
Spanish, French, Italian, and Portuguese), studies in humans,
type of study (systematic reviews and meta-analyses), and yearof publication (since 2005). Two systematic reviews thatsatisfied the criteria were identified manually. To better definecertain eHealth technology descriptions, additional sources ofevidence were considered.

Study Selection
The inclusion criteria were as follows:
1.
2.
3.

Primary impact: EMR, standards, or interoperability on
cost-benefit, orSecondary impact: EMR, standards, or interoperability on
quality of care (clinical outcomes), andReal-life reviews about interventions in in-hospital settings.

Studies in primary or secondary care scenarios, studies without
the primary or secondary impact of eHealth actions, andduplications were excluded. Titles and abstracts of retrieved
papers were independently screened and evaluated by 2
investigators (ZSNR and TAM). Abstracts providing insufficient
information were retrieved for independent, full-text evaluation
by 2 investigators to determine study eligibility. Disagreements
were resolved by consensus. Additional publications wereidentified using the reference lists of selected manuscripts.

http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

Reis et al

Data Extraction and Quality AssessmentZSNR prepared electronic data with paper contents abstractedusing StArt software (Systematic Review System) to organize
the analysis [14]. The data extraction of full-text analysis
included the following: study design, number of studies
evaluated, objectives, type of interventions/clinical data sources,
eHealth interventions and terminology, interface/healthinformationexchanges,
durationoffollow-up,
cost-effectiveness, impact on quality of care, main results control
group, potential bias, limitations, and lessons learned. The
results were summarized into two subgroups according to themodality of intervention:••

Subgroup 1: eHealth systems implementation without healthinformation exchange
Subgroup 2: eHealth systems with health informationexchange functionalities

The methodological quality assessment was based on the
AMSTAR (a measurement tool to assess systematic reviews)
checklist [15].

Results
A total of 288 papers were identified during the initial researchphase, which decreased to 273 after removing 15 duplicates.
After applying our criteria, only six systematic reviews were
included in the final analysis and data-abstraction phase. The
review process is represented in Figure 1, according PRISMAStatement [14].
The primary cause for excluding the 20 studies was mixed or
outpatient settings for eHealth interventions (11 papers of 14
excluded, 79%). The Pan American Health Organization(PAHO) conducted a review of the implementation and effective
use of standards to achieve interoperability in Latin American
and Caribbean countries but without direct or indirect outcomesanalysis [6]. Multimedia Appendix 2 presents a detailed
summary of the 14 full-text excluded systematic reviews.

Characteristics and Quality of the Selected Studies
Evidence of the cost-effectiveness of eHealth interventions that
met the criteria was identified. Only one systematic review ofthe six performed a meta-analysis [16]. The quality assessmentof the included studies followed AMSTAR (a measurement tool
to assess systematic reviews) methodology and resulted in wide
variability of the quality score. Two studies were classified witha moderate rating of quality with 5 positive points among 11
items [16,17], whereas other reports neglected many AMSTAR
criteria [18-21]. Table 1 summarizes the quality assessment
ratings, the study design, and the funding or support of the six
included systematic reviews.

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 3
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Reis et al

Figure 1. Flow of information through the different phases of the systematic review.

http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 4
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Reis et al

Table 1. Quality assessment ratings and characteristics of the six included systematic reviews.
Study

a

AMSTARa score

Funding or Study design

Number of

supportstudies evaluated (most frequent)Control group

Metaanalysis

Yb

Nc

CAd

N/Ae

Thompson et al 2015 [16]

4

7

0

0

Y

RCTf, pre-post studies, descriptive studies

45 total/Metaanalysis: 26

Pre-post implementation (paper vs system)Y

Cheung et al 2015 [17]

5

4

0

2

NCg

RCT, quasi-experimental
studies, descriptive studies

18

Pre-post implementationNde Bruin et al 2014 [20]

2

7

0

2

NC

Quasi-experimental

26

True infectionN
detection by infection control
experts

Mapp et al 2013 [21]

1

7

1

2

NC

Observational, Pilot studies

9

No control

N

Li et al 2013 [18]

4

5

0

2

NC

RCT, quasi-experimental
studies

6

Patient not reported in written notes or before system

N

Govindan et al 2010 [19]

5

4

0

2

Y

Observational: accuracy of
the automated method witha gold standard method

43Standard chart
reviewN

AMSTAR: a measurement tool to assess systematic reviews.

b

Y: yes.

c

N: no.

d

CA: cannot answer.

e

N/A: not applicable.

f

RCT: randomized controlled trial.

g

NC: not commissioned.

Table 2 summarizes the objective, the type ofintervention/clinical data sources, eHealth intervention andterminology, interface/health information exchange, and durationof follow-up of the six included systematic reviews.

Summarized Outcomes
Among the included systematic reviews, only one was classified
as showing an effect on eHealth implementation without
electronic health information exchange (Subgroup 1), and theother five were ranked as showing effects of systems
implementation with incorporated health information exchange
among other electronic data sources (Subgroup 2).

Subgroup 1
Considering eHealth systems implementation without healthinformation exchange, the review of Thompson et al [16]reported a parallel to advances in digital technology and how
different forms of eHealth systems have been developed andimplemented (Table 3).
Types and Functions of Technology Systems
The selected review stated a mix of electronic interventions:
EHR, EMR, computerized decision support systems (CDSS),
computerized provider order-entry (CPOE) and surveillancesystems used by physicians, nurses, allied health professionals,
and managers of health services evaluating evidence frompre-and postsystems implementation. The analysis synthesized
46 publications about systems for diagnosis, treatment, andhttp://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

clinical monitoring. The study included a meta-analysis
extracted from 26 publications to evaluate the effects of different
types of systems regarding health IT in the inpatient of intensive
care unit (ICU) setting on mortality, length of stay (LOS), andcost.
Effects on Quality or Efficiency of CareNot enough evidence showed that electronic interventions canimprove quality and safety of health care. The goals forsecondary outcomes were the effects of health IT in the inpatientand ICU on mortality or LOS. The quality of included studies
and interventions varied significantly, which was highlighted
as the major limitation. Despite this, the surveillance systems
had a pooled odd ratio (OR) of 0.85 (95% CI 0.76-0.94) withmoderate heterogeneity, I2 of 59%.
Effects on Costs
Costs were unable to be evaluated quantitatively because theprimary studies presented mixed and inconclusive results,
leaving us unable to draw a definitive conclusion aboutcost-effectiveness. The analysis of costs was more limited thanthe evidence on quality and efficiency.

Subgroup 2
EHR implementation with health information exchange is a
recent worldwide trend in hospital settings. A summary of theresults of the systematic reviews included in subgroup 2 ispresented in Table 4.

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 5
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Reis et al

Table 2. Descriptive summary of the systematic reviews included in electronic medical records (EMRs)/Interoperability review.
Study

Objective

Type of intervention/eHealth intervention

Interface/health

Clinical data sources

and terminology

information exchange

Duration of follow-up

Multiple health IT inter- EHRd, EMRe,
health IT in the inpa- ventions on diagnosis, CDSSf,CPOEg,
treatment, monitoring,
tient and ICUb on
Surveillance systemcost reduction/No referc
mortality, LOS ,, and encecost

No reference

Cheung et al 2015 [17]To evaluate the effects Integrating bedside
CDSS, PDMS, healthof an information sys- equipment to an infor- information exchange
tem integrated tomation system/vital
PDMSh on organiza- signs, patient monitor,
ventilator, anesthesia
tional and clinicalmachine, dialysis maoutcomes, inchine, IV pump, labiICU /Operating roomvalues, hospital information system, admission,discharge and transfer

PDMS to an informa- 1 day to 1 week;11
tion system/no men- months to 4 years
tion about direction ofdata exchange

de Bruin et al 2014 [20]To evaluate recent
trends in use of electronically available
patient data by electronic surveillance

EHR to HAI sysNo referencetems/no mention
about direction of dataexchange

Thompson et al 2015 [16]To evaluate effects of

No referenceaMapp et al 2013 [21]Li et al 2013 [18]HAIs that utilize EHR
available in hospitals tosurveillance the
HAIs/Medico-administrative data procedures
systems for HAIsj and or discharge reports,
identify consequences free text reports, biofor system effective- chemistry, microbiology, and radiology labonessratory test results, pharmacy dispensing
records, radiology freetext records, vital signs,
electronic discharge
summary

Automated detection
by HAI systems:
EHR, health information exchange, using

To examine early
warning scoring systems and their effectiveness in predicting
a patient's potential
for deterioration andconsiders whether
these scoring systems
prevent unplanned
ICU admissions
and/or death

Instruments and clinicalsupport systems available to assist health carepersonnel in recognizing early clinical deterioration/Vital signs,

EMR, CDSS, health Early warning scoring
information exchange systems that interface
with EMRs and are
based on SBARp
supplemented with
communication
decision aides (algorithms) and clinicalsupport systems/no
mention about direction of data exchange

To evaluate the im-

Decision support/train- CHTs, EMR, CDSS,
ing, emergency referhealth information exrals, supervision, alerts change. Allergy Codeand reminders, client
education, data collection, medicine dosing/Patient demographics, medications, diagnosis, problem lists, comment line, vital signs,
to-do list, LOS, freedaily notes, lab values

pact of the CHTsq on
the quality of physician handoff, patientcare, and physician
work efficiency

Govindan et al 2010 [19]

To identify, describe, Automated harm detecand evaluate the effec- tion on EMR. Goldtiveness of automated standard: chart review
inpatient harm-detection methods

http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

SpO2m, LOCn, UOPo,
nurse/family concerns,
complaints, lab values

ICDk-9, ICD-10, discharge coding, ATCl
code

Automated detection
by surveillance systems: EMR, health information exchange,
using ICD-9, procedure codes, billingcodes

Seven studies: 3 to15
months/two studies:over 24 months to 8
years

Clinical information 1 to 6 months
exchange using CHTs
for physician handoff
for hospitalized patients CHTs/mixed(no interface, unidirectional or bidirectional
interface exchange)

Automated harm de- No reference
tection on EMR, using
field-defined systems,
natural language-processing/Unidirectional
retrospective

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 6
(page number not for citation purposes)

JMIR MEDICAL INFORMATICSaReis et al

IT: information technology.

b

ICU: intensive care unit.

c

LOS: length of stay.

d

EHR: electronic health record.

e

EMR: electronic medical record.

f

CDSS: computerized decision support systems.

g

CPOE: computerized provider order-entry.

hPDMS: Patient data management system.

i

ICU: intensive care unit.

j

HAIs: health care–associated infections systems.

k

ICD: international classification of disease.

iATC: anatomical therapeutic chemical.

m

SpO2: oxygen saturation.

n

LOC: level of consciousness.

o

UOP: urine output.

p

SBAR: situation, background, assessment, recommendation.

q

CHTs: computerized physician handoff tools.

Table 3. Descriptive summary of the results of systematic reviews included in electronic medical records(EMRs)/Interoperability review. Subgroup
1: electronic health (eHealth) systems implementation without health information exchange.
Study

Primary impact:

Secondary impact:

Cost-effectivenessQuality of care/

Main results

Potential bias

Lessons

Selection, measurement

There is not enoughevidence to confidently state that electronicinterventions have theability to achieve thegoal of improving
quality and safety.

Clinical outcome
Thompson et al 2015 [16]Mixed and inconclusive

Mortality: overall

Electronic intervenCPOEa systems did not tions were not shownto have a substantial
show a significant efeffect on mortality,bfect (OR : 0.91, 95%
LOSf, or cost.
CI 0.75-1.10; I2c 66%),
nor EHRd alone (OR:0.96, 95% CI 0.771.19). CDSSe(OR 0.96,
95% CI 0.77-1.19). The
surveillance systems
had a pooled OR of
0.85 (95% CI 0.760.94) with moderateheterogeneity, I259%
LOS: CPOE trended toward a reduction in
LOS (mean decrease,
0.67 days, 95% CI–2.07 to 0.73), thoughwith significant heterogeneity (I282%). Neither CDSS nor surveillance systems trended
toward changes in hospital LOS, and the netpooled effect was not
significant.

a

CPOE: computerized provider order-entry.

b

OR: odds ratio.

c 2

I : measure of heterogeneity.

d

EHR: electronic health record.

e

CDSS: computerized decision support systems.

f

LOS: length of stay.

http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 7
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Reis et al

Table 4. Descriptive summary of the results of systematic reviews included in the electronic health record (EHR)/Interoperability review. Subgroup2: electronic health (eHealth) systems implementation with information exchange.
Study

Primary impact:

Secondary impact:

Cost-effectivenessQuality of care/

Main results

Potential bias

Lessons

Clinical outcome
Cheung et al 2015 [17]Not evaluateda

PDMSbreduced charting time, increased timespent on direct patientcare and reduced theoccurrence of errors(medication errors, intravenous and ventilation
incidents). The effecton documentation was
mixed. Improvement inclinical outcomes when
PDMS was integrated

The effect on docuSelection, measurementation was mixed. ment
Qualitative analysis
showed a significant
decrease in time spenton documentation.
Clinical outcomes: inconclusive.

Improvement in clinical outcomes when
PDMS was integrated
with a CDSS, butthere is scarce literature available. Organizational advantages
included improved accuracy, legibility, data
accessibility, and decision support. Such integration may improve clinical outcomes, although further studies are required for validation.

Electronic surveillanceachieves equal or better
sensitivity than manual
surveillance. Several
studies also reported
time savings of 60% to99.9% or a reduction inchart reviews of 40% to
90.5%.

Driven by the inSelection
creased availability ofelectronic patient data,
electronic

HAIs detection systems use increasingly

An increase occurred in
the number of rapid response calls by nursing
staff, a decrease in un-

Improvement in clini- Selection
cal outcomes whenusing early warning
scoring systems.

Early warning scoring
systems can be more
effective with the integration of algorithms
and clinical support
systems.

Completeness andSelection, measureconsistency of thement
handoff document has
improved. Accuracyof information about
patients during physician handoff.

CHTsgcould potentially enhance work efficiency and continuityof care during physician handoff, but therole in improving
quality is less clear.
The information
available was often
not sufficient to help
on-call physicians
make patient care decisions.

with a CDSSc, but
scarce literature isavailable.
de Bruin et al 2014 [20]Mapp et al 2013 [21]Not evaluateda

Not evaluateda

HAIsdsurveillancesystems use more data, making systems
more sensitive yet lessspecific but also allow
systems to be tailoredto the needs of healthcare institutes’
surveillance programs.

planned ICUfadmissions, and a decrease in
hospital mortality.
Li et al 2013 [18]Not evaluateda

http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

Impact on physician
work efficiency (selfreported time spent on
handing copying patient
information; 50%) andproportionally moretime to see patients.
Time on each patient
during rounding decreased by1.5 min. Impact on quality on
physician handoff:
completeness and consistency of the handoff
document has improved.

more EHReand patient
data as more data
sources become available. Thus, systems
tend to become moresensitive and less specific.

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 8
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
Study

Reis et alPrimary impact:

Secondary impact:

Cost-effectivenessQuality of care/

Main results

Potential bias

Lessons

Clinical outcome
Govindan et al 2010 [19]Not evaluateda

Sensitivities of different
methods ranged from
0.10 to 0.94, specificity
from 0.10 to 0.94,

Automated harm de- Selection, measuretection has the poten- menttial to positively influence clinical practice.

PPVhfrom 0.03 to 0.84,
and NPVifrom 0.70 to
0.96. The field-defined
methods of automatedharm detection will
prove superior to natural language processing,particularly if information about harm is accurately documented.
a

Automated harm detection has the potential to positively influence clinical practice.
Another potential
benefit is the reduction of person-hour
required to harm
surveillance.

Not evaluated in the selected study.

b

PDMS: Patient data management system.

c

CDSS: computerized decision support systems.

dHAIs: health care–associated infections systems.

e

EHR: electronic health record.

f

ICU: intensive care unit.

g

CHTs: computerized physician handoff toolsh

PPV: positive predictive value.

i

NPV: negative predictive value.

Types and Functions of Technology Systems
Most of the reviews use ICUs as settings for eHealthintervention analysis. However, the objectives of interventions
were quite heterogeneous. Two studies reported the effect ofsurveillance systems on harm detection [19] and healthcare–associated infections [20]. Bedside data integration in an
information system [17], continuity of care using physician
handoff tools [18], and prediction of death or unexpected ICU
admission [21] were the proposals of the other reviews.
Regarding application users, two studies focused on patient
outcome results for health care managers [19,20]. Some focused
directly on health care professionals to improve clinical practice[18,19,21]. On the direction of electronic health informationexchange, one review described it as unidirectional [19], three
did not clarify whether the exchange was bidirectional[17,20,21], and one summarized mixed studies including
systems without interfaces [18]. None mentioned interoperability
among electronic health systems. Regarding standards for theexchange of clinical data, four studies reported the use ofterminologies such as International Classification of Disease(ICD) and anatomical therapeutic chemical (ATC) code [18-21].
Effects on Quality or Efficiency of CareAmong reviews focused on improving clinical practice,inconclusive results in direct patient care were reported by
Cheung et al [17]. Mapp et al [21] highlighted an increase innursing staff efficiency regarding rapid calls response, a decreasein unplanned ICU admissions, and hospital mortality. Li et al[18] presented a positive impact on continuity of inpatient care.
With regard to indirect results on patient care, two studies
highlighted the improvement of health data quality in terms of
http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

accuracy, legibility, completeness, and consistency of documents[17,18]. The other reviews focused on electronic surveillance.
The results showed that systems tend to become more sensitiveand less specific than manual monitoring to detect infection[20]. With respect to inpatient harm detection, the automatedsystems allowed rapid scanning of a vast number of patient
records with minimal effort and may identify events as theyoccur in real time [19]. Most automated surveillance systems
were retrospective, but some real-time surveillance alerts that
informed physicians and pharmacists of adverse events werereported [19].
Effects on Costs
None of the reviews evaluated effects of eHealth interventions
on costs.

Discussion
Principal Findings
This study found preliminary benefits in the use of electronicallyavailable inpatient data systems on the quality of care. Despite
the limited number of studies that met the eligibility criteria,
the heterogeneity of electronic systems reported, and different
interventions on hospital routines, the identification of
preliminary secondary benefits on patient mortality was possible[16]. eHealth systems with information exchange functionalities
also showed potential impact on quality of care or patient
outcomes. From five studies, one had inconclusive results on
direct patient care [17] and four presented partial effects, as
nursing staff efficiency led to a faster call response, a decreasein unplanned ICU admissions and hospital mortality [18],improvement of health data quality [17,18], and more efficient
JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 9
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
surveillance programs inside hospitals [19,20]. It is expectedthat the systems able to share health information would improve
care at the time and point of attention, especially the surveillancesystems and those that use common terminologies andvocabularies to support consistency in information collection[6,19,20,22].
However, no substantial review regarding the impact ofelectronic interventions on cost-effectiveness was identified.
Among the six analyses included, only Thompson et al reported
that some preliminary studies have identified decreases in cost,
but the heterogeneity and the absence of information offollow-up impaired a proper analysis of cost-effectiveness [16].
Immediate cost savings are not anticipated for organizations
when choosing to adopt eHealth strategies because the high costof implementation limits the transition from paper-based toelectronic systems and represents a significant challenge to theirwidespread adoption [23]. Regardless, medium and long-term
positive results are expected, and the World Health Organization(WHO) recognized overall eHealth as cost- effective and secure[24]. Potential indirect cost saving was mentioned as a secondary
outcome in three studies, with the reduction of person-hours
harming surveillance and the increase in time spent on direct
patient care [17,18,24].
Unfortunately, no study about interoperability, in the sense ofsyntactic and semantic meaning, on cost benefit was identified.
Importantly, none of the studies in this review properly defined
EHR concept as a longitudinal health record with entries byhealth care practitioners in multiple sites of care or mentioned
interoperability applications among electronic systems.
However, taking the antecedent step toward full interoperability,
an effective information sharing between stakeholders andsystems can be attained through the use of standards [6].
Standards adoption for the exchange of clinical data wasmentioned in four studies [18-21], mostly terminologies
adoptions, but the potential impact of such tools on continuityof care or costs remains an open question that needs
investigation. Although within the limits of hospital systems,
the analysis confirmed the potential to positively impact
physician practice organizations, as previously reported [23].

Reis et al
Further longitudinal research is needed to determine the actual
impact of eHealth adoption on health care costs and clinicaloutcomes.

Limitations
The current results should be interpreted as a whole with thestudy limitations. Only four major databases were searched andgray literature sources were not evaluated. Additionally, thelimitation to English, Spanish, French, Italian, and Portuguese
languages prevented the capture of all relevant studies.
Furthermore, the quality of included studies was poor, and theyvaried regarding the type of eHealth interventions, follow-up
time, and goals. This systematic review summarized primaryand secondary outcomes from different classes of intervention
from which to draw results, analysis, and conclusions. Due tothe variation in scenarios and lack of numeric goals, a
meta-analysis was considered inappropriate.

Conclusions and Lessons Learned
This review identified some benefits on the quality of care but
did not provide evidence that the eHealth interventions had ameasurable impact on cost-effectiveness, mortality, or LOS in
hospital settings. Preliminary evidence indicates that the use ofeHealth interventions with information exchange may improve
clinical process outcomes. The absence of studies precludes theassessment of impact of interoperability on benefits of healthcare or cost, and this aspect needs further research.
Technological barriers might influence eHealth solutions
implementation and data exchange for systems integration orinteroperable interfaces. There are also issues with the lack ofstandardization of most aspects of health information and misuseof terms in the scientific publications. Authors should be explicit
when they are using interfacing syntactic interoperability or
semantic interoperability to reduce the confusion with different
health information exchange possibilities. Further research withlong-term follow-up is needed to determine the actual impactof eHealth adoption on health care costs to demonstrate (1)value for money (including clinical impacts) and (2) the clinicalimpact of semantic and synthetic interoperability.

Acknowledgments
We thank the team who worked with us on a systematic review with a large scope: Maria Beatriz Moreira Alkmim, Renato Minelli
Figueira, João Antonio de Queiroz Oliveira, Gustavo Moreira Alkmim, Maria Clara Noman de Alencar, Nayara Dornela Quintino,
and the undergratuate students—Edson Alexandre Silva Carvalho, Fernanda Cotrim Stefanelli, Gabriel Almeida Silqueira Rocha,
Grace Kelly Matos e Silva, Marcelo Henriques de Camargos, Thales Matheus Mendonça Santos, Thiago Adriano de Deus Queiroz
Santos, and Elsy Dumit Bechara.
Francisco Becerra-Posada and David Novillo-Ortiz are staff members of the PAHO/WHO and are themselves alone responsible
for the views expressed in the paper, which does not necessarily represent the views, decisions, or policies of the PAHO/WHO.

Conflicts of Interest
None declared.

Multimedia Appendix 1
eHealth concepts and definitions.
[PDF File (Adobe PDF File), 49KB-Multimedia Appendix 1]
http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 10(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Reis et alMultimedia Appendix 2
Descriptive summary of the 14 full-text excluded systematic reviews.
[PDF File (Adobe PDF File), 41KB-Multimedia Appendix 2]

References
1.

2.
3.
4.

5.
6.

7.
8.
9.
10.
11.
12.

13.
14.

15.

16.

17.

18.
19.

20.

21.

World Health Organization. Apps.WHO. 2006. Building foundations for eHealth : progress of member states : report of
the Global Observatory for eHealth URL: http://apps.who.int/iris/handle/10665/43599 [accessed 2017-07-13] [WebCite
Cache ID 6rw1isUOl]
World Health Organization. Apps.WHO. 2006. Electronic health records: manual for developing countries URL: http://apps.who.int/iris/handle/10665/207504 [accessed 2017-07-13] [WebCite Cache ID 6sm2rrOne]
Shekelle PG, Morton SC, Keeler EB. Costs and benefits of health information technology. Evid Rep Technol Assess (FullRep) 2006 Apr(132):1-71. [Medline: 17627328]
World Health Organization. WHO. Geneve: WHO Press; 2016. Atlas of eHealth country profiles: the use of eHealth insupport of universal health coverage: based on the findings of the third global survey on eHealth URL: http://www.who.int/
goe/publications/atlas_2015/en/ [accessed 2017-07-14] [WebCite Cache ID 6rF0c5lHC]
ncbi.nlm. 2017. Health Information Exchange URL: https://www.ncbi.nlm.nih.gov/mesh/?term=health+information+exchange[accessed 2017-06-20] [WebCite Cache ID 6rMNCkzRG]
Pan American Health Organization. Webcitation.: Pan American Health Organization; 2016. eHealth in Latin Americathe
Caribbean: interoperability standards review URL: http://www.webcitation.org/6rwrfeqBJ [accessed 2017-07-14] [WebCite
Cache ID 6rwrfeqBJ]
Webcitation. 2005. Interoperability Definition and Background URL: http://www.webcitation.org/6rKvIpuvO [accessed2017-07-14] [WebCite Cache ID 6rKvIpuvO]
International Organization for Standarization - ISO. ISO. Health informatics -- Requirements for an electronic health record
architecture URL: https://www.iso.org/standard/52823.html [accessed 2017-06-19] [WebCite Cache ID 6rL0KEbKn]
International Organization for Standarization - ISO. Webcitation. Standardization and related activities -- General vocabulary
URL: http://www.webcitation.org/6rKvaBnAJ [accessed 2017-07-14] [WebCite Cache ID 6rKvaBnAJ]
Mistry H. Systematic review of studies of the cost-effectiveness of telemedicine and telecare. Changes in the economic
evidence over twenty years. J Telemed Telecare 2012 Jan;18(1):1-6. [doi: 10.1258/jtt.2011.110505] [Medline: 22101609]
Rojas SV, Gagnon M. A systematic review of the key indicators for assessing telehomecare cost-effectiveness. Telemed J
E Health 2008 Nov;14(9):896-904 [FREE Full text] [doi: 10.1089/tmj.2008.0009] [Medline: 19035798]
Liberati A, Altman D, Tetzlaff J, Mulrow C, Gøtzsche P, Ioannidis J. The PRISMA statement for reporting systematic
reviews and meta-analyses of studies that evaluate healthcare interventionsxplanation and elaboration. Br Med J
2009;339:b2700 [FREE Full text] [doi: 10.1371/journal.pmed.1000100]
Whitlock EP, Lin JS, Chou R, Shekelle P, Robinson KA. Using existing systematic reviews in complex systematic reviews.
Ann Intern Med 2008 May 20;148(10):776-782. [Medline: 18490690]
Hernandes E, Zamboni A, Di Thommazo A, Fabbri S. Avaliação da ferramenta StArt utilizando o modelo TAM e oparadigma GQM. In: Proceedings of 7th Experimental Software Engineering Latin American Workshop (ESELAW 2010).
Goiânia: UFG/INF/FUNAPE; 2010 Presented at: 7th Experimental Software Engineering Latin American Workshop(ESELAW 2010); 12/10/2010; Goiânia URL: http://www.portal.inf.ufg.br/~eselaw2010/proceedings-eselaw2010.pdf#page=37
Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement
tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007 Feb 15;7:10 [FREE Fulltext] [doi: 10.1186/1471-2288-7-10] [Medline: 17302989]
Thompson G, O'Horo JC, Pickering BW, Herasevich V. Impact of the electronic medical record on mortality, length ofstay, and cost in the hospital and ICU: a systematic review and meta-analysis. Crit Care Med 2015 Jun;43(6):1276-1282.
[doi: 10.1097/CCM.0000000000000948] [Medline: 25756413]
Cheung A, van Velden FH, Lagerburg V, Minderman N. The organizational and clinical impact of integrating bedsideequipment to an information system: a systematic literature review of patient data management systems (PDMS). Int J Med
Inform 2015 Mar;84(3):155-165. [doi: 10.1016/j.ijmedinf.2014.12.002] [Medline: 25601332]
Li P, Ali S, Tang C, Ghali WA, Stelfox HT. Review of computerized physician handoff tools for improving the quality ofpatient care. J Hosp Med 2013 Aug;8(8):456-463. [doi: 10.1002/jhm.1988] [Medline: 23169534]
Govindan M, Van Citters AD, Nelson EC, Kelly-Cummings J, Suresh G. Automated detection of harm in healthcare withinformation technology: a systematic review. Qual Saf Health Care 2010 Oct;19(5):e11. [doi: 10.1136/qshc.2009.033027][Medline: 20671081]
de Bruin JS, Seeling W, Schuh C. Data use and effectiveness in electronic surveillance of healthcare associated infectionsin the 21st century: a systematic review. J Am Med Inform Assoc 2014;21(5):942-951 [FREE Full text] [doi:10.1136/amiajnl-2013-002089] [Medline: 24421290]
Mapp ID, Davis LL, Krowchuk H. Prevention of unplanned intensive care unit admissions and hospital mortality by early
warning systems. Dimens Crit Care Nurs 2013;32(6):300-309. [doi: 10.1097/DCC.0000000000000004] [Medline: 24100432]

http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 11(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
22.
23.
24.
25.

26.
27.
28.

29.

30.

31.
32.
33.
34.

35.

36.

37.
38.

39.

40.

Reis et al

Hammond WE, Bailey C, Boucher P, Spohr M, Whitaker P. Connecting information to improve health. Health Aff (Millwood)2010 Feb;29(2):284-288 [FREE Full text] [doi: 10.1377/hlthaff.2009.0903] [Medline: 20348075]
Police RL, Foster T, Wong KS. Adoption and use of health information technology in physician practice organisations:systematic review. Inform Prim Care 2010;18(4):245-258 [FREE Full text] [Medline: 22040851]
Asamblea Mundial de la Salud. Apps.WHO. 2005. Cibersalud URL: http://apps.who.int/iris/handle/10665/23104 [accessed2017-07-14] [WebCite Cache ID 6smbcRPBE]
Kawamoto K, Houlihan C, Balas E, Lobach D. Improving clinical practice using clinical decision support systems: asystematic review of trials to identify features critical to success. Br Med J 2005;330(7494):765. [doi:10.1136/bmj.38398.500764.8F]
Mitchell J. From Telehealth to E-health: The Unstoppable Rise of E-health. Canberra, NSW: Commonwealth Departmentof Communications, Information Technology and the Arts; 1999:064275036.
Eysenbach G. What is e-health? J Med Internet Res 2001 Jun;3(2):E20 [FREE Full text] [doi: 10.2196/jmir.3.2.e20][Medline: 11720962]
Pan American Health Organization. PAHO. Washington, DC: PAHO HQ Library Cataloguing-in-Publication; 2011.
Epidemiological Surveillance of Healthcare Associated Infections URL: http://new.paho.org/hq/dmdocuments/2011/
ENG_Modulo_I_final.pdf [accessed 2017-07-14] [WebCite Cache ID 6rwqNuepz]
Main C, Moxham T, Wyatt J, Kay J, Anderson R, Stein K. Computerised decision support systems in order communication
for diagnostic, screening or monitoring test ordering: systematic reviews of the effects and cost-effectiveness of systems.
Health Technol Assess 2017 Jul 14;14(48):1-227 [FREE Full text] [doi: 10.3310/hta14480] [Medline: 21034668]
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information
technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006 May 16;144(10):742-752. [Medline:
16702590]
Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: asystematic review. HIM J 2009;38(3):26-37. [Medline: 19875852]
Wollersheim D, Sari A, Rahayu W. Archetype-based electronic health records: a literature review and evaluation of theirapplicability to health data interoperability and access. HIM J 2009;38(2):7-17. [Medline: 19546483]
Chan AJ, Chan J, Cafazzo JA, Rossos PG, Tripp T, Shojania K, et al. Order sets in health care: a systematic review of theireffects. Int J Technol Assess Health Care 2012 Jul;28(3):235-240. [doi: 10.1017/S0266462312000281] [Medline: 22980699]
Sheehan J, Sherman KA. Computerised decision aids: a systematic review of their effectiveness in facilitating high-quality
decision-making in various health-related contexts. Patient Educ Couns 2012 Jul;88(1):69-86. [doi: 10.1016/j.pec.2011.11.006][Medline: 22185961]
Ojeleye O, Avery A, Gupta V, Boyd M. The evidence for the effectiveness of safety alerts in electronic patient medication
record systems at the point of pharmacy order entry: a systematic review. BMC Med Inform Decis Mak 2013 Jul 01;13:69[FREE Full text] [doi: 10.1186/1472-6947-13-69] [Medline: 23816138]
Walsh C, Siegler EL, Cheston E, O'Donnell H, Collins S, Stein D, Informatics Intervention Research Collaboration (I2RC).
Provider-to-provider electronic communication in the era of meaningful use: a review of the evidence. J Hosp Med 2013
Oct;8(10):589-597 [FREE Full text] [doi: 10.1002/jhm.2082] [Medline: 24101544]
Tenório J, Cohrs F, Pisa I, de Fátima Marin H. Personal health record: a review on current models and experiments. J Health
Inform 2013:91-97 [FREE Full text]
Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, et al. Effectiveness of computerized decision support
systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health 2014
Dec;104(12):e12-e22. [doi: 10.2105/AJPH.2014.302164] [Medline: 25322302]
Nguyen L, Bellucci E, Nguyen LT. Electronic health records implementation: an evaluation of information system impact
and contingency factors. Int J Med Inform 2014 Nov;83(11):779-796. [doi: 10.1016/j.ijmedinf.2014.06.011] [Medline:
25085286]
Fiander M, McGowan J, Grad R, Pluye P, Hannes K, Labrecque M, et al. Interventions to increase the use of electronic
health information by healthcare practitioners to improve clinical practice and patient outcomes. Cochrane Database Syst
Rev 2015 Mar 14(3):CD004749. [doi: 10.1002/14651858.CD004749.pub3] [Medline: 25770311]

Abbreviations
AMSTAR: a measurement tool to assess systematic reviews
ATC: anatomical therapeutic chemical
CDSS: computerized decision support systems
CEN: European Committee for Standardization
CHTs: computerized physician handoff tools
CPOE: computerized provider order-entry
CPR: computer-based patient record
EHR: electronic health record
http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 12(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Reis et al

EMR: electronic medical record
HAIs: health care–associated infections surveillance systems
HL7: Health level seven
I2: measure of heterogeneity
ICD: International Classification of Disease
ICU: Intensive Care Unit
ISO: International Organization for Standardization
IT: information technology
LOC: level of consciousness
LOS: length of stay
NPV: negative predictive value
OR: odds ratio
PAHO: Pan American Health Organization
PDMS: patient data management system
PPV: positive predictive value
PRISMA: preferred reporting items for systematic reviews and meta-analyses
SpO2: oxygen saturation
SBAR: situation, background, assessment, recommendation
UOP: urine output
WHO: World Health Organization

Edited by G Eysenbach; submitted 24.01.17; peer-reviewed by TR Soron, S Collins; comments to author 05.06.17; revised version
received 01.07.17; accepted 12.07.17; published 29.08.17
Please cite as:
Reis ZSN, Maia TA, Marcolino MS, Becerra-Posada F, Novillo-Ortiz D, Ribeiro ALPIs There Evidence of Cost Benefits of Electronic Medical Records, Standards, or Interoperability in Hospital Information Systems?
Overview of Systematic Reviews
JMIR Med Inform 2017;5(3):e26
URL: http://medinform.jmir.org/2017/3/e26/
doi: 10.2196/medinform.7400
PMID: 28851681©Zilma Silveira Nogueira Reis, Thais Abreu Maia, Milena Soriano Marcolino, Francisco Becerra-Posada, David Novillo-Ortiz,
Antonio Luiz Pinho Ribeiro. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 29.08.2017. This is
an open-access article distributed under the terms of the Creative Commons Attribution License(https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information,a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.

http://medinform.jmir.org/2017/3/e26/

XSL• FO
RenderX

JMIR Med Inform 2017 | vol. 5 | iss. 3 | e26 | p. 13(page number not for citation purposes)