JMIR MEDICAL INFORMATICS

Feldman et al

Original Paper

Health Information Exchange Implementation: Lessons Learned
and Critical Success Factors From a Case Study
Sue S Feldman1,2,3*, RN, MEd, PhD; Benjamin L Schooley2,4*, MBA, PhD; Grishma P Bhavsar2*, MPH
1Central Virginia Health Network, Richmond, VA, United States

2

Arnold School of Public Health, University of South Carolina, Columbia, SC, United States

3

School of Information Systems and Technology, Claremont Graduate University, Claremont, CA, United States

4

Department of Integrated Information Technology, University of South Carolina, Columbia, SC, United States*

all authors contributed equally

Corresponding Author:
Sue S Feldman, RN, MEd, PhD
Central Virginia Health Network
4900 Cox Road #200
Richmond, VA, 23060
United States
Phone: 1 6616188805
Fax: 1 6616188805
Email: suefeldman1009@gmail.com

Abstract
Background: Much attention has been given to the proposition that the exchange of health information as an act, and healthinformation exchange (HIE), as an entity, are critical components of a framework for health care change, yet little has been studied
to understand the value proposition of implementing HIE with a statewide HIE. Such an organization facilitates the exchange ofhealth information across disparate systems, thus following patients as they move across different care settings and encounters,
whether or not they share an organizational affiliation. A sociotechnical systems approach and an interorganizational systems
framework were used to examine implementation of a health system electronic medical record (EMR) system onto a statewide
HIE, under a cooperative agreement with the Office of the National Coordinator for Health Information Technology, and its
collaborating organizations.
Objective: The objective of the study was to focus on the implementation of a health system onto a statewide HIE; provide
insight into the technical, organizational, and governance aspects of a large private health system and the Virginia statewide HIE(organizations with the shared goal of exchanging health information); and to understand the organizational motivations andvalue propositions apparent during HIE implementation.
Methods: We used a formative evaluation methodology to investigate the first implementation of a health system onto thestatewide HIE. Qualitative methods (direct observation, 36 hours), informal information gathering, semistructured interviews(N=12), and document analysis were used to gather data between August 12, 2012 and June 24, 2013. Derived from sociotechnicalconcepts, a Blended Value Collaboration Enactment Framework guided the data gathering and analysis to understand organizational
stakeholders’ perspectives across technical, organizational, and governance dimensions.
Results: Several challenges, successes, and lessons learned during the implementation of a health system to the statewide HIE
were found. The most significant perceived success was accomplishing the implementation, although many interviewees also
underscored the value of a project champion with decision-making power. In terms of lessons learned, social reasons were foundto be very significant motivators for early implementation, frequently outweighing economic motivations. It was clear that
understanding the guides early in the project would have mitigated some of the challenges that emerged, and early communication
with the electronic health record vendor so that they have a solid understanding of the undertaking was critical. An HIE
implementations evaluation framework was found to be useful for assessing challenges, motivations, value propositions forparticipating, and success factors to consider for future implementations.
Conclusions: This case study illuminates five critical success factors for implementation of a health system onto a statewide
HIE. This study also reveals that organizations have varied motivations and value proposition perceptions for engaging in theexchange of health information, few of which, at the early stages, are economically driven.

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 1
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Feldman et al(JMIR Med Inform 2014;2(2):e19) doi: 10.2196/medinform.3455
KEYWORDS
health information exchange; blended value collaboration enactment framework; HIE value proposition; interorganizational
systems; HIE implementation

Introduction
Investing in Health Information Technology
The Health Information Technology for Economic and Clinical
Health, HITECH, Act, enacted as part of the American Recovery
and Reinvestment Act of 2009, set a national goal of investingin health information technology to improve health care delivery.
To meet this goal, the electronic exchange of health information
between providers is essential to ensure coordinated, efficient,and quality care. This exchange can be accomplished throughvarious local, regional, or statewide organizations that build theinfrastructure to facilitate secure health information exchange(HIE); the federal government has already entered into
cooperative agreements with 56 states and territories to fundinfrastructure to enable these efforts [1]. HIE is a complex andemergent system of structures and actions, which varies in scopeand scale. The current study discusses HIE as both the act of
exchanging health information between two collaboratingorganizations, and the entity that facilitates such exchange. The
goal of this study, which focuses on the implementation of HIE
as an interorganizational health care system, is to understandthe organizational motivations and value propositions apparent
during HIE implementation. These value propositions areanalyzed through an interorganizational system (IOS)
information technology (IT) governance lens that considers thetechnical, organizational, and governance dimensions of HIE
value. We apply the framework to: (1) evaluate HIE
implementation challenges, successes, and lessons learned; and(2) extract value propositions across organizational stakeholders.

Health Information Exchange
Much attention has been given to the proposition that theexchange of health information is a critical component of aframework for health care change, the Triple Aim being: (1)better patient experiences through quality and satisfaction; (2)better health outcomes of populations; and (3) reduction of percapita cost of health care [2]. These changes will rely onorganizational entities that have entered cooperative agreementswith the federal government to provide technical infrastructure,organizational structure, and governance mechanisms forcompleting the act of HIE [1]. The act of HIE, described various
ways in the literature, can be conducted across affiliatedphysicians’ offices, hospitals, and clinics; or can occur between
completely disparate systems [3,4]. HIE across disparate systems
allows clinical information to follow patients as they moveacross different care settings, whether or not they share anorganizational affiliation. For example, this might include a
hospital or health system connected to an HIE network that is,in turn, connected to several for-profit and not-for-profit
competing hospitals or health systems networks. On a broad
scale, this type of HIE holds great promise for achieving the
Triple Aim goals.
http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Health Information Exchange Benefits and Challenges
The implementation and use of HIE technology have influenced
patient care by allowing providers direct access to healthinformation, reducing time to obtain health information, andincreasing providers’ awareness of patient interactions with thehealth care system [5]. The benefits and challenges of HIE havebeen studied in prior research. Regarding patient experiences,previous studies have found improved coordination of care andenhanced patient health outcomes for human immunodeficiency
virus patients [6], higher patient satisfaction [7], informed
patient care [8], efficient care [8], and positive patient perceptionof impact on care coordination [9]. However, others have foundthat the benefits of HIE in relation to patient outcomes arelimited [10].
From a broader provider and patient perspective, timely sharingof a patient’s clinical information can improve the accuracy of
diagnoses, reduce the number of duplicative tests, prevent
hospital readmissions, and prevent medication errors [3,11,12].
From a public health perspective, the exchange of healthinformation has fostered positive relationships with public healthagencies [13], improved public health surveillance [14], andincreased the efficiency and quality of public health reporting[15].
Though the theoretical case for HIE on reducing the utilizationand cost of health care services is compelling and has received
a great deal of emphasis [16], empirical evidence is stillinconclusive [17,18]. This may reflect the nascent nature of
HIE, especially between disparate systems, and the fact thatthese systems and the context are complex and emergent. For
example, HIE has faced challenges like those of other new IT
initiatives, disparate and noninteroperable information
technologies [19,20]; a range of technical, work flow, andorganizational challenges to exchanging information [17,21];
and a variety of governance challenges [22,23]. Yet the HIEs
that have continued to operate have done so with evolving andmaturing technical, organizational, and governance structures[24-27].
Still, much more research is needed to understand HIE, how it
operates, what factors contribute to success, and even how
success should be defined. At this early phase of HIEdevelopment and implementation, it is important to study thesystem and its context to improve upon existing methods, tools,
and frameworks. This study investigates the value of HIE froman IT implementation perspective. Specifically, it asks what
motivations, challenges, and successes lead to value realization
across organizations working together to on-board to a state
HIE? The sociotechnical systems (STS) approach of this study
applies an IOS framework developed through previous work tounderstand blended value across participating organizations.

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 2
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Sociotechnical Systems Approach
An STS approach examines social/community links to thetechnical [28]. STS design includes several levels of abstraction
including mechanical (hardware), informational (software),psychological (person), and social (community). Such an
inclusive approach is aimed at understanding interdependent
linkages between increasingly complex social and technologicalcomponents. Working together, these components consider
social motivations and accomplish a set of social goals that
otherwise would not be realized.
The highest-order social benefit (human life) of healthinformation sharing are stated quite succinctly by Porter andTeisburg [29], “The social benefits of results information will
be even greater in health care than in the financial markets,
because the physical well being of Americans is at stake.” Social
value factors include the range of intangible and actor-based ororganizational considerations that contribute to collaboration
success. Furthermore, two studies almost 20 years apart suggestthat successful IT project advancement is frequently associatedwith social elements [30,31]. In his book on infrastructuredelivery in public-private collaborations, Mody [32] draws fromthe railway and transportation systems examples to suggest thatsocial considerations, such as being able to deliver goods andinformation to the right place at the right time, might exceed
those of economic returns and could exert greater significant
pressure.
Therefore, examination of the social motivations and benefits
deserve to be considered in a different light than a customary
return on investment model commonly considered in informationexchange in the business world. This blended value proposition
has been defined as the combination of social and economic
value used to maximize total returns where “the core nature ofinvestment and return is not a tradeoff between social andfinancial interest, but rather the pursuit of an embedded value
proposition composed of both” [33]. Emerson [33] continues,
Societies cannot function strictly on the basis of theireconomic enterprise. It is social commerce that allows
individuals and institutions to pursue the traditional
financial returns sought by mainstream financial
capital market players. [J Emerson]
An STS approach, which focuses on systems that are bothtechnologically sound and socially sustainable [34], has been
applied to the study of HIE because of the multiple
organizations, user types, hardware and software technologies,
and sociopolitical motivations and goals involved in its
composition. Though relatively few studies have examined an
HIE network in operation, a sociotechnical approach waspreviously applied and shown to be appropriate to the study ofHIE [5].

Interorganizational Systems Implementations
An IOS is an IT-based system shared by two or more
independent organizations [35]. Prior research on IOSs focusedon the cross-organizational features of an STS [35,36]. Whilethe implementation of IOS has been studied for decades acrossa wide array of industries, few studies have addressed healthcare, and fewer still have addressed HIE. IOS studies show the
http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Feldman et alimportance of: (1) learning from early adopters [37], and (2)evaluating the process of implementation to understand lessons
learned and the real and perceived value of an IOS [38,39].

From Interorganizational System to Health
Information Exchange Implementations
Evaluations of information system collaboration require looking
beyond a single focus and attending to multiple dimensions[40]. This perspective acknowledges that the collaboration ofmultiple stakeholders may hold the potential to create something
new and better, as well as to create public value [41]. Similarly,
a multidimensional perspective is required in evaluating theexchange of health information [42].
Evaluations of HIE and the benefits and challenges of
exchanging health information have been studied in various
contexts. For example, a 2011 study suggested that US $2
million in uncompensated care cost recovery is achievable withuse of the nationwide HIE (now eHealth Exchange) as applied
to disability determination [43]; and a more recent study
estimated the resource utilization impacts resulting from using
eHealth Exchange for emergency department visits [44]. Yet,few studies exist regarding the value of HIE at a statewide level.
While these economic findings are important and may drive asustainable IOS, understanding social value or motivation isimportant to HIE implementation.
As states end their cooperative agreements with the federal
government, it is helpful to understand the challenges, successes,
and lessons learned from an early on-boarder or implementation.
While literature exists about information systems
implementations across various industries, little is known about
health systems implementations between public-private entities(eg, a private hospital and state HIE).
The aim of this case study is to provide insight into the technical,
organizational, and governance aspects of a large private healthsystem (Inova Health System) and the Virginia statewide HIE(ConnectVirginia EXCHANGE), organizations with the sharedgoal of exchanging health information. In this case study, theBlended Value Collaboration Enactment Framework, amultidimensional value framework [45], discussed later in thispaper, provided a conceptual framework for evaluating theimplementation process by which an organization becomesconnected to a system to facilitate the exchange of information(ie, on-boarding), the first, to our knowledge, on-boarding toConnectVirginia EXCHANGE.

Methods
Overview
The study design comprised direct observation, informal
information gathering, document analysis, and semistructured
interviews to study HIE implementation across technical,
organizational, and governance dimensions. The study assessed
the first, to our knowledge, on-boarding of a health care system
onto the Virginia statewide HIE, ConnectVirginia EXCHANGE,
using a formative evaluation of the implementation phase of
the systems development life cycle. The study did not address

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 3
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
the exchange of information, but rather the process of HIE
implementation.

Study Setting and Background
ConnectVirginia EXCHANGE
In March 2010, the Office of the National Coordinator for Health
IT (ONC) awarded state cooperative agreements to the states
and territories in the United States to develop infrastructure
supporting the electronic exchange of health information. At
that time, the Virginia Department of Health (VDH), thestate-designated entity for Virginia, was awarded US $11.6
million. In September 2011, Community Health Alliance (CHA)
was awarded a contract from VDH to build the Virginia
Statewide HIE, ConnectVirginia. The organization toaccomplish this statewide was subsequently initiated. Statewide
HIEs were required to enable information exchange using
standardized technologies, tools, and methods. Between
September 2011 and February 2014, ConnectVirginia designed,
tested, developed, and implemented three technical exchange
services: (1) ConnectVirginia DIRECT Messaging (a secure
messaging system), (2) ConnectVirginia EXCHANGE (thefocus of this study), and (3) a Public Health Reporting Pathway(a pathway with VDH for public health reporting).
ConnectVirginia EXCHANGE is a query/retrieve service inwhich a deliberate query passively returns one or more
standardized continuity of care documents (CCDs; these provide
a means of sharing standardized health data between
organizations) from any other “system” on-boarded andconnected to ConnectVirginia. This study examines and reportson the first implementation to ConnectVirginia EXCHANGE
by Inova Health System (Inova).

Feldman et al

Inova Health System
Inova [46] primarily serves the Northern Virginia andWashington, DC, markets and includes five hospitals with morethan 1700 licensed beds and 16,000 employees. This
comprehensive network of inpatient hospitals also includes
outpatient services and facilities, primary and specialty care
physician practices, and health and wellness initiatives. The
inpatient facilities use a well known electronic health record(EHR), and the affiliated outpatient practices have access tothat EHR. In keeping with Inova’s vision to increase value forpatients and build an integrated network within and outside of
their own hospitals, Inova became the first node to on-board to
the ConnectVirginia EXCHANGE.

Electronic Health Record System
The EHR software involved in this study is primarily for
mid-size and large medical groups, hospitals, and integrated
health care organizations spanning clinical, access, and revenue
functions. It provides an intranetwork data-sharing pathway(this specific EHR to this specific EHR), as well as an external
data-sharing pathway (this specific EHR to a different EHR orsystem). Use of the external data-sharing pathway is the subjectof this implementation study.

Research Process
ConnectVirginia initiated and managed the on-boarding process.
The on-boarding process for Inova began with a kick-off
meeting on August 2, 2012, and concluded with a test to
exchange electronic documents with ConnectVirginia on April
26, 2013 (184 total workdays). Along with Inova andConnectVirginia, implementation involved two criticalsubcontractors: (1) MEDfx (the software vendor forConnectVirginia), and (2) MedVirginia (the CCD content
consultant). Figure 1 shows the evaluation timeline.

Figure 1. Evaluation timeline.

Formative Evaluation of Information Systems
Implementations
To assess the HIE implementation process from the perspectiveof an STS, this study used a formative evaluation methodology.
Formative evaluations are widely used in young and developing
initiatives to enable continuous improvement throughout thedevelopment and implementation stages [47,48]. From apractical perspective, this approach allows organizations to learn
from past mistakes and develop better methods for assessing
success [42,48]. This methodology allowed researchers toinvestigate the first implementation of ConnectVirginia
EXCHANGE for a new and emergent type of system (ie, HIE)
that is rapidly expanding across thousands of US health caresystems.
http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

To study IT implementations, Cooper and Zmud [49] proposeda diffusion process model of IT implementation that includes
factors influencing implementation. Their model captures boththe process and its context, subcategorized into stages. For
example, their diffusion process model of IT implementation
proposes six stages: (1) initiation, (2) adoption, (3) adaptation,(4) acceptance, (5) routinization, and (6) infusion. The present
HIE evaluation covers only the adoption and adaptation stagesof Cooper and Zmud’s model, which include: (1) gaining
organizational backing for implementing IT applications, and(2) developing and installing IT applications, and developing
and revising organizational policies and procedures for ongoinguse of the IT applications. The classic system development life
cycle recognizes four distinct implementation phases that canbe used in IT evaluations: (1) preimplementation, (2) during
JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 4
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
implementation, (3) postimplementation, and (4) routine
operation [42]. Evaluations such as this study conducted on the“during implementation” stage, use qualitative methodology[47,48]. It has been suggested that evaluations during this stage
may be more important than those providing proof of outcomes[50], as the former can provide guidelines and lessons learnedfor others. The methods applied herein aim to extract valuable
measures, and disseminate lessons learned for other HIE
implementation efforts.

Information Technology Implementation Measures
Evaluations of the exchange of health information can bechallenging [51], partly due to the lack of any single model for
HIE [50]. Implementation measures are generally chosen for
their value to stakeholders [52]. Evaluations should determine
not only how well a system works, but also how well it works
for particular users in a particular setting [42].
Several measures have been applied to evaluations of IT
implementations. Categories span different levels of abstraction
including: (1) technical, (2) organizational, and (3) governance.
In prior research, implementation measures pertaining to bothtechnical and organizational dimensions included: (1) degree
and type of data usage [50,53-55], (2) level of complexity ofbusiness processes [56], (3) completeness of information[47,50,54], (4) resistance to change [56], (5) unintended
consequences [50,53], and (6) facilitators [47] and barriers[47,50] to implementation. Organizational and governancedimensions in implementations include: (1) communication[47], (2) trust [47], (3) organizational structure [53], (4)sustainability [12,54,57,58], (5) roles and power relations
between participants [56], (6) levels of leadership commitment[47], and (7) representativeness and motivations of stakeholders[47,50,57].
The technical, organizational, and governance aspects of HIE,
as well as their interactions with each other; provide a basis forthe evaluation measurements currently utilized [42]. These
measures were applied to the study of ConnectVirginia
EXCHANGE within the context of a previously tested analyticalframework for HIE [43].

Analytical Framework
Enactment theory describes how people act within organizations[59]. When people carry out an act, they take into account theirpast experiences, events, and structures; determine a course ofaction; and then set that course into action. It is a form of social
construction. Fountain’s technology enactment framework builds
on enactment theory and considers that technical factors andorganizational structures are embedded within each collaboratingorganization, and that the relationship between multiple factors
is critical [60]. Others have suggested that while technicalperformance is a crucial element in any resulting informationexchange between organizations, successful interorganizational

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Feldman et aldata exchanges frequently hinge on organizational andgovernance factors [56,61-63]. However, other research notesthat motivational factors and context can be the true
underpinnings of collaboration [64,65]. As such, collaborations
for information exchange require organizations to look beyond
a single focus and give attention to multiple dimensions ofcollaboration [40].
Based on the aforementioned work of Fountain, Schooley, andEmerson, and because of its prior use and demonstrated utility
in assessing multi-organizational HIE efforts, the Blended Value
Collaboration Enactment Framework was used to guide
implementation and evaluation [45] (Figure 2 shows thisframework). Framework development drew upon STS concepts
and frameworks. Its importance for this study is that theframework: (1) considers each organizational stakeholder andits respective social and economic motivations for participatingin HIE; (2) differentiates between technical, organizational, andgovernance dimensions; and (3) focuses on determining value
propositions across stakeholders. For this study, and within thecontext of HIE, technical is defined as elements associated withthe system or infrastructure; organizational is defined as
elements associated with any and all of the stakeholders; andgovernance is defined as elements associated with decision
making [45].
The above framework also considers the value proposition of
HIE across stakeholders, including the social and economic
motivations that lead to a more successful and sustainable HIE.
A value proposition can be defined as the implicit promise of
mutual value to the organization and its customers and/or
partners [66]. For example, an in-depth case study of the fashionindustry found that interorganizational value propositions could
have both “hard” elements (economic gain, technologicalmastery, etc) and “soft” elements (brand identity, trust
relationships, etc) [67]. Past research on HIE has illustrated thateach stakeholder organization has its own value-driven
motivations for participating in the exchange of healthinformation, social including clinical (eg, “Is this the right thing
to do for public health and wellness?”) versus economic (eg,“How does this impact our financial bottom line?”).
The intended output of the Blended Value Collaboration
Enactment Framework is the resulting system performance [45].
Since this study reports on the implementation of HIE and noton its actual use (from which system performance would bederived), the “output” section of Figure 2 has been modified in
Figure 3 to reflect critical success factors, as is more appropriate
for implementation studies [68]. The framework also proposesthat motivations and value propositions may change of overtime (T1 and T2 in Figure 2). This study investigates only the
implementation stage and does not evaluate how these
dimensions change over time. Therefore, only the unshaded
areas of the framework are germane to this analysis.

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 5
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Feldman et al

Figure 2. Blended Value Collaboration Enactment Framework [45]. T1 and T2= changes over time.

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 6
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Feldman et al

Figure 3. Blended Value Collaboration Enactment Framework. T1 and T2=changes over time.

Observation, Informal Information Gathering, andDocument Analysis
Data collection took place August 2012-June 2013 by one of
the authors, who was the external evaluator to ConnectVirginia
and not part of the implementation team. A total of 36 hours ofobservation of the on-boarding process occurred across planning,
coordination, implementation, and problem-solving meetings
held either in-person or via conference calls. Each organization
was represented in each meeting, and the meetings provided anenvironment for conducting informal information gathering.
Various documents, such as meeting notes and detailed meeting
minutes, were also analyzed.

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Semistructured Interviews
Qualitative methods were employed to understand how and why
the factors in each dimension contribute to or influence theoverall implementation and value derived. At the end of the
project (between May 8 and June 24, 2013), 12 60-minute,semistructured in-person interviews were conducted across thefive participating organizations (ConnectVirginia, MEDfx,
MedVirginia, EHR vendor, and Inova). Table 1 provides
additional details about the interviewees. Purposive sampling
was used to select interviewees based on: (1) their belonging
to one of the above-mentioned organizations, and (2) theirdegree of participation in the on-boarding process. For example,
individuals who participated in the majority of project manager(PM), technical, or system testing meetings were invited for
JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 7
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
interviews, and all invitees agreed to participate. Persons suchas consumers of the exchanged health information (ie, clinicians)
were not interviewed because, at the time of the study, there
was no routine exchange of information for real-world use.
Interviews were conducted by one of the authors with expertisein conducting interviews, and who was also the external
evaluator to ConnectVirginia and not part of the implementation
team.

Feldman et al
Interview questions were designed to develop a clearer pictureof the on-boarding process, to recreate the actual timeline, andto support information from calls, documents, and informal
information gathering. Table 2 provides a sampling of interview
questions. Not all questions were appropriate for all
interviewees, and therefore were not asked to all interviewees.
Likewise, if interviewees had in-depth knowledge of a particular
process, secondary questions were asked that may or may nothave been used for other interviewees.

Table 1. Interviewee’s by organization, position, and role.
Organization

PositionRole during implementation

ConnectVirginia

Executive Director

Oversight

PM

Daily operations management of the implementation

Chief Operations Officer

Oversight

PM

Daily operations management of the implementation

Chief Information Officer

Oversight

Systems Analyst

CCD content validation

EHR vendor

Application Support Specialist (x3)

Provided vendor support during the implementation

Inova

Executive Vice President and Chief
Technology Officer

Oversight and internal champion

MEDfx

MedVirginia

Senior Vice President and Chief Informa- Oversighttion Officer
PM

Daily operations management of the implementation

Table 2. Sample interview questions.
Sample interview question types
Technical
What were the initial technical processes involved in on-boarding to ConnectVirginia?
What technical advances and/or information could have streamlined the on-boarding process?
What technical challenges emerged and how were they addressed?
Were any technical “workarounds” employed? If so, please explain.
What technical processes were particularly successful and why?
To what extent was the technical assistance that you received helpful?
Please describe your current level of HIE (eg, within your organization, outside your organization, labs, etc).
Organizational
To what extent did organizational leadership impact the on-boarding process?
What is the value proposition of on-boarding to ConnectVirginia?
What organizational challenges emerged and how were they addressed?
What is needed to have HIE become a standard of care?
What was particularly successful regarding organizational leadership?
Governance
What were the key elements of the governance structure within your organization for on-boarding to ConnectVirginia?
What governance structures do you see as vital for sustainability or growth of HIE across ConnectVirginia?
To what extent were on-boarding guides governing implementation useful, helpful, or challenging?

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 8
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Data Analysis
Interviews were transcribed verbatim and imported into
ATLAS.ti, a qualitative data analysis software application [69].
The Blended Value Collaboration Enactment Framework wasused to guide the analysis. Each dimension from the framework(technical, organizational, and governance) was used to provide
a predefined coding structure frame. The framework also
provided predefined coding categories for critical success factors
and value proposition of the implementation (Figure 4 shows
this coding structure). Interview transcripts were coded and data
attributed to the appropriate category. Challenges, lessons
learned, and sustainability were not part of the predefined
categories; so new code categories were created. Data werecoded and then checked by multiple researchers for interrater
reliability.

Feldman et al
All interview data were analyzed to understand the challenges,
successes, and lessons learned within each dimension (technical,
organizational, and governance) and within each collaboratingorganization (ConnectVirginia, MEDfx, EHR vendor, and Inova)of on-boarding to an HIE network (in this case ConnectVirginia
EXCHANGE). Data were also analyzed to gain insight into
issues that would provide meaningful information regarding
factors contributing to success, value, and sustainability. Using
ATLAS.ti, this was performed by comparing organizations andcode families. For example, all stakeholder groups and all codes
related to challenges were selected to compare stakeholder
positions relative to challenges. Data were then exported to
Excel to view frequencies.

Figure 4. Coding structure. CCD=continuity of care document, EHR=electronic health record.

Results
Lessons Learned
The lessons learned, as derived from the challenges andsuccesses, are summarized in Table 3 across technical,
organizational, and governance dimensions. Each subsequent

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

subsection (technical, organizational, and governance) serves
to unpack those lessons learned in terms of challenges andsuccesses. Collectively, the findings not only provide a
retrospective account of Inova’s efforts in on-boarding toConnectVirginia EXCHANGE, but also offer insights into
various other stakeholders for future on-boarding efforts.

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 9
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Feldman et al

Table 3. Lessons learned from challenges and successes by dimension (technical, organizational, and governance).
Lessons learned

Challenges

Successes

Technical

Determine the most efficient environment for testing, Willingness to develop workarounds to unexpected software
decoupled from decision processes, actions, and depen- challenges, such as incompatible EHR versions.
dencies from other stakeholders.
Provide oversight and follow-up to increase technical Gain commitment from implementation site to set high priorunderstanding of appropriate on-boarding guides across ity on HIE implementation.
all stakeholders.
Use an EHR system specific CCD for validation, nota vendor supplied CCD template.
Conduct testing and implementation in clearly communicated iterations.
Articulate goals and priorities with vendors.
Understand roles and required resources in order tominimize time gaps and maximize efficiency.

Organizational

Account for competing IT priorities across organizations.

Participation of health system leadership.

Understand, communicate, and appreciate varying
stakeholder value proposition/motivations.

Timely and accurate communication, especially by and between the HIE and the health system.
Allocate appropriate human resources at the outset.

Governance

Ensure governance is in place, including policies,
Project champion possesses decision-making power, or, as
procedures, guidelines, and oversight across all orga- needed delegates appropriate decision making power to others.
nizations.
Obtain commitments from governance body early on
to facilitate project continuity.

Technical Dimension
This on-boarding effort between Inova and ConnectVirginia
entailed a range of technical activities and coordination toachieve success. There were four major challenges: (1) the
testing environment, (2) the on-boarding guides, (3) the CCD,
and (4) the vendors, and various successes contributed to thelessons learned.

Challenges
The Testing Environment
Regarding the testing environment, many interviewees felt thatsignificant time early in implementation was spent determining
which Inova environment would be used for testing. Inova had
two environments capable of producing CCDs: (1) test, and (2)proof of concept (POC); the latter is connected to the EHR
vendor’s connected network. Inova elected to use the POC
environment, which created work inefficiencies across Inova
and the EHR vendor. Analysis revealed the challenge, every
time Inova wanted to conduct software tests, it required theEHR vendor team to sign into POC, input scenarios, and start
the testing. The EHR vendor team considered these manualsteps as wasting time, which resulted in testing delays. Aninterviewee discussed this issue,
Had we chosen in the very beginning to use the test,
we would not have had any difficulty setting up test
patients to test in ConnectVirginia...[Newon-boarders] should think long and hard about their
testing environment and it should be done early inthe process. [An interviewee]
http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

A review of meeting notes shows that this process tookapproximately four weeks, whereas participants believed itshould have taken less than two weeks. During this time,
frustrations from Inova, ConnectVirginia, and MEDfx wereobserved by one of the researchers, who was consistently on
the conference calls. These frustrations were a direct result of
Inova’s reliance on the vendor related to CCD testing.
The lesson learned from the challenge of choosing a testing
environment was that this should be done independent of otherstakeholder actions. Determine the most efficient environment
for testing, decoupled from decision processes, actions, andother dependencies from other stakeholders.

The On-Boarding Guides
ConnectVirginia provides four guides to assist on-boarders: (1)checklist, (2) implementation, (3) testing, and (4) content, which,unless specified, will be referred to collectively as “on-boarding
guides.” Not all guides are meant to be read by all parties. For
example, the EHR vendor should read content guides, andtesting guides should be read by the on-boarding organization.
Many of those involved with this implementation had theperspective that the on-boarding guides were not read by thecorrect parties (or at all by anyone), but that, had they been,certain on-boarding tasks could have gone more smoothly andchallenges could have been avoided. For example, the checklist
is to assist in creating a shared understanding of the data needed,
relative to the data available; for example, it requires data on
all the laboratory tests that could possibly be run by the
on-boarding organization. Several interviewees suggested that
this document is long, complex, difficult to read, andoverwhelming to the reader, and therefore does not get
JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 10(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
completed. However, it is an important element in setting up
the CCD for validation. An interviewee suggested that it would
be better to go through the checklist section by section todetermine whether the data are present or absent.
The implementation and testing guides are provided to createa shared understanding and level set of expectations about therequirements for implementation and testing. From the meeting
notes and interviews, it was clear that these guides were notread to the degree needed for the project. An interviewee
explained, “Regardless of how painful [you] think it might be,read the [on-boarding] guides cover to cover.” This sameinterviewee thought the on-boarding guides were very well
written and provided plenty of the necessary helpful information.
Another interviewee suggested that discussing the on-boarding
guides among the implementation group would allow time forconversation and might raise issues not easily discovered by anindividual, which are illuminated when discussed in a group.
Last, the content guide is provided to streamline CCD validation
testing. This guide defines what the CCD should have in termsof object identifiers and the corresponding descriptions. Several
interviewees felt many misunderstandings could have been
avoided had this guide been read and discussed.
The lesson learned was to provide oversight and follow-up to
increase technical presentation, reading and understanding ofappropriate on-boarding guides across all stakeholders.

Continuity of Care Documents
The challenges with CCDs and HIEs are well documented in
the literature [43,45,70]. Initially, the EHR vendor wanted toprovide only template CCDs and not CCDs specific to Inova’s
EHR system, which includes customization. MedVirginia, which
conducted the CCD content validation, requested CCDs specificto Inova’s system to ensure that a CCD could be sent andreceived from Inova’s customized and nuanced system. Usinga standardized vendor template does not account for healthsystem customizations and risks not passing validation in theeHealth Exchange environment. An interviewee noted,
A scrubbed [vendor template] CCD will probably nothave any issues, but when it comes to testing a CCDfrom the actual system, there are going to be issues,
so you should not expect that just because the sample[vendor template] passed, that the node CCD will
pass; it probably won’t. [An interviewee]
Meeting notes and interviews reflect the opinion that this issue
required too much time devoted in isolation and should havebeen handled along with other on-boarding tasks. For example,
many participants commented that the technical piece (getting
systems to talk) and the content piece (the CCD) should havebeen conducted in parallel, rather than sequentially. Aninterviewee thought the CCD issue could have been managedin parallel to solving an issue with handshakes (ie, bidirectional
system-to-system acknowledgement),
While the CCD is the end point, the handshake had
issues. [The Inova] server has to be recognized by
MEDfx before any exchange can even happen. [Aninterviewee]

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Feldman et al
Once CCD validation was underway, many felt that the
on-boarding process, while excellent and thorough, needed to
strike a better balance between the acceptable and the desirable.
Several interviewees felt that if all parties had agreed onprioritization of issues (ie, with less attention to details that do
not matter), the CCDs could have passed testing much sooner
and had an earlier completion date.

The Vendors
For example, the top priority for the EHR vendor was to get thetechnical pieces to work from their end of the HIE. The vendor’sgoal was to focus on passing a CCD; not passing a CCD that
conforms to the updated eHealth Exchange specifications. Since
EHRs undergo a wide variety of customization, this is a criticalrequirement nuance relative to future participation in eHealthExchange. While it may have been acceptable to use a template
CCD, it was desirable (because of customizations and future
eHealth Exchange participation) to conduct CCD validation
with a system CCD. Participants other than the EHR vendor
felt the vendor was not focused on developing a technology tosupport all other stakeholder value propositions and motivations.
An interviewee described how differing priorities acrossstakeholders impacted the HIE outcome,Given the experience with [the EHR vendor], it isimportant to communicate to clients that this [theirdecision to limit the CCD] might impact what type ofapproval is granted at the end of on-boarding,because [Inova] ended up with a conditional approval
based on some of the things that we knew [the EHR
vendor] wasn’t going to budge on. [An interview]As noted above, Inova received only conditional approval asan HIE participant/node. The conditional approval was the resultof a 90 day medical record date range limitation imposed by
the EHR vendor’s CCD implementation for this project. The
US Social Security Administration requires more than threemonths of medical records in order to conduct disability
determinations, but the vendor’s implementation allowed only
three months’ worth of data. In this regard, an interviewee
observed,
Once we got into the testing process, that unveiled a
lot of proprietary issues with [the EHR vendor], eventhough they say their CCD is compliant. They havecertain things that are built into their product, mainly
from a competitive stand point. Inova was very reliant
on [the EHR vendor], and I think somewhat unaware
where those proprietary issues might impact
on-boarding to any statewide HIE. [An interviewee]
Interviewees had comments about both the EHR vendor andMEDfx. Many considered the EHR vendor inflexible andunknowledgeable, especially with regard to the 90 day medical
record date range limitation and requirements by the U.S. Social
Security Administration, a critical component in the value
proposition. Interviewees also felt that MEDfx was developing
as the project advanced. According to interviewees, this project
represented the first implementation of the latest eHealthExchange compliant gateway. MEDfx, the gateway provider,
had completed development of the gateway in June 2012, but
JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 11(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
had not yet completed testing. Thus, Inova became the test bed
for the gateway. This process of testing during implementation
contributed to the perception that MEDfx was developing as
the project was advancing. Both vendors (the EHR vendor andMEDfx) had to fix some bugs that were exposed during testing,and some interviewees thought the fixes should have been
completed before implementation or at least done more
expeditiously.
The lesson learned was to use an EHR system specific CCD for
validation, not a vendor supplied CCD template.
Data gathered from weekly on-boarding meetings, together with
document review, reflected the EHR vendor’s resistance to meetthe needs of a maturing and broader HIE, such as a statewide
HIE. An interviewee noted,[The EHR vendor] is a very restricted vendor aroundallowing third parties to do things like this. We
probably lost a month in this go around. [AnInterviewee]
Another interviewee said, “I regret that anyone thought [theEHR vendor] would change their mind.”
The lesson learned was that given the complexity of managing
expectations across multiple stakeholders, conduct testing andimplementation in clearly communicated iterations.
Several other challenges surfaced regarding software versions
that were incompatible with the latest data exchange standards.
The EHR vendor software version that Inova used for this
implementation (released in 2010) was not compliant with theupcoming eHealth Exchange specifications. The vendor will
not release a version compliant with the new specifications until
the 2012 version. Discussions with the EHR vendor suggested
that, due to the relative newness of the current 2010 version,
health systems will likely not deploy the 2012 version for sometime. This discrepancy in EHR specifications created significant
challenges, in terms of the CCD content, to completing
on-boarding. However, the EHR vendor’s perspective differedfrom that of ConnectVirginia regarding the ability to on-board,
saying, “We have many other clients that have on-boarded toeHealth Exchange, and none of them have these [CCD content]
issues that ConnectVirginia is citing.” In response,
ConnectVirginia participants explained that the clients to which
the EHR vendor refers had been on-boarded under the old
Nationwide Health Information Network specifications
established by ONC, rather than the new and required eHealthExchange standards established in September 2012.
ConnectVirginia must on-board to eHealth Exchange under the
updatedeHealthExchange
specifications.
Thus,
misunderstandings about the required specifications caused
significant implementation delays. Because this was the first
on-boarding with the EHR vendor, there were many unknowns.
It became apparent that an early meeting with the EHR vendor
was critical. Many felt this would have created a sharedunderstanding of some of the nuances of each other’s systems
and of stakeholders’ motivations, while also fostering
conversations about how each system adheres to theimplementation specifications.

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Feldman et al
The lesson learned was to establish early meetings with vendors
to articulate goals and priorities.
Interviews revealed challenges with understanding eachstakeholder’s roles. Several interviewees felt that time was lost
determining who was responsible for certain things, and tasks
were not done because one person thought another was
responsible. Likewise, better understanding was needed of thetechnical resources available: (1) Are the right people working
in the right place?; and (2) Is the testing environment one thatwill facilitate on-demand testing?. An interviewee felt that two
MEDfx people had the knowledge collectively, but lacked depth
individually. This type of situation led to delays or multiple
attempts to get questions answered. Another interviewee felt it
was critical to have representation across integrated delivery
teams, primarily because policy issues needed to be addressedsaying, “An interface group will build a pipe for your data to
pass, but there are lots of rules regarding audit streams.” A
majority of interviewees thought many of these late questions
or realizations could have been avoided by earlier and better
understanding of the on-boarding guides provided to Inova andthe EHR vendor. Much that was done was conductedsequentially; many interviewees thought the technical piece(getting the systems to talk) and the content piece (the CCD)
should have been done in parallel and speculated that doing sowould have saved a lot of time.
The lesson learned was to conduct clear communication early
on to discuss and understand roles and required resources inorder to minimize time gaps and maximize efficiency.

Successful Software Redevelopment
Regarding the incompatible software versions described above
with the 2010 EHR version, almost all the interviewees withknowledge of this issue commented on MEDfx’s willingness
to develop new code to address this challenge. While software
redevelopment took time, causing unanticipated delays,
everyone saw this as a significant success. A participantsummarized the thoughts of many,
We put a lot of responsibility on MEDfx to makeadjustments on their side to accommodate the factthat [Inova] was running a version that only
supported 2010. Thankfully, they were flexible enoughto accommodate that. [A participant]
While Inova could have on-boarded to ConnectVirginia without
this modification, ConnectVirginia would not have been able
to on-board to eHealth Exchange, thus minimizing the value
for Inova and any organization on-boarding after Inova. Allinterviewees felt that getting Inova on-boarded was a great
success in and of itself.
All stakeholders had competing priorities, but participants notedthat Inova’s may been the most significant. Although they werein the middle of an EHR implementation, they chose to pioneer
on-boarding to the ConnectVirginia EXCHANGE.
The lesson learned was to gain commitment from technology
stakeholders to be willing to develop workarounds to unexpectedsoftware challenges, such as incompatible EHR versions.

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 12(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
Another lesson learned was to determine conflicting priorities
across stakeholders at the outset. Gain commitment from
implementation site to set high priority on HIE implementation.

Organizational Dimension
Organizational factors were also instrumental to the success of
the Inova on-boarding experience. Concurrent EHR
implementation and strong leadership contributed to theorganizational challenges, successes, and lessons learned.

Challenging Competing Priorities
The major challenge involved a concurrent EHR implementation
at Inova and understanding each stakeholder’s value proposition
and motivation. Early on, the concurrent EHR implementation
created a situation of competing priorities. However, once roles
were more clearly defined regarding the ConnectVirginia
implementation, it was felt that resources were available andengaged. As one interviewee observed, “[The Inova internal
champion] kept us [Inova team] moving because we were very
busy with a lot of other stuff including [EHR] implementation.”
The lesson learned was to account for major competing IT
priorities at each participating organization. A concurrent EHR
implementation will likely compete directly with the HIE
implementation.
The value proposition and corresponding motivation for
on-boarding to ConnectVirginia EXCHANGE varied with thestakeholder. Several Inova participants commented that,
although the initial economic value proposition to Inova wasnonexistent, the motivation to move forward was very wellaligned with their vision to “reinvent hospital-based care toincrease value for our patients” and to “look outside our
hospitals to build an integrated network of providers andprograms to support our community.” The culture of this vision
was embedded in Inova employee beliefs. The words of several
were summed up by one Inova interviewee, “On-boarding toConnectVirginia [exchange] aligns with the Inova vision, fulfills
our desire to be part of transforming the Commonwealth [ofVirginia] into a great place to be a patient, and success for Inova
means great benefit for the community.”
Additional motivations for Inova involved the desire to beleaders in the HIE trend. Some interviewees questioned whether
or not the EHR vendor understood why this was so important
to Inova and ConnectVirginia independently and collectively,
and interviewees suggested the EHR vendor was sometimes
argumentative with requests from the on-boarding team, “Their[the EHR vendor’s] resistance to ConnectVirginia’s success iswhat concerns me.” To the other stakeholders, it seemed that
the EHR vendor did not have a clear motivation and was simply
responding reluctantly to client requests. These differing viewson value created communication breakdowns, frustrations, andinefficiencies in the on-boarding process. These breakdowns
and frustrations were observed numerous times on various
conference calls with the implementation team. There weretimes when it would take three or four calls to resolve one issue.
Such events led to inefficiencies in the on-boarding process.
The lesson learned was to understand, communicate, andappreciate varying stakeholder value proposition/motivations.
http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Feldman et al

Successful Leadership
Leadership was an important factor in this on-boarding process.
Almost all interviewees commented that Inova’s internal
champion, the Executive Vice President and Chief Technology
Officer of Inova, was a critical component in the success of the
project. Many suggested that his role on the ConnectVirginia
Governing Body put him in a position to be an internal champion
not only for Inova, but for ConnectVirginia as well, with hissolid understanding of what ConnectVirginia was trying toaccomplish and why it was important. During an interview, he
stated,I thought Inova needed to learn what HIE is andneeded to get its feet wet with the on-boarding so it
could be connected. I believe in HIE. [Executive Vice
President and Chief Technology Officer of Inova]
Scheduling sensitivities around Inova’s EHR implementation
created some time periods when key people were unavailable.
When this resulted in a lack of progress between meetings, theExecutive Vice President and Chief Technology Officer of Inova
could help guide the Inova team with managing those competing
priorities. His unique combination of being an internal champion
and a decision-maker greatly enhanced the success of thisproject. An interviewee further qualified the role of an internal
champion, “This cannot be a technical champion, but a true
champion...a true leader.” Leadership in terms of project
management was considered solid. Several interviewees
commented on Inova’s PM, and one summarized the words ofmany,
She [the PM] was prepared, answered emails
promptly and completely, and executed well. It was
extremely helpful to have her. [Interviewee]
The lesson learned was that the participation of health system
leadership is critical to success.
Communication was another area of success, and many felt thatPMs from both Inova and ConnectVirginia greatly contributed
to that success. As mentioned above, the PM from Inova wasalways prepared and answered emails promptly, and many others
commented on the PM from ConnectVirginia. An interviewee
capsulized ConnectVirginia’s communication efforts,
I appreciate the fact that they controlled a lot of the
documentation. They scheduled the weekly calls, set
up the agendas, sent out the minutes, and managed
any outstanding items across everyone. [Interviewee]
Several interviewees mentioned that the meeting minutes werevery thorough.
The lesson learned was that timely and accurate communication,
especially by and between the HIE and health system, isessential.
Despite the fact that this project competed with resources for
the Inova EHR implementation, many felt there were appropriate
resources. However, most interviewees observed that initiallythe correct resources were not allocated. Since this was the first
on-boarding, there were vague expectations about the level ofand appropriateness of resources. Teams required skills sets,

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 13(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
knowledge, and experience that were not available at the outset.
An interviewee noted,
Too many assumptions were made. We need to havea better kick-off to level set expectations and roles.
[An Interviewee]
Most interviewees felt that, by the second month, the teams
were appropriately resourced, and what was originally achallenge became a success. Several commented that there was
not a lot of movement regarding the resources, which added tothe teams’ strength individually and collectively. Anotherinterviewee summed up the lesson learned,
Put your best resources around standing this up,
because it requires you to pay attention to detail. Thisis more than an IT project; this is not a simple
interface project. [An Interviewee]
The lesson learned was to allocate appropriate human resourcesat the outset.

Governance Dimension
Intra and interorganizational decision-making power and clear
role definition have been shown to decrease intra andinterorganizational issues [71]. Furthermore, the nature of therelationships between decision makers is important in navigating
variable governance processes and structures and in sharing
decisions. Governance is the establishment of oversight,
standardized policies and procedures, and mechanisms to ensure
operation of an organization [72]. Thus, governance factors
such as the on-boarding guides, project resources, and a project
champion contributed to the lessons learned and the critical
success factors of the Inova on-boarding project.

Challenging Identification of Appropriate Policies andGuidelines
Identifying the appropriate policies and guidelines acrossstakeholders was challenging, as was selecting the best people
to provide oversight. Unfortunately, a structured governanceprocess did not predate this project, this was the first time thisparticular group of organizations had worked together, and thefirst time on-boarding to ConnectVirginia had taken place. As
noted previously, providing oversight and policy enforcement
for people to read the on-boarding guides proved challenging.
If on-boarding guides had been read thoroughly, it may havebeen easier to identify the correct governance resources or, atleast, ask questions regarding resource selection. Regarding thegovernance structure for the project, one interviewee
commented, “The technology supports the business, but thebusiness does not go anywhere without the right folks.” Anotherinterviewee said that she was, “...challenged to put together agovernance group that could attend the weekly on-boarding
meetings, as those were a great way of getting decisions made.”
These deficits resulted in the organizational and technicalchallenges described earlier, including missing nuances of the
on-boarding process, difficulties selecting key project resourcesat the outset of the project, and not providing a structure
whereby team members could request guidance in resource
selection. Fortunately, these issues were identified and quickly
rectified within the first two months of the project.
http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Feldman et al
The lesson learned was to ensure governance is in place,including policies, guidelines, and oversight across allorganizations.
Most interviewees agreed that the on-boarding guides provided
by ConnectVirginia were useful in explaining appropriate
governance such as policies, procedures, etc, once they were
read. The challenge was getting people to read them. Time wasshort, priorities competitive, and resources thin. However,
several interviewees agreed that thorough reading of the
on-boarding guides just before the kick-off meeting would havehelped ensure that proper governance decisions were made,especially in regards to policy decisions. It was also thoughtthat a thorough reading would have mitigated some downstream
misunderstandings and poor understanding of the system
requirements. Other than that, some interviewees thought theConnectVirginia PM could have done more to ensure that those
responsible for governance understood the on-boarding guides
pertaining to their part of the project. For example, at one
on-boarding meeting, one individual from Inova asked to go
through one of the on-boarding guides. An interviewee
commented, “Sometimes we all need to have our hand held,
and if that is what it would have taken to make sure everyone
went through the on-boarding guides, then so be it.”
The lesson learned was to obtain commitments from thegovernance body early on to facilitate project continuity.

Successful Project Champion
Most interviewees agreed that the real success in this project,from a governance perspective, was having a project champion
with decision-making power. Several times policy or procedure
decisions needed to be made; and because the Executive Vice
President and Chief Technology Officer of Inova was involved,
the appropriate questions were asked and decisions made. Aninterviewee gave a good example of how the project champion
provided appropriate decision-making authority,
When something comes up as an issue, helping tofigure out if it is a technical issue or a policy issue.
Then figuring out whose issue it is; is it a node
[hospital system] issue, is it the vendor, or is it
ConnectVirginia? [An Interviewee]
In these situations, the Executive Vice President and Chief
Technology Officer of Inova was able to provide guidance on
issues involving the node or Inova’s EHR vendor. Regarding
MEDfx and issues attributed to them, the PM had authority toprovide the guidance needed to move forward. Regarding CCDcontent validation, which was conducted by MedVirginia, it
was felt that the person on the on-boarding calls did not havedecision-making authority, and thus needed to seek guidance
after the call. Yet, interviewees felt her follow-up
communications were timely and comprehensive.
The lesson learned was that a project champion is essential who
possesses decision-making power, or, as needed, delegates
appropriate decision-making power to others.
It was clear to participants how critical it was to ensure fromthe beginning that a proper oversight structure is in place,including the people involved in the project. As mentioned in
JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 14(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Feldman et al

the challenges section, the on-boarding guides provided byConnectVirginia helped to provide guidance in this regard andto identify a governance structure. In addition to identifying
project resources, early identification of an internal champion
is essential for project success. But, as one interviewee
mentioned, it is sometimes difficult to have the internal
champion with decision-making authority at the weekly
meetings.

Stakeholder Perceived Value
Analysis of interviews, observations, and project documents,
taken together, also revealed a crosscutting theme in terms ofthe goals, priorities, motivations, and perceived value ofengaging in HIE. Earlier on in the implementation, an importantsuccess factor would have been to have a better understandingof how well the organizational goals of each participatingorganization aligned with one another; the implementation
priorities and motivations (social and economic) for eachorganization to participate; and the perceived value that eachorganization expected to gain as a result of participating. These
are illustrated in Table 4. Providing this information may haveavoided some of the challenges, constraints, and tensions
experienced, especially with the EHR vendor.
It was felt by many interviewees that had something like thebelow matrix existed, that clarity and insight would have been
gained early on.

Table 4. Matrix of goals, priorities, motivations, and perceived value propositions across implementation stakeholders.
ConnectVirginia

Inova

MEDfx

EHR vendor

Implementation goalalignment

AlignedAlignedAlignedNot aligned

Implementation priority

High

High

High

Low

Motivation

Social highSocial highSocial low

Social low

Economic moderate

Economic low

Economic highEconomic low

Fulfills the contract terms

None apparent

Perceived value

Provides exchange of medical infor- HIE leader in the statemationmedical information at the
point of care
Fulfills the contract terms

Social Security Administration disability determination

Discussion
Principal Findings
The main findings of this case study included several challenges,
successes, and lessons learned during the implementation of ahealth system on-boarding to a statewide HIE. Figure 5 showsa summary of the Blended Value Collaboration Enactment
Framework as applied to these findings and illustrates thetechnical, organizational, and governance collaboration thattook place between ConnectVirginia and Inova, along withcritical success factors and associated value proposition details.
This case study demonstrates that interorganizational
governance of HIE implementation is replete with interrelated
and overlapping technical, organizational, and governanceissues. The complexities of collaboration appear to assist as
well as detract from realizing a set of common goals. For the
expanded view of HIE (ie, across states and the nation), thebroader significance of this case study is the proposition thatsuccessful implementation of a large-scale emergent HIE systemshould consider the expected and realized blended value acrossall participants. Consistent with the literature, while economic

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderXvalue is an important goal, the organizations presented here
have regarded the value proposition primarily to include social
value, believing that economic value will follow at some point
in time.
As mentioned earlier, the sociotechnical approach allows forunderstanding independent linkages between complex social
and technological components. Much of what was learned fromthis first on-boarding effort is related to accomplishing tasksearlier in the process, rather than allowing them to be discovered
in course. While this may be common knowledge in moreestablished IT implementations, the field of statewide HIE
on-boarding implementations is undeveloped in this area. Theinterviews illuminated some common issues such as
interoperable HIE and better care. However, it was notable thatboth organizations, when asked about their motivation tocollaborate, cited the social good that would result from creatinga critical mass and contributing to the Commonwealth’s HIE
initiative. Since Inova has no other organizations with which
to exchange, and ConnectVirginia does not yet receive revenue
from on-boarding organizations, both organizations were
motivated primarily for the social good.

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 15(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Feldman et al

Figure 5. Blended Value Collaboration Enactment Framework as applied to findings. T=technical, O=organizational, G=governance, and HIE=healthinformation exchange.

Application of the Framework
Turning to the Blended Value Collaboration Framework as
applied to the findings, Figure 5 illustrates the interdependent
linkages between the technical, organizational, and governancedimensions for each organization. When these individual
dimensions come together in the collaboration there arecontributing factors that facilitate enactment of the collaboration.
At this early point in the project, both organizations were heavily
weighted toward social motivations, recognizing that asConnectVirginia matures the economic motivation will growmore prominent.

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Frequently in public-private collaborations, the value
propositions of collaborating organizations are not aligned.
However, in this case, we found the value propositions between
ConnectVirginia and Inova to be very well aligned and centeredon organizational missions and goals, better care for consumers,
and being leaders in HIE. This is very consistent with the IOS
literature [41,42]. As mentioned earlier, this study focused on
the implementation, so there was no usage, and only one periodof time was studied. Thus, the evolutionary changes over time
portion of the framework were not addressed in these findings(gray portion in Figure 5). This sociotechnical approachfacilitates the consideration of the social and the technical

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 16(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
perspectives, and their contribution to the overall value
proposition.
Supporting Mody’s [32] assertion, describer earlier in this paper,that social considerations could exert more pressure thaneconomic considerations, this framework highlights social
reasons as very significant motivators for early
adopters/implementers, frequently outweighing economic
motivations. Lacking a framework that considers social
motivations, the natural tendency in IT projects might be to
analyze, or only consider, economic motivators, and then judge
the value proposition on that basis. The Blended Value
Collaboration Enactment Framework fills a gap in, and makes
a contribution to, the STS framework literature. Its application
promotes the assessment of a wide range of observed issues(across technical, organizational, and governance dimensions)in relation to an interorganizational health IT implementation,comparing them with IOS goals, motivations, andintraorganizational priorities, and then determining the successfactors and value propositions from the results. Used as aheuristic, the framework may provide for a broader and more
inclusive evaluation of an IOS health IT implementation.
Furthermore, the current framework enables examination ofchanging motivations and value propositions over time.
Future studies should revisit the findings reported here to
analyze such changes. As time progresses and ConnectVirginia
matures, an increased economic motivation is expected fromboth organizations. In the future, organizations are also expected
to on-board primarily for perceived potential economic factors,
although these are yet to be realized. Future research should
assess a wide range of economic and clinical factors associatedwith HIE value; while continuing to define, include, and broadensocial factors and public value. Mixed-method case studies canbe used in this regard to more fully understand the breadth anddepth of mediating, moderating, and control variables to assess
in future quantitative studies. Studying HIE implementations
broadly across the United States through survey research isdesperately needed as most studies, including those referenced
in this manuscript, consist of small sample sizes. In terms ofcontent, the value proposition in HIE is a moving target, as boththe act of HIE and entity called HIE continues to evolve andFeldman et alchange. New government requirements and incentives, newbusiness models to facilitate HIE, and increased societal demandfor better and less expensive health care are expected to continue
to shape the HIE landscape. Knowing this evolution will occur
should not deter near-term research. These studies are neededin order to make the ongoing practical impact discussed at theoutset of this manuscript, to address the triple aim of health carebroadly across regions.

Limitations
This case study examines the implementation of one healthsystem on-boarding to a statewide HIE. As such, generalizability
may be limited. Another factor holding potential to contributeto this limitation includes that the Chief Medical Information
Office of the health system was the statewide HIE Governing
Body Vice-Chair, which may have served to introduce
motivations and bias that a different implementation would nothave experienced. However, it may also be commonplace forexisting health care leaders in regions and states to take a strongrole in HIE governance. Further research is needed to apply theprinciples from this study to other implementations, so as togain generalizability of the findings.

Conclusions
This study focuses on the evaluation of HIE implementation in
Virginia. From a practical perspective, the study provides a setof lessons learned for others who are implementing systems
across a statewide HIE. This study also includes considerations
for eHealth Exchange implementation. As mentioned earlier
and substantiated in the literature, on-boarding to eHealthExchange is part of the economic value proposition equation.
On-boarding to ConnectVirginia with a CCD that will not passtesting when ConnectVirginia on-boards to eHealth Exchange
eliminates a critical value proposition component. From amethodological perspective, it provides an example of how suchan HIE implementation can be studied, and from a theoreticalperspective, this study builds on the literature on IOS for healthcare, addressing the core questions: (1) What value propositions
motivate an organization to participate in HIE implementation?;and (2) What success factors should be targeted in HIE
implementation evaluation?.

Acknowledgments
The authors acknowledge and thank the interviewees who participated in this study. Their time was very much appreciated. Theirvarious perspectives provided valuable input to this case study and useful information for other organizations in the on-boarding
process. The authors also thank the reviewers whose suggestions made this manuscript more valuable.

Conflicts of Interest
While this case study was being conducted, the corresponding author was contracted as the external evaluator, a federal cooperative
agreement requirement, to CHA, the subcontractor responsible for building ConnectVirginia. There are no other real or perceived
conflicts of interest.

References
1.

2.

State health information exchange. State health information exchange cooperative agreement program URL: http://www.
healthit.gov/policy-researchers-implementers/state-health-information-exchange [accessed 2014-04-03] [WebCite Cache
ID 6OYsOqD8R]
The IHI triple aim. URL: http://www.ihi.org/engage/initiatives/TripleAim/Pages/default.aspx [accessed 2014-04-03][WebCite Cache ID 6OYsd691v]

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 17(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
3.
4.
5.
6.

7.
8.

9.
10.
11.

12.
13.

14.

15.
16.
17.
18.

19.
20.
21.

22.
23.

24.

25.

Furukawa MF, Patel V, Charles D, Swain M, Mostashari F. Hospital electronic health information exchange grew substantiallyin 2008-2012. Health Aff (Millwood) 2013 Aug;32(8):1346-1354. [doi: 10.1377/hlthaff.2013.0010] [Medline: 23918477]
Vest JR, Gamm LD. Health information exchange: Persistent challenges and new strategies. J Am Med Inform Assoc
2010;17(3):288-294 [FREE Full text] [doi: 10.1136/jamia.2010.003673] [Medline: 20442146]
Unertl KM, Johnson KB, Gadd CS, Lorenzi NM. Bridging organizational divides in health care: An ecological view ofhealth information exchange. JMIR Med Inform 2013 Oct 29;1(2):e3. [doi: 10.2196/medinform.2510]
Shade SB, Chakravarty D, Koester KA, Steward WT, Myers JJ. Health information exchange interventions can enhancequality and continuity of HIV care. Int J Med Inform 2012 Oct;81(10):e1-e9. [doi: 10.1016/j.ijmedinf.2012.07.003] [Medline:
22854158]
Vest JR, Miller TR. The association between health information exchange and measures of patient satisfaction. Appl Clin
Inform 2011;2(4):447-459 [FREE Full text] [doi: 10.4338/ACI-2011-06-RA-0040] [Medline: 23616887]
McGee MK. InformationWeek. 2010. Health information enhances decision making URL: http://www.informationweek.com/
healthcare/clinical-information-systems/health-information-exchange-enhances-decision-making/d/d-id/
1090004?page_number=1 [accessed 2014-04-03] [WebCite Cache ID 6OYsvJgCx]
Dimitropoulos L, Patel V, Scheffler SA, Posnack S. Public attitudes toward health information exchange: Perceived benefits
and concerns. Am J Manag Care 2011 Dec;17(12 Spec No):SP111-SP116 [FREE Full text] [Medline: 22216769]
Hincapie A, Warholak T. The impact of health information exchange on health outcomes. Appl Clin Inform 2011;2(4):499-507[FREE Full text] [doi: 10.4338/ACI-2011-05-R-0027] [Medline: 23616891]
Frisse ME, Johnson KB, Nian H, Davison CL, Gadd CS, Unertl KM, et al. The financial impact of health informationexchange on emergency department care. J Am Med Inform Assoc 2012;19(3):328-333 [FREE Full text] [doi:10.1136/amiajnl-2011-000394] [Medline: 22058169]
Kaelber DC, Bates DW. Health information exchange and patient safety. J Biomed Inform 2007 Dec;40(6 Suppl):S40-S45[FREE Full text] [doi: 10.1016/j.jbi.2007.08.011] [Medline: 17950041]
Hessler BJ, Soper P, Bondy J, Hanes P, Davidson A. Assessing the relationship between health information exchanges andpublic health agencies. J Public Health Manag Pract 2009;15(5):416-424. [doi: 10.1097/01.PHH.0000359636.63529.74][Medline: 19704310]
Dobbs D, Trebatoski M, Revere D. The northwest public health information exchange's accomplishments in connecting ahealth information exchange with public health. Online J Public Health Inform 2010;2(2) [FREE Full text] [doi:10.5210/ojphi.v2i2.3210] [Medline: 23569585]
Shapiro JS, Mostashari F, Hripcsak G, Soulakis N, Kuperman G. Using health information exchange to improve publichealth. Am J Public Health 2011 Apr;101(4):616-623. [doi: 10.2105/AJPH.2008.158980] [Medline: 21330598]
Committee on Quality of Health Care in America, Institute of Medicine. Crossing the quality chasm: A new health system
for the 21st century. Washington, D.C: National Academy Press; 2001.
Fontaine P, Ross SE, Zink T, Schilling LM. Systematic review of health information exchange in primary care practices.
J Am Board Fam Med 2010;23(5):655-670 [FREE Full text] [doi: 10.3122/jabfm.2010.05.090192] [Medline: 20823361]
Bailey JE, Wan JY, Mabry LM, Landy SH, Pope RA, Waters TM, et al. Does health information exchange reduce unnecessary
neuroimaging and improve quality of headache care in the emergency department? J Gen Intern Med 2013 Feb;28(2):176-183[FREE Full text] [doi: 10.1007/s11606-012-2092-7] [Medline: 22648609]
Brailer DJ. Interoperability: The key to the future health care system. Health Aff (Millwood) 2005;Suppl Web
Exclusives:W5-19 [FREE Full text] [doi: 10.1377/hlthaff.w5.19] [Medline: 15659454]
Kuperman GJ. Health-information exchange: Why are we doing it, and what are we doing? J Am Med Inform Assoc
2011;18(5):678-682 [FREE Full text] [doi: 10.1136/amiajnl-2010-000021] [Medline: 21676940]
Unertl KM, Johnson KB, Lorenzi NM. Health information exchange technology on the front lines of healthcare: Workflow
factors and patterns of use. J Am Med Inform Assoc 2012;19(3):392-400 [FREE Full text] [doi:10.1136/amiajnl-2011-000432] [Medline: 22003156]
Miller RH, Miller BS. The Santa Barbara County care data exchange: What happened? Health Aff (Millwood)
2007;26(5):w568-w580 [FREE Full text] [doi: 10.1377/hlthaff.26.5.w568] [Medline: 17670775]
California Healthcare Foundation. Achieving the right balance: Privacy and security policies to support electronic healthinformation exchange. 2012 URL: http://www.chcf.org/~/media/MEDIA%20LIBRARY%20Files/PDF/A/
PDF%20AcheivingBalancePrivacySecurityHIE.pdf [accessed 2014-04-03] [WebCite Cache ID 6OYtRTlwS]
Adler-Milstein J, McAfee AP, Bates DW, Jha AK. The state of regional health information organizations: Current activities
and financing. Health Aff (Millwood) 2008;27(1):w60-w69 [FREE Full text] [doi: 10.1377/hlthaff.27.1.w60] [Medline:
18073225]
McDonald CJ, Overhage JM, Barnes M, Schadow G, Blevins L, Dexter PR, INPC Management Committee. The Indiana
network for patient care: A working local health information infrastructure. An example of a working infrastructure
collaboration that links data from five health systems and hundreds of millions of entries. Health Aff (Millwood)
2005;24(5):1214-1220 [FREE Full text] [doi: 10.1377/hlthaff.24.5.1214] [Medline: 16162565]

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Feldman et al

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 18(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
26.

27.

28.

29.
30.
31.
32.
33.

34.
35.
36.

37.
38.

39.

40.

41.
42.
43.

44.

45.
46.
47.
48.
49.

Stead WW, Kelly BJ, Kolodner RM. Achievable steps toward building a national health information infrastructure in the
United States. J Am Med Inform Assoc 2005;12(2):113-120 [FREE Full text] [doi: 10.1197/jamia.M1685] [Medline:
15561783]
Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B. The value of health care information exchange
and interoperability. Health Aff (Millwood) 2005;Suppl Web Exclusives:W5-10 [FREE Full text] [doi: 10.1377/hlthaff.w5.10][Medline: 15659453]
Whitworth B, Ahmad A, Soegaard M, Dam RF. The encyclopedia of human-computer interaction. 2nd ed. Aarhus, Denmark:The Interaction Design Foundation; 2013. Socio-technical system design URL: http://www.interaction-design.org/
encyclopedia/socio-technical_system_design.html [accessed 2014-04-03] [WebCite Cache ID 6OYtl5VWu]
Porter M, Teisberg E. Creating value-based competition on results. In: Redefining health care. USA: Harvard Business
School Press; 2006.
Bowen PL, Cheung MD, Rohde FH. Enhancing IT governance practices: A model and case study of an organization's
efforts. International Journal of Accounting Information Systems 2007 Sep;8(3):191-221. [doi: 10.1016/j.accinf.2007.07.002]
Reich BH, Benbasat I. An empirical investigation of factors influencing the success of customer-oriented strategic systems.
Information Systems Research 1990 Sep;1(3):325-347. [doi: 10.1287/isre.1.3.325]
Mody A. Infrastructure delivery: Private initiative and the public good. Washington, D.C: World Bank; 1996.
Emerson J. California Management Review. 2003. The blended value proposition: Integrating social and financial returns
URL: http://www.blendedvalue.org/the-blended-value-proposition-integrating-social-and-financial-returns/ [accessed
2014-06-08] [WebCite Cache ID 6QBGzxC7N]
Whitworth B. The social requirements of technical systems. In: Moor AD, editor. Handbook of research on socio-technicaldesign and social networking systems (2-volumes). USA: Information Science Reference; 2009.
Cash JI, Kosynski BR. Harv Bus Rev. 1985 Apr. IS redraws competitive boundaries URL: http://hbr.org/1985/03/
is-redraws-competitive-boundaries/ar/1 [accessed 2014-07-28] [WebCite Cache ID 6RPGlR1oq]
Williams CB, Federowicz J. A framework for analyzing cross-boundary e-government projects: The CapWin example. :Proceedings of the 2005 National Conference on Digital Government Research, DG. O; 2005 Presented at: USA; May
2005; Atlanta, Georgia, USA p. 15-18 URL: http://dl.acm.org/citation.
cfm?id=1065337&dl=ACM&coll=DL&CFID=516585855&CFTOKEN=36368055
Premkumar G, Ramamurthy K. The role of interorganizational and organizational factors on the decision mode for adoptionof interorganizational systems. Decision Sciences 1995 May;26(3):303-336. [doi: 10.1111/j.1540-5915.1995.tb01431.x]
Melville N, Kraemer K, Gurbaxani V. MIS Quarterly. 2004. Review: Information technology and organizational performance:
An integrative model of IT business value URL: http://misq.org/
review-information-technology-and-organizational-performance-an-integrative-model-of-it-business-value.html [accessed
2014-07-28] [WebCite Cache ID 6RPHHC9DK]Barua A, Konana P, Whinston AB, Yin F. MIS Quarterly. 2004. An empirical investigation of net-enabled business value
URL: http://misq.org/an-empirical-investigation-of-net-enabled-business-value.html [accessed 2014-07-28] [WebCite
Cache ID 6RPHR7DeR]
Delone WH, McLean ER. Journal of Management Information Systems. 2003. The DeLone and McLean model of information
systems success: A ten-year update URL: http://www.mesharpe.com/MISVirtual/07Delone.pdf [accessed 2014-07-28][WebCite Cache ID 6RPHZlKCg]
Bardach E. Getting agencies to work together: The practice and theory of managerial craftmanship. Washington, D.C:
Brookings Institution Press; 1998.
Yusof MM, Papazafeiropoulou A, Paul RJ, Stergioulas LK. Investigating evaluation frameworks for health information
systems. Int J Med Inform 2008 Jun;77(6):377-385. [doi: 10.1016/j.ijmedinf.2007.08.004] [Medline: 17904898]
Feldman SS, Horan TA. Collaboration in electronic medical evidence development: A case study of the Social Security
Administration's MEGAHIT System. Int J Med Inform 2011 Aug;80(8):e127-e140. [doi: 10.1016/j.ijmedinf.2011.01.012][Medline: 21333588]
Saef S, Bourne C, Bush J, Scott L, Gaafary H, Keenan K, et al. The impact of a health information exchange on resource
use and medicare-allowable charges at eleven emergency departments operated by four major hospital systems in a midsizedSoutheastern city: An observational study using clinician estimates. Annals of Emergency Medicine 2013 Oct;62(4):S97.
[doi: 10.1016/j.annemergmed.2013.07.090]
Feldman SS. Public-private interorganizational sharing of health data for disability determination. Ann Arbor, MI: ProQuest,
UMI; 2011.
Inova overview. URL: http://www.inova.org/about-inova/index.jsp [accessed 2014-04-03] [WebCite Cache ID 6OYuGVJGe]
Ash JS, Guappone KP. Qualitative evaluation of health information exchange efforts. J Biomed Inform 2007 Dec;40(6Suppl):S33-S39 [FREE Full text] [doi: 10.1016/j.jbi.2007.08.001] [Medline: 17904914]
Johnson KB, Gadd C. Playing smallball: Approaches to evaluating pilot health information exchange systems. J Biomed
Inform 2007 Dec;40(6 Suppl):S21-S26 [FREE Full text] [doi: 10.1016/j.jbi.2007.08.006] [Medline: 17931981]
Cooper RB, Zmud RW. Information technology implementation research: A technological diffusion approach. Management
Science 1990 Feb;36(2):123-139. [doi: 10.1287/mnsc.36.2.123]

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

Feldman et al

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 19(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
50.

51.
52.
53.

54.

55.

56.

57.
58.
59.
60.
61.
62.

63.
64.
65.
66.
67.
68.
69.
70.
71.
72.

Feldman et al

Hripcsak G, Kaushal R, Johnson KB, Ash JS, Bates DW, Block R, et al. The United Hospital Fund meeting on evaluating
health information exchange. J Biomed Inform 2007 Dec;40(6 Suppl):S3-10 [FREE Full text] [doi: 10.1016/j.jbi.2007.08.002][Medline: 17919986]
Ammenwerth E, Gräber S, Herrmann G, Bürkle T, König J. Evaluation of health information systems-problems andchallenges. Int J Med Inform 2003 Sep;71(2-3):125-135. [Medline: 14519405]
Marchibroda JM. Health information exchange policy and evaluation. J Biomed Inform 2007 Dec;40(6 Suppl):S11-S16[FREE Full text] [doi: 10.1016/j.jbi.2007.08.008] [Medline: 17981099]
Kern LM, Kaushal R. Health information technology and health information exchange in New York State: New initiatives
in implementation and evaluation. J Biomed Inform 2007 Dec;40(6 Suppl):S17-S20 [FREE Full text] [doi:10.1016/j.jbi.2007.08.010] [Medline: 17945542]
Labkoff SE, Yasnoff WA. A framework for systematic evaluation of health information infrastructure progress incommunities. J Biomed Inform 2007 Apr;40(2):100-105 [FREE Full text] [doi: 10.1016/j.jbi.2006.01.002] [Medline:
16530489]
Vest JR, Jasperson '. How are health professionals using health information exchange systems? Measuring usage for
evaluation and system improvement. J Med Syst 2012 Oct;36(5):3195-3204 [FREE Full text] [doi:10.1007/s10916-011-9810-2] [Medline: 22127521]
Schooley BL, Horan TA. Towards end-to-end government performance management: Case study of interorganizational
information integration in emergency medical services (EMS). Government Information Quarterly 2007 Oct;24(4):755-784.
[doi: 10.1016/j.giq.2007.04.001]
Glasgow RE. eHealth evaluation and dissemination research. Am J Prev Med 2007 May;32(5 Suppl):S119-S126. [doi:10.1016/j.amepre.2007.01.023] [Medline: 17466816]
McGowan JJ, Cusack CM, Poon EG. Formative evaluation: A critical component in EHR implementation. J Am Med
Inform Assoc 2008;15(3):297-301 [FREE Full text] [doi: 10.1197/jamia.M2584] [Medline: 18308984]
Weick KE. Small wins: Redefining the scale of social problems. American Psychologist 1984;39(1):40-49. [doi:10.1037/0003-066X.39.1.40]
Fountain JE. Building the virtual state: Information technology and institutional change. Washington, D.C: Brookings
Institution Press; 2001.
Geels FW. From sectoral systems of innovation to socio-technical systems: Insights about dynamics and change fromsociology and institutional theory. Research Policy 2004 Sep;33(6-7):897-920. [doi: 10.1016/j.respol.2004.01.015]
Kern LM, Barron Y, Abramson EL, Patel V, Kaushal R. HEAL NY: Promoting interoperable health information technology
in New York State. Health Aff (Millwood) 2009;28(2):493-504 [FREE Full text] [doi: 10.1377/hlthaff.28.2.493] [Medline:
19276009]
Markus ML. Power, politics, and MIS implementation. Commun. ACM 1983;26(6):430-444. [doi: 10.1145/358141.358148]
Ash JS, Anderson NR, Tarczy-Hornoch P. People and organizational issues in research systems implementation. J Am
Med Inform Assoc 2008;15(3):283-289 [FREE Full text] [doi: 10.1197/jamia.M2582] [Medline: 18308986]
Kuperman GJ. Doing interdisciplinarity: Motivation and collaboration in research for sustainable agriculture in the UK. J
Am Med Inform Assoc 2011;18(5):678-682 [FREE Full text] [doi: 10.1111/j.1475-4762.2008.00859.x]
Ramírez R. Value co-production: Intellectual origins and implications for practice and research. Strat. Mgmt. J 1999
Jan;20(1):49-65 [FREE Full text] [doi: 10.1002/(SICI)1097-0266(199901)20:1<49::AID-SMJ20>3.0.CO;2-2]
Bititci US, Martinez V, Albores P, Parung J. Creating and managing value in collaborative networks. International Journalof Physical Distribution & Logistics Management 2004;34(3):251-268. [doi: 10.1108/09600030410533574]
Friesen ME, Johnson JA. The success paradigm: Creating organizational effectiveness through quality and strategy. Westport,
Conn: Quorum Books; 1995.
ATLAS. ti qualitative data anaylsis URL: http://www.atlasti.com/index.html [accessed 2014-04-03] [WebCite Cache ID
6OYuNTpeB]
D'Amore JD, Sittig DF, Wright A, Iyengar MS, Ness RB. The promise of the CCD: Challenges and opportunity for quality
improvement and population health. AMIA Annu Symp Proc 2011;2011:285-294 [FREE Full text] [Medline: 22195080]
Phillips N, Lawrence TB, Hardy C. Inter-organizational collaboration and the dynamics of institutional fields. Journal ofManagement Studies 2000 Jan;37(1):23-43. [doi: 10.1111/1467-6486.00171]
Contractor FJ, Lorange P. Trends in international collaborative agreements. In: Cooperative strategies in international
business. Lexington, Mass: Lexington Books; 1988.

Abbreviations
CCD: continuity of care document
CHA: Community Health Alliance
EHR: electronic health record
HIE: health information exchange
Inova: Inova Health System
http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 20(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Feldman et al

IOS: interorganizational system
IT: information technology
ONC: Office of the National Coordinator for Health Information Technology
PM: project manager
POC: proof of concept
STS: sociotechnical systems
VDH: Virginia Department of Health

Edited by G Eysenbach; submitted 06.04.14; peer-reviewed by B Tulu; comments to author 28.04.14; revised version received 09.06.14;
accepted 10.07.14; published 15.08.14
Please cite as:
Feldman SS, Schooley BL, Bhavsar GP
Health Information Exchange Implementation: Lessons Learned and Critical Success Factors From a Case StudyJMIR Med Inform 2014;2(2):e19
URL: http://medinform.jmir.org/2014/2/e19/
doi: 10.2196/medinform.3455
PMID: 25599991©Sue S Feldman, Benjamin L Schooley, Grishma P Bhavsar. Originally published in JMIR Medical Informatics(http://medinform.jmir.org), 15.08.2014. This is an open-access article distributed under the terms of the Creative Commons
Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction
in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete
bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license
information must be included.

http://medinform.jmir.org/2014/2/e19/

XSL• FO
RenderX

JMIR Med Inform 2014 | vol. 2 | iss. 2 | e19 | p. 21(page number not for citation purposes)