HEALTH 

INFORMATION 

TECHNOLOGY

HHS Should Assess 

the Effectiveness of 

Its Efforts to Enhance

Patient Access to and 

Use of Electronic 

Health Information

Report to Congressional Requesters

March 2017

GAO-17-305

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-17-305, a report to 

congressional requesters

March 2017

HEALTH INFORMATION TECHNOLOGY

HHS Should Assess the Effectiveness of Its Efforts to 

Enhance Patient Access to and Use of Electronic 

Health Information 

What GAO Found

Since 2009, the Department of Health and Human Services (HHS) has invested 

over $35 billion in health information technology, including efforts to enhance 

patient access to and use of electronic health information. One of the largest 

programs is the Centers for Medicare & Medicaid Services’ (CMS) Medicare 

Electronic Health Record Incentive Program (Medicare EHR Program), which, 

among other things, encourages providers to make electronic health information 

available to patients. Program data for 2015 show that health care providers that 

participated in the program (3,218 hospitals and 194,200 health care 

professionals such as physicians) offered most of their patients the ability to 

electronically access health information. Patients generally described this access 

as beneficial, but noted limitations such as the inability to aggregate their 

longitudinal health information from multiple sources into a single record. 

Data from the 2015 Medicare EHR Program show that relatively few patients 

electronically access their health information when offered the ability to do so. 

Patients GAO interviewed described primarily accessing health information 

before or after a health care encounter, such as reviewing the results of a 

laboratory test or sharing information with another provider.

Average Percentage of Patients of 2015 Medicare EHR Program Participating Providers Who 

Were Offered Access and Electronically Accessed Available Health Information

While HHS has multiple efforts to enhance patients’ ability to access their 

electronic health information, it lacks information on the effectiveness of these 

efforts. The Office of the National Coordinator for Health Information Technology 

(ONC) within HHS collaborates with CMS to assess CMS’s Medicare EHR 

Program as well as its own efforts to enhance patient access to and use of 

electronic health information. However, ONC has not developed outcome 

measures for these efforts consistent with leading principles for measuring 

performance. Without such measures, HHS lacks critical information necessary 

to determine whether each of its efforts are contributing to the department’s 

View GAO overall goals, or if these efforts need to be modified in any way. -17-305. For more information, 

contact Carolyn Yocom at (202) 512-7114 or 

yocomc@gao.gov.

Why GAO Did This Study

HHS’s goal is that all Americans will be 

able to electronically access their 

longitudinal health information, that is, 

their health information over time. 

HHS’s efforts to achieve this goal 

include the Medicare EHR Program 

and other efforts to encourage 

providers to make patient health 

information available and for patients 

to access such information. 

GAO was asked to review the state of 

patients’ electronic access to their 

health information. This report (1) 

describes the electronic access to 

health information available to patients, 

and patients’ views of this access, (2) 

describes the extent to which patients 

electronically access their health 

information, and actions providers 

reported taking to encourage such 

access, and (3) evaluates HHS’s 

efforts to advance patients’ ability to 

electronically access their health 

information. GAO analyzed data from 

HHS and other sources; reviewed 

applicable strategic planning 

documents; surveyed a generalizable 

sample of providers that participated in 

the Medicare EHR program; and 

interviewed HHS officials and a 

nongeneralizable sample of patients, 

providers, and health information 

technology product developers. 

What GAO Recommends

GAO recommends that HHS 1) 

develop performance measures to 

assess outcomes of key efforts related 

to patients’ electronic access to 

longitudinal health information, and 2) 

use the information from these 

measures to help achieve program 

goals. HHS concurred with the 

recommendations.
Page i GAO-17-305 Health Information Technology

Letter 1

Background 5

Most Patients of Providers Participating in the 2015 Medicare 

EHR Program Were Offered Electronic Access to Their Health 

Information and Noted Benefits and Limitations of Such Access 9

Few Patients of Providers Participating in the Medicare EHR 

Program Electronically Accessed Their Health Information, and 

Providers Reported Taking Steps to Encourage Access 16

HHS Has Taken Steps to Increase Patients’ Electronic Access to 

Health Information but Lacks Outcome Measures to Determine 

the Effectiveness of These Efforts 26

Conclusions 31

Recommendations 31

Agency Comments and our Evaluation 32

Appendix I Methodology for Medicare Electronic Health Record Incentive 

Program (Medicare EHR Program) Data Analysis and Provider 

Survey 35

Appendix II ONC Efforts to Further Patients’ Ability to Access Longitudinal 

Electronic Health Information 42

Appendix III Comments from the Department of Health and Human Services 44

Appendix IV GAO Contact and Staff Acknowledgments 49

Tables

Table 1: Description of Process and Outcome Measures 8

Table 2: Estimated Percentages of Providers Who Participated in 

the 2014 Medicare EHR Program that Routinely Offer 

Various Types of Health Information through Patient 

Portals 13

Contents
Page ii GAO-17-305 Health Information Technology

Table 3: Estimated Percentages of Methods Used by Providers 

Who Participated in the 2014 Medicare EHR Program to 

Encourage Patients to Electronically Access Their Health 

Information 23

Table 4: Description of Stratified Sampling Frame and Sample 

Sizes 39

Table 5: Description of Valid Responses 39

Figures

Figure 1: Average Percentage of Patients of 2015 Medicare EHR 

Program Participating Providers Who Were Offered 

Access and Electronically Accessed Available Health 

Information 17

Figure 2: Average Percentage of Patients of 2015 Medicare EHR 

Program Health Care Professionals Who Electronically 

Accessed Available Health Information, by Provider 

Characteristic 19

Figure 3: Average Percentage of 2015 Medicare EHR Program 

Participants’ Patients Who Electronically Accessed 

Available Health Information, by Percentage of County 

Residents Over Age 65 20

Figure 4: Average Percentage of 2015 Medicare EHR Program 

Health Care Professionals’ Patients Who Electronically 

Accessed Available Health Information, by EHR System 

Vendor 25

Figure 5: Average Percentage of 2015 Medicare EHR Program 

Hospitals’ Patients Who Electronically Accessed 

Available Health Information, by EHR System Vendor 26
Page iii GAO-17-305 Health Information Technology

Abbreviations

C-CDA Consolidated-Clinical Document Architecture

CMS Centers for Medicare & Medicaid Services

EHR electronic health record

HHS Department of Health and Human Services

HITECH Act Health Information Technology for Economic and

 Clinical Health Act 

HRSA Health Resources and Services Administration

IT information technology

ONC Office of the National Coordinator for Health 

 Information Technology

PHR personal health record

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-17-305 Health Information Technology

441 G St. N.W.

Washington, DC 20548

March 15, 2017

Congressional Requesters

Since the enactment of the Health Information Technology for Economic 

and Clinical Health (HITECH) Act in 2009, the federal government has 

invested over $35 billion to accelerate the development and adoption of 

health information technology (IT). These efforts are aimed at 

encouraging investment in and the use of electronic health records (EHR) 

and related technologies that, among other things, enable patients to 

access their health information when they need it in a useful electronic 

format. This health information can include, for example, summaries of 

vital signs, such as blood pressure; laboratory test and other screening 

results; and information on present and past prescriptions. One of the 

goals of the Office of the National Coordinator for Health Information 

Technology (ONC), an office within the Department of Health and Human 

Services (HHS), is for patients to have the ability to electronically access 

and aggregate their longitudinal health information in order to provide 

comprehensive information on their health over time.1 According to ONC, 

when patients have such access to their health information, they are 

better able to monitor chronic conditions, ensure their health information 

is accurate, and share their health information with providers when 

needed.2

In a 2016 report to Congress, ONC noted that patients’ ability to 

electronically access their health information has increased over time, 

though there are limitations to this access.3 ONC reports that in 2012, 

only one-quarter of hospitals provided patients with the ability to 

 1Department of Health and Human Services, Office of the National Coordinator for Health 

Information Technology, Connecting Health and Care for the Nation: A Shared Nationwide 

Interoperability Roadmap (FINAL Ver. 1.0) (October 2015). A fundamental component of 

ONC’s goal is the widespread ability of patients to electronically view, download, and 

transmit their health information to a third party (which we refer to collectively as 

“accessing health information” in this report).

2Department of Health and Human Services, Office of the National Coordinator for Health 

Information Technology, 2016 Report to Congress on Health IT Progress: Examining the 

HITECH Era and the Future of Health IT (November 2016).

3Department of Health and Human Services, Office of the National Coordinator for Health 

Information Technology, 2016 Report to Congress on Health IT Progress: Examining the 

HITECH Era and the Future of Health IT (November 2016).

Letter
Page 2 GAO-17-305 Health Information Technology

electronically view their information; in 2015, 95 percent of hospitals 

offered this ability.4 However, ONC has reported that electronic health 

information is often spread across multiple providers using disparate 

systems that are not interoperable.5 ONC has reported that this dispersal 

makes it cumbersome for individuals to aggregate their electronic health 

information from different providers into a single longitudinal health 

record.

You asked that we review the status of health IT interoperability from the 

perspective of the patient, specifically focusing on patients’ ability to 

access, control, and contribute to their electronic health information. In 

this report, we

(1) describe the extent and type of electronic access to health information 

currently available to patients, and patients’ views about this access;

(2) describe the extent to which patients electronically access their health 

information, and actions providers reported taking to encourage such 

access; and

(3) evaluate HHS’s efforts to advance patients’ ability to electronically 

access their health information.

To address our first two objectives, we undertook several steps. We 

conducted a literature search to identify articles published between 2014 

and 2015 that describe how patients electronically access their health 

information. We identified and reviewed 51 such articles.6 In addition, we 

analyzed data provided by the Centers for Medicare & Medicaid Services 

(CMS) for the Medicare EHR Incentive Program (Medicare EHR 

Program) for program year 2015 showing the extent to which providers—

hospitals and health care professionals (e.g., physicians)—participating in 

 4J. Henry, Y. Pylypchuk, and V. Patel, “Electronic Capabilities for Patient Engagement 

Among U.S. Non-Federal Acute Care Hospitals: 2012-2015,” ONC Data Brief, no. 38 

Office of the National Coordinator for Health Information Technology (Washington, D.C., 

September 2016).

5When systems are interoperable, they are able to exchange electronic information with 

other systems and process the information without special effort on the part of the user.

6We limited our search to 2014 and 2015 to capture the most recently available articles at 

the time our search was conducted. Databases searched included BIOSIS Previews, 

Embase, MEDLINE, PsycINFO, and SciSearch. We used different combinations of terms 

such as “medical records,” “patient access,” “patient portal,” and “consumer access.”
Page 3 GAO-17-305 Health Information Technology

this program reported offering patients in their hospital or practice the 

ability to electronically access health information and the extent to which 

patients used these features.7 We also supplemented these data with 

other types of data from CMS, ONC, the Health Resources and Services 

Administration (HRSA), and the U.S. Census Bureau to identify the 

characteristics of these providers and patients. We assessed the 

reliability of these data by (1) performing electronic testing of required 

data elements, (2) reviewing existing information about the data and the 

system that produced them, and (3) consulting agency officials who are 

knowledgeable about the data. We determined that these data were 

sufficiently reliable for the purposes of our reporting objectives. (See 

Appendix I for more details of our analysis.) We also interviewed relevant 

experts to learn about patients’ ability to electronically access health 

information. Specifically, we interviewed knowledgeable individuals in the 

area of health IT and patient electronic access from Mathematica Policy 

Research, the American Health Information Management Association, 

Healthwise, the National Association for Trusted Exchange, and the Open 

Notes Foundation, as well as individuals conducting advocacy work in this 

area. We selected these organizations and individuals based on mentions 

in previous interviews and references in literature.

 7Starting in 2011, the Medicare EHR Program provided incentive payments for certain 

eligible providers, including certain hospitals and health care professionals, such as 

physicians, that successfully attested to the Medicare EHR program. Providers 

successfully attested to the program when they demonstrated meaningful use of certified 

EHR technology and met other program requirements established by CMS. Our analysis 

of Medicare EHR Program data examined providers who successfully attested to the 

Medicare EHR Program, who we refer to in this report as participants. While a similar 

program operates under Medicaid, for the purposes of this report, we only examined data 

from the Medicare EHR Program. In program year 2015, there were 194,200 health care 

professionals (e.g., physicians) and 3,218 hospitals that participated in the Medicare EHR 

Program, which is a subset of all Medicare providers eligible to participate in the program. 

CMS estimated in a 2015 final rule that there could be approximately 595,100 nonhospital-based Medicare professionals and approximately 4,900 hospitals eligible for the 

Medicare EHR program in 2015. Therefore, approximately 33 percent of eligible 

professionals and 66 percent of eligible hospitals participated in the Medicare EHR 

Program in 2015.

We analyzed data on two measures related to patient electronic access in the Medicare 

EHR Program for which CMS collects information: (1) the percentage of a provider’s 

patients who were offered the ability to electronically view, download, and transmit their 

health information, and (2) the percentage of a provider’s patients who actually 

electronically viewed, downloaded, or transmitted their health information.
Page 4 GAO-17-305 Health Information Technology

As part of our work for our first two objectives, we also obtained 

information on the perspectives of patients, providers, and health IT 

developers on patient electronic access to health information, as follows.

• To obtain patients’ perspectives, we interviewed a nongeneralizable 

sample of 33 patients to learn about their experiences electronically 

accessing their health information. Patients Like Me, a patient 

advocacy organization, agreed to let us post a request on its website 

seeking to interview patients. The request directed patients to contact 

us directly if they were interested in talking about their experiences 

electronically accessing their health information. We were able to 

successfully schedule and complete interviews with 33 of these 

patients. During these interviews, we asked the patients about 

instances when they electronically accessed their health information, 

challenges they encountered when accessing or attempting to access 

such information, benefits they experienced as a result of such 

access, and improvements they would like to see.

• To obtain providers’ perspectives, we conducted a nationally 

representative survey of 1,867 providers who participated in the

Medicare EHR Program for the 2014 program year. We received valid 

survey responses from 175 hospitals and 253 health care 

professionals. The weighted response rate, which accounts for the 

differential sampling fractions within strata, is 21 percent for the full 

sample, 28 percent for eligible hospitals, and 20 percent for eligible 

professionals. In the survey, we asked providers about the methods 

they use to give their patients electronic access to their health 

information, the perceived benefits of giving patients such access, and 

the steps providers have taken to encourage electronic access by 

their patients, among other things. (See Appendix I for a more 

detailed description of our provider survey.) We also interviewed 6 

providers who volunteered to speak with us after responding to our 

survey to learn about their perspectives on patient access to 

electronic health information. The views of these 6 providers are not 

generalizable to all providers.

• To obtain health IT developers’ perspectives, we interviewed a 

nongeneralizable sample of individuals from 16 companies in the field 

of health IT. We identified these companies through interviews and 

internet searches as companies that are developing or have 

developed products that facilitate patients’ electronic access to health 

information. Specifically, we interviewed 10 health IT developers who 

are working to develop or who have developed applications intended 

to enable consumers of their products to access their electronic health 

information and aggregate it into a single location; representatives 
Page 5 GAO-17-305 Health Information Technology

from 2 EHR vendors that offer portals that providers can purchase so 

that their patients can electronically access their health information; 

representatives from 2 companies that are working on health IT 

projects related to consumer access to health information; and 

representatives from 2 other companies that promote the 

development of health IT platforms that enable consumers to 

electronically access their health information.

To evaluate HHS’s efforts to advance patients’ ability to electronically 

access their health information, we reviewed relevant documents, such as 

strategic plans and agency budget justifications, for both ONC and CMS 

for fiscal years 2015 and 2016, the most recent years available. We 

compared the information and documentation to relevant criteria from 

GAO’s body of work on effectively managing performance under the 

Government Performance and Results Act of 1993.8 We also interviewed 

officials from CMS and ONC about each agency’s efforts to further the 

ability of patients to electronically access their health information and the 

extent to which their agencies are evaluating these efforts.

We conducted this performance audit from September 2015 to March

2017 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

The largest federal investments in health IT and patient electronic access 

to health information are the Medicare and Medicaid EHR Incentive 

Programs. These programs provide incentives to hospitals and health 

care professionals that are able to demonstrate meaningful use of a 

certified EHR system. Providers must attest that they have met certain 

measures in order to receive payment, with the required functions 

increasing in complexity as providers move through the stages of the 

 8See GAO, Executive Guide: Effectively Implementing the Government Performance and 

Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1,1996); GAO, Managing for 

Results: Critical Issues for Improving Federal Agencies’Strategic Plans, 

GAO/GGD-97-180 (Washington, D.C. September 16, 1997).

Background

Medicare and Medicaid 

EHR Incentive Programs 

and Patient Electronic 

Access
Page 6 GAO-17-305 Health Information Technology

program. Among the measures in the current programs are two that are 

specifically designed to capture the extent to which patients are able to 

electronically access their health information. Unless providers claimed 

an exclusion from reporting these measures, providers were required to 

successfully complete them in order to receive incentive payments for 

program year 2015.9 The measures are as follows:

(1) Ability to electronically access health information. More than 50 

percent of a providers’ patients must be provided timely access to 

view online, download, and transmit to a third party their health 

information,10 and

(2) Actual electronic access. At least one of a provider’s patients must 

electronically view, download, or transmit to a third party their 

information during the 90-day reporting period.11

Providers participating in these EHR Programs must use certified EHR 

technology, which is technology that has been determined to conform to 

standards and certification criteria developed by ONC. These criteria do 

not specify a particular technical method for providing patients with 

access to their health information, but do specify parameters for 

accessing certain types of health information. According to ONC, many 

providers use some type of patient portal to provide access to these types 

of health information. A patient portal is a secure online website that gives 

patients 24-hour access to their personal health information and medical 

records from anywhere with an Internet connection. Patient portals are 

 9Providers can claim an exclusion from reporting these measures if they meet any of the 

following criteria: (1) Any eligible professional who neither orders nor creates any of the 

information listed for inclusion as part of both measures, except for “Patient name” and 

“Provider’s name and office contact information,” may exclude both measures. (2) Any 

eligible professional who conducts 50 percent or more of his or her patient encounters or 

any eligible hospital that is located in a county that does not have 50 percent or more of its 

housing units with 4 Mbps broadband availability according to the latest information 

available from the Federal Communications Commission on the first day of the EHR 

reporting period may exclude only the second measure. (3) Providers attesting to the 

program for the first or second year were not required to report the second measure. 

10A hospital or critical access hospital’s patients include all patients who are discharged 

from the inpatient or emergency department during the 90-day reporting period. 

Information must be made available within 36 hours of discharge to be considered timely. 

A professional’s patients include all unique patients seen by the professional during the 

90-day reporting period. Information must be made available within 4 business days of the 

information being made available to the professional, and is subject to the professional’s 

discretion to withhold certain information. 

11This threshold was 5 percent of a provider’s patients for the 2014 program. 
Page 7 GAO-17-305 Health Information Technology

purchased by a provider and generally only include health information 

generated and made available by that individual provider.

ONC released the most recent strategic plan for health IT, developed with 

input from federal and nonfederal stakeholders, in September 2015. This 

plan guides the actions of multiple federal agencies with regard to health 

IT. The plan outlines four primary goals, each with its own objectives for 

using health IT, to improve the health and well-being of individuals and 

communities.12 Two of these goals are to transform health care delivery 

and community health and advance person-centered and self-managed 

care.13 In addition to this strategic plan, in 2015 ONC developed, with 

input from federal and nonfederal stakeholders, a Shared Nationwide 

Interoperability Roadmap (which we refer to in this report as the 

Roadmap).14 The Roadmap proposes specific actions to advance the 

nation towards an interoperable health IT system that collectively 

improves health. The Roadmap includes the goals that patients can 

access their longitudinal electronic health information, contribute to this 

information, send and receive that information through a variety of 

technologies, and use that information to manage their health and 

participate in shared decision making with their health care providers. 

These Roadmap goals support the HHS strategic plan goals of advancing 

care and transforming health care delivery and community health.

Performance measures assess performance via ongoing monitoring and 

reporting of program accomplishments, which include progress toward 

pre-established goals. Our previous work has found that performance 

measures can serve as an early warning system to management and as 

a vehicle for improving accountability to the public.15 We have also 

 12The Office of the National Coordinator for Health Information Technology, Office of the 

Secretary, U.S. Department of Health and Human Services, Federal Health IT Strategic 

Plan 2015-2020, September 21, 2015.

13The other two goals in the strategic plan are to foster research, scientific knowledge, 

and innovation and enhance the nation’s health IT infrastructure.

14The Office of the National Coordinator for Health Information Technology, Connecting 

Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap, FINAL 

Version 1.0, October 6, 2015.

15GAO, Performance Measurement and Evaluation: Definitions and Relationships, 

GAO-11-646SP (Washington, D.C.: May 2, 2011).

Federal Health Information 

Technology Strategic 

Plans

Performance Management 

and Assessment
Page 8 GAO-17-305 Health Information Technology

published guidance on assessing performance which states that it is 

important for performance measures to be tied to program goals and for 

agencies to ensure that their activities support their organizational 

missions and move them closer to accomplishing their strategic goals.16

In addition, our guidance to federal agencies on designing evaluations 

suggests that performance measures should include both process and 

outcome measures. (See table 1.)17 Outcome measures are particularly 

useful in assessing the status of program operations, identifying areas 

that need improvement, and ensuring accountability for end results. 

Furthermore, our guidance on assessing performance notes that leading 

organizations should not only establish performance measures but also 

use information from these performance measures to continuously 

improve processes, identify program priorities, and set improvement 

goals.

Table 1: Description of Process and Outcome Measures

Type of measure Description Example 

Process Addresses the type or level of program activities 

conducted and the direct products or services 

delivered by a program 

The number of participants enrolled in a job training 

program 

Outcome Addresses the results of products and services The percentage of participants who find employment 

after successfully completing the job training program 

Source: GAO. | GAO-17-305

 16GAO/GGD-96-118. 

17GAO, Designing Evaluations: 2012 Revision, GAO-12-208G (Washington, D.C.: January

31, 2012).
Page 9 GAO-17-305 Health Information Technology

Data from CMS show that most patients who received their health care 

from providers participating in the Medicare EHR Program had the ability 

to electronically access their health information. Information from our 

survey of providers and interviews with patients and providers show that 

this access is typically offered through patient portals and the type of 

information offered varies. In interviews, patients described the benefits 

and limitations of accessing their health information electronically.

CMS data show that providers who participated in the 2015 Medicare 

EHR Program reported offering most of their patients the ability to 

electronically access their health information. In 2015, all participating 

hospitals and nearly all participating health care professionals reported 

offering electronic access to health information to 88 and 87 percent of 

their patients respectively, on average.18 This means that the providers 

gave the patients they saw or discharged all of the information necessary 

to electronically view, download, and transmit the patients’ health 

information, such as a website address, a username and password, and 

instructions for logging onto the website.

Our survey of providers who participated in the 2014 Medicare EHR 

Program and interviews with providers further illustrate the extent to 

which providers offered their patients electronic access to their health 

information. Our survey found that nearly all providers routinely provided 

new patients with access to this information (92 percent of health care 

professionals and 91 percent of hospitals). Providers we interviewed also 

 18For program year 2015, all 3,218 participating hospitals reported that they met the 

program requirement to offer electronic access to over 50 percent of their patients. For the 

same program year, 192,258 of the 194,200 participating health care professionals 

reported meeting the requirement. Health care professionals could participate in the 

program without reporting that they offered electronic access if they reported to CMS that 

they neither order nor create relevant information.

Most Patients of 

Providers 

Participating in the 

2015 Medicare EHR 

Program Were 

Offered Electronic 

Access to Their 

Health Information 

and Noted Benefits 

and Limitations of 

Such Access

Almost 90 Percent of 

Patients Receiving Care 

from Providers 

Participating in the 2015 

EHR Program Had 

Electronic Access to Their 

Health Information
Page 10 GAO-17-305 Health Information Technology

described circumstances in which a patient may not have been offered 

access. These circumstances included instances such as in emergency 

care, when offering electronic access may not be appropriate at the point 

of care, or for behavioral health data, when it might not be in the best 

interest of the patient to access the information.

ONC has published information on the extent to which non-federal acutecare hospitals and office-based physicians provide their patients with 

access to their health information. ONC reported that in 2015, almost all 

hospitals (95 percent) offered patients the ability to electronically view 

their health information, and about 7 out of 10 (69 percent) hospitals 

provided their patients with the ability to view, download, and transmit 

their health information electronically.19 In addition, ONC recently reported 

that among office-based physicians, 63 percent provided patients with the 

ability to electronically view their health information.20 ONC reported in 

another data brief that in 2014, nearly 4 in 10 Americans were offered 

electronic access to their medical records.21

 19J. Henry, Y. Pylypchuk, and V Patel, “Electronic Capabilities for Patients among U.S. 

Non-Federal Acute Care Hospitals: 2012-2015.” ONC Data Brief, no.38. Office of the 

National Coordinator for Health Information Technology (Washington, D.C., Sept. 2016).

20Office of the National Coordinator for Health Information Technology, “Office-based 

Physician Electronic Patient Engagement Capabilities,” Health IT Quick-Stat, no. 54. 

(Washington, D.C.: December 2016), accessed February 2017, 

https://dashboard.healthit.gov/quickstats/pages/physicians-view-download-transmitsecure-messaging-patient-engagement.php.

21V. Patel, W. Barker and E Siminerio, “Trends in Consumer Access and Use of Electronic 

Health.” ONC Data Brief, no.30. Office of the National Coordinator for Health Information 

Technology (Washington, D.C., Oct., 2015).
Page 11 GAO-17-305 Health Information Technology

Our survey of providers who participated in the 2014 Medicare EHR 

Program found that most providers offered electronic access to patients 

through a patient portal, and our interviews suggest that patients often 

received access to a different portal for each provider.22 A patient portal is 

a secure website that allows patients to access information contained in 

their provider’s EHR system and is managed by the provider. EHR 

vendors and providers we interviewed noted that patients generally have 

to manage separate login information for each provider-specific portal. 

Many patients we interviewed confirmed this; for example, a number of 

patients we interviewed said that they had access to more than one 

portal, each of which contained their health information from a different 

care setting (e.g., hospital stays, general practitioner, and different 

specialists).

The types of health information that providers made available to patients 

varied, but our survey of providers indicated that most routinely offer 

access to most types of patient information. The Medicare EHR Program 

requires that participants make certain types of information available, 

such as laboratory test results and current medications.23 According to 

our survey, an estimated 94 percent of hospitals and 77 percent of health 

care professionals routinely offered access to laboratory test results, 

which are required by the program.24 (See table 2.) Our survey also 

showed that fewer providers routinely offered access to certain 

information that is not required by the program and that they find less 

helpful for their patients to view. For example, 46 percent of hospitals and 

54 percent of health care professionals reported routinely offering access 

to clinician notes, which are not required by the program. Additionally, our 

survey showed that fewer providers find it helpful for patients to view 

22Information is from the 2016 GAO survey of 1,867 hospitals and health care 

professionals in the United States that participated in the Medicare EHR Program for the 

2014 program year. This information is based on the data from the valid survey responses 

we received from 175 hospitals and 253 health care professionals.

23To meet the requirements of the Medicare EHR Program’s Consumer Electronic Access 

measures, health care professionals and hospitals must make information such as the 

following available to patients: current and past problems list, procedures, laboratory test 

results, current medication list, medication history, current medication allergy list, 

medication allergy history, vital signs, and care plan field(s) including goals and 

instructions. In circumstances where there is no information available to populate one or 

more of the fields previously listed, the provider may indicate that the information is not 

available and still meet the measures. 

24All estimates from this survey were calculated with a 95 percent confidence level and 

have a margin of error of plus or minus 7 percentage points or fewer.

Providers Who 

Participated in the 2014 

EHR Program Generally 

Used Patient Portals to 

Make Various Types of 

Health Information 

Available to Patients
Page 12 GAO-17-305 Health Information Technology

clinician notes and radiological images than for patients to view 

information such as laboratory results and current medications. 

Representatives from two hospitals we interviewed explained that their 

hospitals relied on a committee to decide what information to make 

available to patients through the portal and how soon after it is available 

to the provider to make it available to the patient. One EHR vendor we 

spoke with noted that the vendor automatically makes almost all 

information, including clinician notes, available to patients through the 

vendor’s patient portal by default, though many vendors allow providers to 

limit the types of information they routinely make available to patients.
Page 13 GAO-17-305 Health Information Technology

Table 2: Estimated Percentages of Providers Who Participated in the 2014 Medicare EHR Program that Routinely Offer 

Various Types of Health Information through Patient Portals

Type of Health 

Information

Required by the 

Medicare EHR program 

to be electronically 

available to patientsa

Providers

reporting that they offered

information when availableb

Providers

reporting that it is helpful for

patients to view the informationc

Hospitals 

Health Care 

Professionals Hospitals

Health Care 

Professionals

Allergies Yes 92% 81% 84% 70%

Clinical history Yes 83% 77% 76% 71%

Clinician notes No 46% 54% 51% 42%

Current medications Yes 92% 82% 87% 83%

Immunization history No 82% 67% 81% 73%

Laboratory test results Yes 94% 77% 87% 80%

Problems/conditions Yes 91% 80% N/A N/Ad

Radiology images No 13% 31% 25% 27%

Radiology result reports No 79% 65% 77% 65%

After-visit summary Yes 85% 77% 81% 78%

Source: GAO survey of Medicare EHR Program participants. | GAO-17-305

Notes: Data w ere from the 2016 GAO survey of hospitals and health care professionals in the United 

States that participated in the Medicare Electronic Health Record (EHR) Program for the 2014 

program year. Percentages are based on the data from the valid survey responses GAO received 

from 175 hospitals and 253 health care professionals.

All estimates in this table have been calculated using the appropriate sampling w eights and survey 

design variables and have a margin of error at the 95 percent confidence level of plus or minus 7 

percentage points or few er.

a

To meet the requirements of the Medicare EHR Program’s Patient Electronic Access measures, 

hospitals and health care professionals must make information such as the follow ing available to 

patients: current and past problems list, procedures, laboratory test results, current medication list, 

medication history, current medication allergy list, medication allergy history, vital signs, and care plan 

field(s) including goals and instructions. In circumstances where there is no information available to 

populate one or more of the fields previously listed, the provider may indicate that the information is 

not available and still meet the measures.

b

These columns contain the percentage of providers who answered “yes” to the survey question “in 

the past 6 months, have you or your hospital routinely entered the follow ing health information in your 

[patient health portal] (w hen available for the patient)?”

c

These columns contain the portion of providers w ho answered “yes” to the survey question “do you

or your hospital find it helpful for patients to electronically view the following types of health 

information via patient health portals?”

d

This question w as not included on the survey.

Patients we interviewed said that the type of information made available 

in their portals was incomplete and inconsistent across providers. Though 

many patients talked about accessing their lab results through their portal, 

multiple patients said that their results were not always available for them 

to view. For example, one patient said that sometimes her lab results are 

posted, and other times they are never made available to her, and she 
Page 14 GAO-17-305 Health Information Technology

does not have a sense of when the results will be made available. Three 

patients expressed frustration that their vital signs information such as 

weight and blood pressure was not available through their portals, 

particularly since they knew that their providers collected this information 

during visits. Another patient said that she has observed a lot of variability 

in what information providers make available through their portals, with 

some doctors providing access to detailed information such as clinical 

notes and lab results, and others only making basic information available, 

such as appointment reminders and vital signs information collected 

during visits.

During our interviews with 33 patients who have accessed their health 

information electronically, patients described numerous benefits from the 

ability to electronically access their health information. These benefits 

included the ability to communicate better with their health care providers, 

track health information over time, and share information with other 

providers. Multiple patients described circumstances in which they used 

information in their portal to improve their interactions with their provider 

and adhere to provider recommendations. For example, one patient 

described how he logged into the portal after a visit to review instructions 

from his provider that he had forgotten. Patients also noted that 

electronically accessing their health information made them feel 

empowered or more proactive to manage their health, particularly over 

time. For example, patients described using their electronic access to 

view specific test results over time to see whether their condition was 

changing, or to access diagnostic information that gave them the ability to 

do more research on their medical condition. One patient described using 

the information in her portal to notice a trend in her lab results and also 

learn of a condition she had of which her provider failed to inform her. 

Patients also described using their patient portals to share information 

with other providers. Multiple patients described printing out medical 

information from their portals, such as lab results, and bringing that 

information to appointments with other providers. Patients noted that 

portals make sharing health information very convenient.

However, patients also described some limitations with their access, 

many of which were related to the functionality of the portal. Patients we 

interviewed stated that they were able to view their health information 

electronically, but many patients said that it was not clear that this 

information could be electronically downloaded or transmitted. Patients 

also expressed frustration with the amount of time and effort it took to set 

up electronic access through their providers, managing multiple 

Patients Described 

Numerous Benefits and 

Some Limitations with 

Their Electronic Access to 

Health Information
Page 15 GAO-17-305 Health Information Technology

passwords for their many portals, and understanding each portal’s user 

interface. Many patients said that the information itself was often incorrect 

or not presented in helpful ways, and some patients noted that there was 

no simple way to correct or denote incorrect information within the portal. 

For example, one patient said that another person’s information was 

included in her record, and it took multiple requests to her provider to 

remove this information from her record. Another patient was frustrated 

that information about his weight that was captured in his yearly physical 

was not available in the portal in a way that would allow him to track his 

weight over time.

Multiple patients said that an overall limitation is that they could not 

aggregate all of their health information into a single longitudinal health 

record. While there are health IT products available to help patients and 

providers aggregate information, they are not in high demand. For 

example

• patient-purchased personal health records (PHR) can enable patients 

to aggregate electronic information from disparate sources into a 

single record. Health IT developers said that there are PHR products 

available for patients who attempt to generate such a record, but our 

survey and patient interviews indicate that these products are not 

widely used. Health IT developers noted that these products have 

limited functionality because they or the users (e.g., patients) cannot 

access information stored in EHR systems, and one developer noted 

that a lack of standardization limits the ability to present information in 

a meaningful way.25 Patients we interviewed generally stated that they 

were not using these products, and health IT developers agreed that 

consumer demand is low. Additionally, relatively few hospitals and 

health care professionals we surveyed reported having the capability 

to submit information to PHR products.

• Provider-purchased products can also help patients and providers 

aggregate longitudinal health information, according to health IT 

product developers and EHR vendors we interviewed. For example, 

one health IT developer explained that providers can currently 

purchase their product to display information from multiple EHR 

systems in a single portal; this product would need to be purchased 

 25For example, health IT developers stated that the lack of open application programming 

interfaces means they cannot access information stored in EHR systems. Application 

programming interfaces are technology that allows one software program to access the 

services or information provided by another software program.
Page 16 GAO-17-305 Health Information Technology

separately from the EHR system and would require additional 

configuration. However, according to our survey, we estimate that 

most providers offer patient portals that are packaged with their EHR 

system. One EHR vendor representative said that the company was 

currently in the process of developing a product that will enable 

patients to access information from multiple providers using their EHR 

system. However, that product has not been released for provider and 

patient use.

Few Patients of 

Providers 

Participating in the 

Medicare EHR 

Program 

Electronically 

Accessed Their 

Health Information, 

and Providers 

Reported Taking 

Steps to Encourage 

Access
Page 17 GAO-17-305 Health Information Technology

Providers participating in the Medicare EHR Program in 2015 reported 

that relatively few patients electronically accessed their health information 

when it was made available to them.26 In other words, few of these 

patients logged into a patient portal and viewed, downloaded, or 

transmitted their health information. Our analysis of 2015 Medicare EHR 

Program data collected by CMS showed that among participating 

hospitals, 15 percent of their patients electronically accessed their 

available health information; among physicians and other health care 

professionals, this percentage was twice as much, with about a third (30 

percent) of their patients accessing their available health information. 

(See fig. 1.)

Figure 1: Average Percentage of Patients of 2015 Medicare EHR Program Participating Providers Who Were Offered Access 

and Electronically Accessed Available Health Information

Notes: We analyzed data that HHS collected from providers for the 2015 Medicare Electronic Health 

Record (EHR) Program. We refer to providers who successfully attested to the program in a given 

year as “participants.” This figure reports data from the tw o program measures related to patient 

electronic access to health information; in 2015, 192,258 health care professionals and all 3,218 

hospitals reported offering electronic access to health information to their patients, and 176,474 

professionals and 2,985 hospitals reported that one or more of their patients actually accessed their 

health information. We calculated the percentage of patients w ho accessed their available information 

by dividing the number of patients w ho accessed their information by the number of patients w ho 

w ere offered access for each provider.

 26The participating providers we refer to are the 194,200 health care professionals (e.g., 

physicians) and 3,218 hospitals that participated in the Medicare EHR program for 

program year 2015. Providers could participate in the program without meeting measures 

related to consumer electronic access if they claimed exclusion due to certain 

characteristics, such as limited broadband availability in their community. 

Less than One-third of 

Patients Electronically 

Accessed Their Health 

Information, and Patients 

Reported Such Access 

Generally Occurs Before 

or After Seeing a Provider
Page 18 GAO-17-305 Health Information Technology

Examining access rates by provider characteristics, our analysis shows 

that some types of non-hospital based providers reported relatively low 

percentages of patients accessing their health information electronically in 

2015. Analyzing 2015 Medicare EHR Program data supplemented with 

other HHS data, we found that among non-hospital based providers 

participating in the 2015 program:

• health care professionals located in areas with a higher (i.e., above 

the national median) percentage of residents in poverty and located in 

rural areas reported lower levels of electronic access to health 

information, compared with professionals in lower-poverty areas and 

professionals located in urban areas;

• health care professionals with 50 or fewer group practice members 

reported notably lower levels of electronic access to health 

information compared with professionals with larger numbers of group 

practice members; and

• health care professionals other than general or specialty 

practitioners—including chiropractors, dentists, and podiatrists—

reported notably lower levels of electronic access to health 

information compared with professionals in general practice or 

specialty practice. (See fig. 2.)
Page 19 GAO-17-305 Health Information Technology

Figure 2: Average Percentage of Patients of 2015 Medicare EHR Program Health Care Professionals Who Electronically 

Accessed Available Health Information, by Provider Characteristic

Note: We analyzed data that HHS collected from health care professionals (e.g., physicians) for the 

2015 Medicare Electronic Health Record (EHR) Program. We refer to providers who successfully 

attested to the program in a given year as “participants.” We analyzed tw o program measures related 

to patient electronic access to health information in conjunction w ith HHS’s Physician Compare file, 

the 2015-2016 Area Health Resources File, and information from the U.S. Census Bureau. The 

number in parentheses next to each characteristic represents the percentage of the health care 

professionals that falls into the category; these numbers may not sum to 100 because of rounding.

We did not control for any other characteristics when computing the average for each category.

a

We categorized a provider as practicing in a higher poverty area if they w ere located in a county w ith 

higher than the median percentage of the population living under the poverty line, according to HHS’s 

2015-2016 Area Health Resources File.

b

”Other” provider types include chiropractors, podiatrists, and dentists.

c

We used Medicare EHR Program data to calculate the percentage of each health care professional’s 

patients w ho accessed their available information by dividing the reported number of patients w ho 

accessed their information by the reported number of patients w ho were offered access.

Examining access rates by age, our analysis of data from the 2015 

Medicare EHR Program and data from HRSA’s Area Health Resources 

File indicates that the level of electronic access to health information 

reported by both hospitals and health care professionals was lower 
Page 20 GAO-17-305 Health Information Technology

among those located in areas with a higher percentage of the population 

over age 65. (See fig. 3.)

Figure 3: Average Percentage of 2015 Medicare EHR Program Participants’ Patients Who Electronically Accessed Available 

Health Information, by Percentage of County Residents Over Age 65

Note: We analyzed data that the Centers for Medicare & Medicaid Services collected from hospitals 

and health care professionals (e.g., physicians) for the 2015 Medicare Electronic Health Record 

(EHR) Program. We refer to providers w ho successfully attested to the program in a given year as 

“participants.” We analyzed tw o program measures related to patient electronic access to health 

information in conjunction w ith the Health Resources and Services Administration’s 2015-2016 Area 

Health Resources File. We did not control for any other provider characteristics when computing the 

average for each category.

a

We used Medicare EHR Program data to calculate the percentage of patients w ho accessed their 

available information for each provider by dividing the number of patients w ho accessed their 

information by the number of patients w ho were offered access.

b

We categorized a provider as practicing in an area w ith a higher population over age 65 if they w ere 

located in a county w ith higher than the median percentage of the population over age 65, according 

to the Health Resources and Services Administration’s 2015-2016 Area Health Resources File.

The findings from our analysis of access rates by patient age are 

consistent with other evidence suggesting that older patients may be less 

likely to access their health information electronically compared with 

younger patients. Providers we interviewed and who responded to our 

survey, as well as health IT developers we interviewed, said that a 

patient’s age affected the extent to which she electronically accesses her

health information. Multiple providers who responded to our survey and 

that we interviewed conveyed that, in their experience, older patients are 

less likely to electronically access their information. Providers and health 

IT developers noted that younger patients and those with chronic 

conditions are most likely to want electronic access to their health 

information.
Page 21 GAO-17-305 Health Information Technology

Some providers we surveyed and interviewed attributed the lack of 

interest in accessing health information electronically among older 

patients to a decreased likelihood of having access to a computer or webenabled device. One provider stated that his hospital serves a large 

elderly population and that this was the biggest challenge to meeting the 

requirement under the 2014 Medicare EHR Program that over 5 percent 

of patients access their health information electronically. A recent data 

analysis by ONC found no differences in rates of access to or the viewing 

of online medical records by age, but the analysis did find that individuals 

between the ages of 50 and 59 had significantly higher rates of 

electronically communicating with health care providers, looking up test 

results online, and using smartphone health applications compared with 

individuals 70 years or older.27 More generally, a 2013 survey conducted 

by the Pew Research Center found that adults age 65 or older were most 

likely to say that they never go online.28

Age is not the only determinant as to whether patients electronically 

access their health information. According to studies we reviewed, 

patients may not access their health information frequently because they 

do not have a reason to do so. In 2015, ONC reported that for 2013 and 

2014, about three-quarters of surveyed individuals who reported that they 

did not access their medical records online indicated that they did not do 

so because they did not have a need to use the information.29 Similarly, 

another study found that most patients who report rarely or never 

accessing electronic health information say that they do not have a need 

to do so.30

According to patients we interviewed, patients who electronically access 

their health information typically do so before or after a health care 

encounter. For example, patients we interviewed said that they accessed 

information in their portal to review information before or after an 

 27V. Patel, W. Barker, and E. Siminerio, “Disparities in Individuals’ Access and Use of 

Health Information Technology in 2014,” ONC Data Brief No. 34.Office of the National 

Coordinator for Health Information Technology (Washington, D.C., Feb. 2016).

28Pew Research Center, Who’s Not Online and Why. (Washington, D.C.: Sept. 25, 2013).

29V. Patel, W. Barker, and E. Siminerio, “Trends in Consumer Access and Use of 

Electronic Health Information.” ONC Data Brief No. 30. Office of the National Coordinator 

for Health Information Technology (Washington, D.C., Oct. 2015).

30National Partnership for Women and Families. Engaging Patients and Families: How 

Consumers Value and Use Health IT. (Washington, D.C., Dec. 2014).
Page 22 GAO-17-305 Health Information Technology

encounter with a provider—for example, to review lab test results, 

communicate with their providers about a recent appointment, or share 

information between providers during visits. About half of the patients we 

interviewed also described using portals offered by their providers to 

access “convenience features” related to receiving health care, such as 

features used to see appointment reminders, request medication refills, 

message their provider, or schedule an appointment. Similarly, one of the 

studies we reviewed found that consumers expressed preferences to use 

online access to their health information primarily for needs that occur 

before or after a health care encounter (e.g., to view recently completed 

lab work or notes from a recent physician visit) or because they are 

accessing convenience features offered in their provider’s portal, such as 

online appointment scheduling or to request medication refills.31

In our survey of 2014 Medicare EHR Program participants, providers 

reported using a variety of outreach strategies and other efforts to 

encourage their patients to access the health information made available 

to them. These methods include promoting the use of patient portals and 

providing prizes and other incentives to access the portals. (See table 3.)

 31National Partnership for Women and Families. Engaging Patients and Families: How 

Consumers Value and Use Health IT. Washington, D.C.: (December 2014).

Providers Reported Taking 

a Variety of Actions to 

Encourage Patient 

Electronic Access
Page 23 GAO-17-305 Health Information Technology

Table 3: Estimated Percentages of Methods Used by Providers Who Participated in the 2014 Medicare EHR Program to 

Encourage Patients to Electronically Access Their Health Information

Method

Estimated percentage of 

hospitals using method

Estimated percentage of health 

care professionals using method

Educational brochures or other written materials about the 

patient health portal 96% 78%

Steps taken to improve patient health portal usability or design 64% 66%

Promoting the patient health portal at every patient interaction 64% 79%

Provide access to computers in hospital or practice 58% 46%

Reminder emails 55% 58%

Incentives given to clinical staff 7% 14%

Raffle giveaways/prizes for patients 6% 10%

Discounts on procedures 2% 3%

Other 2% 1%

Source: GAO survey of Medicare EHR Program participants. | GAO-17-305

Notes: Data w ere from the 2016 GAO survey of hospitals and health care professionals in the United 

States that successfully attested to the Medicare Electronic Health Record (EHR) Program for the 

2014 program year. Percentages are based on the data from the valid survey responses we received 

from 175 hospitals and 253 health care professionals.

All estimates in this table have been calculated using the appropriate sampling w eights and survey 

design variables and have a margin of error at the 95 percent confidence level of plus or minus 7 

percentage points or few er.

Providers we interviewed similarly reported undertaking a variety of 

efforts to encourage patients to electronically access their health 

information. For example, a hospital representative stated that to increase 

patient access, staff members tell patients about the portal and take steps 

to register patients for the portal at every interaction. Another hospital 

representative explained that hospital staff individually assist patients and 

even help patients obtain a private e-mail address to register for the 

portal, if necessary. Yet another hospital representative said that the 

hospital staff wore buttons instructing patients to ask staff about the 

portal, and the hospital also installed billboards to remind patients to ask 

staff about the portal. Despite these efforts, this hospital representative 

said that they struggled to meet the patient electronic access 

requirements under the Medicare EHR program.

Our interviews with the 33 patients and analysis of Medicare EHR 

program data and our survey data indicate that the type of portal that 

providers use may influence the extent to which patients access their 

available health information. In particular, patients we interviewed noted 

that they sometimes experienced technical difficulties when attempting to 

access information through the portal or were confused by the portal’s 
Page 24 GAO-17-305 Health Information Technology

user interfaces. For example, patients noted that they were sometimes 

unable to access information in their portals due to the sites being down 

for maintenance or that their portals were not optimized for viewing on a 

mobile device, which limited their ability to use the portal. Several patients 

also expressed frustration with the user interface of the portal offered by 

their providers, noting that it was difficult to navigate and find the 

information they wanted. About two-thirds of the providers we surveyed 

reported taking steps to improve their patient portal’s usability or design. 

Our provider survey data indicate that most providers offer patient portals 

that are packaged with their EHR system and therefore provided by the 

same vendor. One vendor we interviewed noted that it allows for some 

customization for each customer. We viewed demonstrations of three 

EHR systems’ patient portals, and observed that the portal design does 

vary by vendor. For example, the portals we viewed had differences in 

their interfaces, including where to access health information and how 

tabs were labeled. Our analysis of Medicare EHR Program data from 

ONC and CMS confirms that the type of portal itself may affect the extent

to which patients access their available health information; the average 

percentage of patients that accessed their available health information 

varied depending on the provider’s reported EHR vendor. (See figs. 4 

and 5.)
Page 25 GAO-17-305 Health Information Technology

Figure 4: Average Percentage of 2015 Medicare EHR Program Health Care Professionals’ Patients Who Electronically 

Accessed Available Health Information, by EHR System Vendor

Note: We analyzed data that HHS collected from health care professionals (e.g., physicians) for the 

2015 Medicare Electronic Health Record (EHR) Program. We refer to providers who successfully 

attested to the program in a given year as “participants.” We analyzed tw o program measures related 

to patient electronic access to health information in conjunction w ith data on the vendor of the 

certified EHR system(s) used to support a provider’s participation in the 2015 program. We limited our 

analysis to the 176,998 health care professionals who reported using a single vendor to support their 

participation. We did not control for any other provider characteristics when computing the average 

for each vendor.

a

These EHR vendors represent the top 10 most frequently used EHR vendors reported by health care 

professionals using a single vendor for the 2015 Medicare EHR Program.

b

We used 2015 Medicare EHR Program data to calculate the percentage of patients w ho accessed 

their available information for each provider by dividing the number of patients w ho accessed their 

information by the number of patients w ho were offered access.
Page 26 GAO-17-305 Health Information Technology

Figure 5: Average Percentage of 2015 Medicare EHR Program Hospitals’ Patients Who Electronically Accessed Available 

Health Information, by EHR System Vendor

Note: We analyzed data that HHS collected from hospitals for the 2015 Medicare Electronic Health 

Record (EHR) Program. We refer to providers who successfully attested to the program in a given 

year as “participants.” We analyzed tw o program measures related to patient electronic access to 

health information in conjunction w ith data on the vendor of the certified EHR system(s) used to 

support a provider’s participation in the 2015 program. We limited our analysis to the 2,188 hospitals 

w ho reported using a single vendor to support their participation. We did not control for any other 

provider characteristics when computing the average for each vendor.

a

These EHR vendors represent the top 5 most-reported EHR vendors by hospitals using a single 

vendor for the 2015 Medicare EHR Program.

b

We used 2015 Medicare EHR Program data to calculate the percentage of patients w ho accessed 

their available information for each provider by dividing the number of patients w ho accessed their 

information by the number of patients w ho were offered access.

HHS Has Taken 

Steps to Increase 

Patients’ Electronic 

Access to Health 

Information but Lacks 

Outcome Measures 

to Determine the 

Effectiveness of 

These Efforts
Page 27 GAO-17-305 Health Information Technology

HHS officials said that two agencies, CMS and ONC, have programs or 

other efforts aimed at increasing the ability of patients to electronically 

access their health information, including the ability to access longitudinal 

health information and aggregate it in a single location. In the case of 

CMS, agency officials told us that the Medicare and Medicaid EHR 

Programs have made a significant contribution towards achieving these 

goals. The two programs require participating hospitals and health care 

professionals to provide electronic access to health information to a 

specified portion of patients.32 According to CMS officials, the programs 

support HHS’s strategic goals to improve health care through the 

meaningful use of health information technology.33

In the case of ONC, agency officials identified multiple efforts they are 

undertaking to increase patients’ ability to electronically access their 

health information, including longitudinal health information. Some 

examples of these efforts include the following:

• Patient Engagement Playbook. The playbook is a tool developed by 

ONC to assist providers in engaging patients with health IT by, for 

example, using patient portals to engage patients in their health and 

care.

• Blue Button Initiative. This initiative includes three distinct efforts – a 

connector, a voluntary pledge program, and a research project. The 

connector is a website that helps patients locate their health 

information online and assists in the development of apps and tools to 

help consumers understand their health information. The voluntary 

pledge encourages public and private organizations—such as 

providers, hospitals, technology companies, and non-profit 

organizations—to commit to making health information available to 

patients electronically and to encourage patient access. The research 

effort is designed to understand the unmet needs and challenges 

facing stakeholders.

 32In 2015 providers participating in the Medicare EHR Program had to offer at least 50 

percent of their patients online access to health information (specifically, the ability to 

view, download, and transmit health information) in order to successfully attest to the 

program. 

33One of the goals in HHS’s Federal Health IT Strategic Plan is to “Transform Health Care 

Delivery and Community Health.” CMS officials said that of the objectives under the goal, 

“improve health care quality, access, and experience through safe, timely, effective, 

efficient, equitable, and person-centered care,” is the objective that supports their work 

with the EHR Programs. 

HHS Programs and Other 

Efforts Aim to Further 

Patients’Ability to Access 

Health Information 

Electronically, Including 

Longitudinal Health 

Information
Page 28 GAO-17-305 Health Information Technology

• Health IT Certification Standards and Certification Criteria. These 

standards and criteria identify certain vocabularies and structured 

formats that must be included in certified EHR systems and other 

EHR technology that providers are required to use in order to 

participate in the EHR programs.

• Consumer Health Data Aggregator Challenge. By awarding private 

sector innovation, this challenge aims to spur the development of 

third-party consumer-facing applications that use open, standardized 

application programming interfaces to help consumers aggregate their 

data in one place.

(See Appendix II for a list of ONC’s programs and efforts most directly 

related to increasing patients’ ability to electronically access their health 

information.)

Both CMS and ONC officials told us that their efforts aim to increase the 

extent to which patients can electronically access their health information. 

Officials said that their efforts are guided by goals such as the Roadmap’s 

long-term milestone of enabling patients to access longitudinal health 

information, contribute to their electronic health information (e.g., send 

data from wearable devices to their electronic health record), and direct 

their health information into any location of their choice (e.g., to a PHR 

application purchased by the patient that aggregates all their health 

information in a single location). According to ONC officials, the agency’s 

efforts all support HHS’s Federal Health IT Strategic Plan as well as 

ONC’s Roadmap, which establishes several milestones for the agency’s 

ongoing efforts to increase patients’ ability to access their health 

information electronically. These milestones are the following:

1) a majority of individuals are able to securely access their electronic 

health information and direct it to the destination of their choice (to be 

achieved between 2015 and 2017);

2) individuals regularly access and contribute to their longitudinal 

electronic health information via health IT, send and receive that 

information through a variety of emerging technologies, and use that 

information to manage their health and participate in shared decisionmaking with their care, support, and service teams (to be achieved 

between 2018 and 2020); and

3) individuals are able to seamlessly integrate and compile longitudinal 

electronic health information across online tools, mobile platforms, 

and devices to participate in shared decision-making with their care, 

support, and service teams (to be achieved between 2021 and 2024).
Page 29 GAO-17-305 Health Information Technology

HHS does not have information on the effectiveness of CMS’s and ONC’s 

efforts to increase the ability of patients to access their health information 

electronically. Although ONC measures some progress related to these 

efforts and the Medicare EHR Program, ONC does not directly measure 

the impact of these efforts on increasing patients’ electronic access to 

health information. In the case of CMS, officials told us that while they 

track the number of providers that participate in the Medicare EHR 

Program, the agency does not directly measure the extent to which the

program specifically affects patients’ ability to access their health 

information electronically. However, HHS officials stated that they do 

monitor the program by seeking public comments during the rulemaking 

process and by publicly reporting statistics. Officials told us that ONC 

collaborates with CMS to monitor and review the EHR Programs and has

used the results of these analyses to modify the programs over time. 

ONC officials told us there is a data use agreement in place that allows 

ONC to analyze Medicare EHR Program data. Additionally, ONC 

commissions evaluations of programs initiated under the HITECH Act, 

including the Medicare EHR Program. While ONC’s data analyses and 

commissioned evaluation provide information concerning patient access 

to electronic health information and patient engagement, these efforts do 

not measure the impact of the Medicare EHR Program on patients’ ability 

to access their health information electronically.

In the case of ONC, ONC measures a range of outcomes associated with 

its multiple efforts, but the office does not measure the extent to which its 

individual efforts are having an effect on patients’ ability to access their 

health information electronically—by determining, for example, if 

providers that participate in these initiatives have higher rates of patient 

access. ONC officials stated that they use metrics as a means of 

assessing whether the technologies and resources made available 

through ONC’s efforts are being utilized. For example, ONC officials told 

us they count the number of website visits to the Patient Engagement 

Playbook page, the number of providers and other stakeholders who 

have pledged to make electronic health information available to their 

patients through ONC’s Blue Button Initiative, and the number of times 

patients access educational videos about their right to access their health 

information online.34

 34ONC officials told us that, based on a recent assessment of the Blue Button Initiative, 

they have plans to develop outcome measures for this effort. However, these officials 

were unable to provide us with details regarding the specific outcome measures they are 

considering or how they might be implemented or assessed. 

HHS Cannot Determine 

Whether Its Agencies’

Efforts Are Helping 

Increase Patients’

Electronic Access to 

Health Information
Page 30 GAO-17-305 Health Information Technology

According to officials, ONC also uses nationally representative surveys of 

hospitals, other providers, and patients that are fielded by various 

organizations to measure the extent to which patients access their health 

information electronically; however, the surveys cannot be used to 

measure whether, or to what extent, ONC’s efforts most directly related to 

patient access are achieving their intended effects.35 ONC’s survey data 

identify, for example, how many patients reported being able to 

electronically view, download, or transmit their health information as well 

as if patients sent their health information to an app, mobile device, or 

PHR. The survey of patients provides information on how patients are 

accessing their health information and what they do with that information 

once accessed. For example, the survey asks patients whether they have 

attempted to electronically send their health information to another 

electronic location such as a PHR application. Finally, the survey also 

asks patients about the extent to which they experience any challenges 

when electronically accessing their health information. ONC officials told 

us that they plan to conduct a consumer survey with different questions in 

2017; however, ONC has not finalized the questions for this survey.

According to ONC officials, these surveys help the agency understand 

other factors, such as how broadband access and language influence 

patient access and whether progress is being made generally towards the 

Roadmap goal of increasing patients’ ability to access their health 

information electronically.

HHS lacks information on the effectiveness of CMS’s and ONC’s efforts 

because it has not developed outcome measures. For example, ONC 

cannot determine if patient electronic access is higher for participants in 

the Blue Button Initiative compared with non-participants or if providers 

who use the Patient Engagement Playbook achieve more patient 

electronic access than non-users. In our prior work we have identified the 

use of outcome measures as a leading principle for measuring 

performance.36 Guidance for federal agencies based on these principles 

calls for federal agencies to include outcome measures that address the 

 35ONC used information from the following surveys: 1) an annual mail or secure online 

site survey of hospitals, conducted by the American Hospital Association since 2008; 2) 

an annual mail/phone survey of office-based physicians, conducted by the Centers for 

Disease Control and Prevention’s National Center for Health Statistics from 2008-2012; 

and 3) a consumer phone survey, conducted by MITRE and NORC at the University of 

Chicago that was fielded in 2012, 2013, and 2014. 

36See GAO, Executive Guide: Effectively Implementing the Government Performance and 

Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1,1996). 
Page 31 GAO-17-305 Health Information Technology

status of program operations, identify areas that need improvement, 

ensure accountability for end results, and measure progress towards 

agency strategic goals—in this case, HHS’s goals related to increasing 

patients’ ability to access their health information electronically. Without 

outcome-focused performance measures, HHS cannot determine 

whether, or to what extent, each of its efforts are contributing to the 

department’s overall goals, or if these efforts need to be modified in any 

way.

Through CMS’s Medicare and Medicaid EHR Programs and ONC’s 

multiple individual initiatives, HHS supports a wide range of efforts 

intended to increase patients’ electronic access to their health

information. HHS’s investment in these efforts has been significant—

since 2009 HHS has spent over $35 billion on the development and 

adoption of health information technology. CMS’s and ONC’s efforts aim 

to encourage the use of technologies that allow patients to electronically 

access their longitudinal health information, contribute to that information, 

and direct it to any location of their choice. While HHS’s investment in 

health information technology is significant, HHS lacks the ability to 

determine whether, or to what extent, CMS’s and ONC’s efforts are 

helping HHS achieve its goals. ONC is largely responsible for measuring 

the nation’s progress towards increasing patients’ electronic access to 

health information. However, ONC has not developed outcome measures 

to directly measure the effectiveness of its individual efforts, identify areas 

that need improvement, and ensure accountability for achieving results. 

Without such outcome-focused performance measures linked to relevant 

agency goals, ONC—and by extension, HHS—cannot determine 

whether, or to what extent, each of the programs and efforts is 

contributing to overall goals, or if these efforts need to be modified in any 

way.

To help ensure that its efforts to increase patients’ electronic access to 

health information are successful, the Secretary of HHS should direct 

ONC to take two actions.

• First, develop performance measures to assess outcomes of key 

efforts related to patients’ electronic access to longitudinal health 

information. Such actions may include, for example, determining 

whether the number of providers that participate in these initiatives 

have higher rates of patient access to electronic health information.

Conclusions

Recommendations
Page 32 GAO-17-305 Health Information Technology

• Second, use the information these performance measures provide to 

make program adjustments, as appropriate. Such actions may 

include, for example, assessing the status of program operations or 

identifying areas that need improvement in order to help achieve 

program goals related to increasing patients’ ability to access their 

health information electronically.

We provided a draft of this report to HHS for its review and comment. 

HHS provided written comments, which are reprinted in appendix III. HHS 

also provided technical comments, which we incorporated as appropriate. 

In its written comments, HHS concurred with both of our 

recommendations.

With regard to our first recommendation, which calls for HHS to develop 

performance measures to assess the outcomes of key efforts related to 

patients’ electronic access to longitudinal health information, HHS noted 

that ONC is committed to assessing the effects of health IT adoption and 

use. HHS detailed efforts on the part of ONC and CMS to assess 

progress in patients’ access to their electronic health information and said 

that the department has used these assessments to modify its programs 

for encouraging such use over time. HHS stated that there has been an 

increase in patients’ ability to electronically access and use their health 

information and noted that we said this in our report. With regard to our 

statement that ONC is primarily responsible for assessing the effects of 

the Medicare EHR Program, HHS raised concerns that this statement 

was misleading because assessing the impact of the program is a 

coordinated effort between ONC and CMS. In response, we changed our 

description of the roles of ONC and CMS to reflect HHS’s comment. 

While HHS has worked to assess the impact of its efforts, it agreed that 

ONC has not developed a specific means for measuring outcomes 

associated with ONC’s efforts aimed at furthering patients’ ability to 

electronically access their health information. HHS also noted that ONC is 

required by HITECH and the Medicare Access and Children’s Health 

Insurance Program Reauthorization Act to develop performance 

measures for the adoption of EHRs and related efforts to facilitate the 

electronic use and exchange of health information. HHS stated that these 

required performance measures involved nationwide surveys that go 

beyond the scope of the Medicare EHR Program data discussed in this 

report. Therefore, HHS stated that ONC would make every effort to 

develop performance measures for patient education and outreach 

initiatives but would have to balance these efforts with its efforts to 

Agency Comments

and our Evaluation
Page 33 GAO-17-305 Health Information Technology

develop measures for the adoption of EHRs, interoperability, and patient 

engagement nationwide. 

In concurring with our second recommendation, that ONC use the 

information the performance measures provide to make program 

adjustments, HHS stated that it is committed to using performance 

measures to guide program improvement. 

We are sending copies of this report to the appropriate congressional 

committees, the Secretary of Health and Human Services, and other 

interested parties. In addition, the report is available at no charge on the 

GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact 

me at (202) 512-7114 or at yocomc@gao.gov. Contact points for our 

Office of Congressional Relations and Office of Public Affairs can be 

found on the last page of this report. Other major contributors to this 

report are listed in appendix IV.

Carolyn L. Yocom

Director, Health Care
Page 34 GAO-17-305 Health Information Technology

List of Requesters

The Honorable Lamar Alexander 

Chairman

Committee on Health, Education, Labor, and Pensions

United States Senate 

The Honorable John Thune 

Chairman

Committee on Commerce, Science, and Transportation

United States Senate 

The Honorable Michael Enzi 

Chairman

Subcommittee on Primary Health and Retirement Security

Committee on Health, Education, Labor, and Pensions

United States Senate 

The Honorable Richard Burr 

United States Senate 

The Honorable Pat Roberts 

United States Senate 
Appendix I: Methodology for Medicare 

Electronic Health Record Incentive Program 

(Medicare EHR Program) Data Analysis and 

Provider Survey

Page 35 GAO-17-305 Health Information Technology

This appendix provides additional details regarding our analysis of 

Medicare EHR Program data and our nationally representative provider 

survey. The data analysis and the provider survey were used, in part, to 

describe the extent and type of electronic access to health information 

currently available to patients, the extent to which patients electronically 

access their health information, and the actions providers are taking to 

encourage such access.1

We analyzed data from the Centers for Medicare & Medicaid Services 

(CMS) as supplemented with other government data to (1) determine the 

number of providers—that is, hospitals and health care professionals 

(e.g., physicians)—that participated in the 2015 Medicare EHR Program; 

(2) determine the number of program participants who reported each of 

two measures related to patient electronic access to health information; 

(3) determine the extent to which program participants are offering 

patients, and patients are using, the ability to electronically access their 

health information; and (4) examine the characteristics of providers that 

were associated with higher or lower percentages of patients who actually 

accessed their available health information.

We assessed the reliability of these data by (1) performing electronic 

testing of required data elements, (2) reviewing existing information about 

the data and the system that produced them, and (3) consulting agency 

officials who are knowledgeable about these data. We determined that 

these data were sufficiently reliable for the purposes of our reporting 

objectives.

Number of providers that participated in the Medicare EHR Program.

To determine the number of providers that participated in the Medicare 

EHR Program in 2015, we analyzed data extracted from CMS’s National 

Level Repository that represented all successful attestations. CMS 

collected these data from January 2016 to March 2016. We counted the 

number of unique providers that were included in the 2015 program data 

(whom we refer to as “participants”).

Number of program participants who reported two measures related 

to patient electronic access to health information. To determine the 

 1Throughout the report, we use the terms “electronic access” and “access” to collectively 

refer to patients’ ability to electronically view, download, or transmit their health 

information.

Appendix I: Methodology for Medicare 

Electronic Health Record Incentive Program 

(Medicare EHR Program) Data Analysis and 

Provider Survey

Methodology for Analysis 

of Medicare EHR Program 

Data
Appendix I: Methodology for Medicare 

Electronic Health Record Incentive Program 

(Medicare EHR Program) Data Analysis and 

Provider Survey

Page 36 GAO-17-305 Health Information Technology

number of participants who reported two measures related to patient 

electronic access to health information, we counted the number of unique 

providers who reported a number for (1) the percentage of patients who 

were offered the ability to electronically view, download, and transmit their 

health information, and (2) the percentage of patients who actually 

electronically viewed, downloaded, or transmitted their health information.

The extent to which program participants are offering patients—and 

patients are using—the ability to electronically access their health 

information. To determine the extent to which program participants 

offered patients the ability to electronically access their health information, 

we computed the average of the reported percentages of patients who 

were offered the ability to view, download, and transmit their health 

information by their provider. To determine the extent to which program 

participants’ patients actually used the ability to electronically access their 

health information, we computed the average of the reported percentage 

of patients who actually viewed, downloaded, or transmitted their health 

information. To determine the extent to which program participants’

patients actually used the ability to electronically access their health 

information when it was available, we divided the number of patients who 

actually accessed their health information by the number of patients who 

were offered access for each participant, and computed the average.

Characteristics of providers associated with higher or lower 

percentages of patients who actually accessed their available health 

information. To examine the characteristics of providers that were 

associated with higher or lower percentages of patients who actually

accessed their available health information, we analyzed data on provider 

characteristics from CMS, the Office of the National Coordinator for 

Health Information Technology (ONC), the Health Resources and 

Services Administration (HRSA), and the U.S. Census Bureau. Each 

characteristic is divided into two or more categories. For example, the 

characteristic “geographic region” is divided into four categories—

Midwest, Northeast, South, and West regions. As part of this analysis, we 

computed the average percentage of patients who actually accessed their 

available health information for providers within each characteristic 

category without controlling for other characteristics. We examined the 

following provider characteristics:

Regional characteristics. We analyzed data on the following regional 

characteristics using providers’ business zip code:
Appendix I: Methodology for Medicare 

Electronic Health Record Incentive Program 

(Medicare EHR Program) Data Analysis and 

Provider Survey

Page 37 GAO-17-305 Health Information Technology

• Metropolitan status. We used the 2015-2016 HRSA Area Health 

Resources File to determine whether providers were located in a 

metropolitan area—an area that has at least one urbanized area of 

50,000 people, among other criteria. We then categorized providers in 

metropolitan areas as being located in urban areas and providers that 

were not as being in rural areas.

• Geographic region. We used information from the U.S. Census 

Bureau to identify the U.S. census region—Midwest, Northeast, 

South, or West—where providers were located or practiced.

• County residents living in poverty. We used information from the 

HRSA Area Health Resources File to calculate the 2014 national 

median percentage of counties’ residents living under the poverty line. 

We then categorized providers into “higher poverty” areas if they were 

located in a county above the national median percentage of residents 

living in poverty and “lower poverty” areas if they were located in a 

county below or equal to the median.

• County residents over age 65. We used information from the HRSA 

Area Health Resources File to estimate the 2014 national median of 

counties’ percentage of residents over age 65. We then categorized 

providers into “higher population over 65” areas if they were located in 

a county above the national median percentage of residents over age 

65 and “lower population over age 65” areas if they were located in a 

county below or equal to the median.

Hospital type. We analyzed data on the following categorizations of 

hospital type:

• Hospital classification. We determined whether hospitals were 

classified as acute care hospitals or critical access hospitals by using 

data from CMS’s Hospital Compare file.

• Ownership type. We used data on ownership type from CMS’s 

Hospital Compare file to create three categories of ownership: (a) forprofit, (b) nonprofit, and (c) government-owned.

Health care professional characteristics. We analyzed data on the 

following categorizations of professional characteristics:

• Health care professional specialty. We obtained data on 

professionals’ primary specialty from CMS’s Physician Compare file. 

We then consolidated these specialties into the following three 

categories: (a) general practice physician, (b) specialty practice 
Appendix I: Methodology for Medicare 

Electronic Health Record Incentive Program 

(Medicare EHR Program) Data Analysis and 

ProviderSurvey

Page 38 GAO-17-305 Health Information Technology

physician, and (c) other, which includes chiropractors, podiatrists, and 

dentists.

• Number of health care professionals in the practice. We estimated the 

number of professionals in each practice by using data from CMS’s 

Physician Compare file. We subsequently created three practice size 

categories: (a) practice of 1 to 10 professionals, (b) practice of 11 to 

50 professionals, and (c) practice of 51 or more professionals.

We surveyed a nationally representative sample of providers who 

participated in the 2014 Medicare EHR Program about how they are 

providing patients with the ability to electronically access health 

information.

The survey was designed to collect information from providers related to 

patient electronic access to health information, including the methods and 

specific technology providers use to give patients electronic access to 

their health information, the types of health information providers make 

available through these technologies, and any methods providers use to 

encourage patient electronic access. The survey was also designed to 

capture providers’ perspectives on the benefits of patients having such 

electronic access to their health information—specifically whether 

providers saw it beneficial for patients to electronically view, download, or 

transmit certain types of health information.

The target population for this survey was all hospitals and health care 

professionals who reported to the Medicare EHR Program that 5 percent 

or more of their patients viewed, downloaded, or transmitted their health 

information for the 2014 program year.2 Using 2014 program data 

provided to us by CMS, we identified 60,321 hospitals and health care 

professionals to be included in the population for this survey. We selected 

a stratified random sample of 1,867 hospitals and providers as described 

in table 4 below. We stratified the population by type (hospitals and health 

 2To participate in the Medicare EHR Program, providers had to report two measures 

related to patient electronic access or claim an exclusion for these measures.These two 

measures were (1) Ability to electronically access health information—in 2014 and 2015, 

more than 50 percent of a providers’ patients must have their information available 

electronically with the ability to view, download, and transmit to a third party in a timely 

manner (i.e., within 36 hours of discharge for hospitals or within 4 business days that the 

information is available to a health care professional), and (2) Actual electronic access—in 

2014, more than 5 percent of a provider’s patients must electronically view, download, or 

transmit their information. In 2015, one patient must electronically view, download, or 

transmit their information.

Methodology for Provider 

Survey
Appendix I: Methodology for Medicare 

Electronic Health Record Incentive Program 

(Medicare EHR Program) Data Analysis and 

Provider Survey

Page 39 GAO-17-305 Health Information Technology

care professionals) and the reported percentage of patients who 

electronically accessed their health information in 2014. We computed the 

sample sizes separately for hospitals and professionals needed to 

achieve a precision of plus or minus 5 percentage points or fewer at the 

95 percent confidence level. We then increased the sample sizes in each 

group for expected response rates of about 50 and 30 percent for 

hospitals and health care professionals, respectively. (See table 4.)

Table 4: Description of Stratified Sampling Frame and Sample Sizes

Stratum Population size Sample size 

Hospitals - bottom 50 percent of patients who electronically accessed their 

health information 909 315

Hospitals - top 50 percent of patients who electronically accessed their health 

information 908 314

Health care professionals - bottom 50 percent of patients who electronically 

accessed their health information 29,251 619

Health care professionals - top 50 percent of patients who electronically 

accessed their health information 29,253 619

Total 60,321 1,867

Source: GAO Analysis of CMS Data. | GAO-17-305

A link to this web-based survey was emailed to these 1,867 providers via 

the email addresses included in the program data provided by CMS. We 

received valid responses from 428 (23 percent) out of the 1,867 hospitals 

and health care professionals selected in our stratified random sample. 

The weighted response rate, which accounts for the differential sampling 

fractions within strata, is 21 percent for the full sample, 28 percent for 

hospitals, and 20 percent for health care professionals. (See table 5.)

Table 5: Description of Valid Responses

Stratum Sample size 

Valid 

responses

Weighted

response rate

Hospitals - bottom 50 percent of patients who electronically accessed 

their health information 315 74

28%

Hospitals - top 50 percent of patients who electronically accessed their 

health information 314 101

Health care professionals - bottom 50 patients who electronically 

accessed their health information 619 115

20% Health care professionals - Top 50 patients who electronically accessed 

their health information 619 138

Total 1,867 428 21%

Source: GAO Analysis of CMS Data. | GAO-17-305
Appendix I: Methodology for Medicare 

Electronic Health Record Incentive Program 

(Medicare EHR Program) Data Analysis and 

Provider Survey

Page 40 GAO-17-305 Health Information Technology

We conducted an analysis of our survey results to identify potential 

sources of nonresponse bias using two methods. First, we examined the 

response propensity of the sampled hospitals and health care 

professionals by several demographic characteristics. These 

characteristics included region, metropolitan status, specialty type, size of 

practice, hospital type, and hospital ownership type. Second, we 

compared weighted estimates from respondents and nonrespondents to 

known population values for measures that are related to the survey 

outcomes for which we were most interested. We conducted statistical 

tests of differences, at the 95 percent confidence level, between 

estimates and known population values, and between respondents and 

nonrespondents. These analyses were conducted separately for hospitals 

and health care professionals.

Based on this analysis, we did not observe significant differences in 

response propensities or between known population values and 

estimates for nearly all of the characteristics we examined. However, we 

did observe significant differences by ownership type for hospitals and by 

region for health care professionals. Specifically, we found that 

proprietary and physician-owned hospitals were significantly underrepresented by our respondents. Additionally, we found that professionals 

in the Northeast and South were significantly under-represented, while 

professionals in the Midwest and West were over-represented by our 

respondents.

To ensure that the survey results appropriately represented the 

population of 60,321 hospitals and health care professionals, we 

weighted the results from the 428 respondents by the inverse of the 

probability of selection (base weight) and a nonresponse adjustment 

factor to account for nonresponse and the differences in response 

propensities we identified. The nonresponse adjustment factor was 

calculated using weighting class adjustments where adjustment cells 

were based on strata, hospital ownership type, and region.

We repeated the nonresponse bias analysis using the adjusted weights 

and found no significant differences with known population and the 

weighed estimates for all of the characteristics we examined. This 

provided us with evidence that the nonresponse weighting class 

adjustments help mitigate any potential nonresponse bias introduced by 

the differences in response propensities we identified.

Based on the results of this nonresponse bias analysis and the weighting 

adjustments, we determined that weighted estimates generated from 
Appendix I: Methodology for Medicare 

Electronic Health Record Incentive Program 

(Medicare EHR Program) Data Analysis and 

Provider Survey

Page 41 GAO-17-305 Health Information Technology

these survey results are generalizable to the population of hospitals and 

health care professionals and are sufficiently reliable for the purposes of 

this report.

Because we followed a probability procedure based on random 

selections, our sample is only one of a large number of samples that we 

might have drawn. Since each sample could have provided different 

estimates, we express our confidence in the precision of our particular 

sample’s results as a 95 percent confidence interval (e.g., plus or minus 7 

percentage points). This is the interval that would contain the actual 

population value for 95 percent of the samples we could have drawn. As 

a result, we are 95 percent confident that each of the confidence intervals 

in this report will include the true values in the study population.
Appendix II: ONC Efforts to Further Patients’ 

Ability to Access Longitudinal Electronic 

Health Information

Page 42 GAO-17-305 Health Information Technology

Office of the National 

Coordinator for Health 

Information Technology (ONC) 

Effort

Description

Blue Button Pledge Program Voluntary pledge where organizations, such as providers and hospitals, commit to advance efforts 

to increase patient access to and use of their health data.

Blue Button Connector Website to help consumers locate where to find health information online and assist the 

development of apps and tools to helps consumers understand and use their health information.

Blue Button Research Project Research designed to understand the experience of stakeholders surrounding patient access to 

their own health information and use of electronic health information. The research focuses on 

how empathy can help to understand the unmet needs and challenges facing stakeholders.

Consumer Education Video Educational video for consumers about their right to access their health information. 

Intervention to Increase 

Enrollment in Patient Portals

Intervention aimed at increasing the percentage of patients who enroll in online portals to view, 

download, and transmit their health records and communicate online with their clinicians.

Patient Story Collection Four videos that show how health information technology is being used for patient engagement, 

access, and care coordination. 

Strategies for Improving Patient 

Engagement Through Health IT

Interactive document that walks providers through strategies they can use to engage patients with 

the use of health IT. 

Personal Health Record Privacy 

Considerations Tool

Tool that identifies key considerations for adopting health information exchange based on 

personal health records. 

Health Insurance Portability and 

Accountability Act (HIPAA) Access 

Infographic

Infographic regarding an individual’s right to access their medical records (developed and 

published in conjunction with the Department of Health and Human Services (HHS) Office for Civil 

Rights).

Access Videos for Consumers Easy–to–understand videos for consumers, captioned in English and Spanish, about individuals’

right to access their health information under HIPAA, addressing issues including fees and 

requesting that health information be sent to a third party (developed and published in conjunction 

with the HHS Office for Civil Rights).

Examining Oversight of the 

Privacy & Security of Health Data 

Collected by Entities Not 

Regulated by HIPAA

Published report on gaps in legal oversight between the collection of electronic health information 

regulated by HIPAA and not regulated by HIPAA so that consumers can be better aware of the 

privacy and security conditions of how they manage their digital health. 

Application Programming Interface 

(API) Task Force

Task force to identify priority recommendations for ONC that will help enable consumers to 

leverage API technology to access patient data, while ensuring the appropriate level of privacy 

and security protection.

Tech Lab Provides internal and external stakeholders with common connection points to ONC’s standards 

and technology efforts. Tech Lab is organized around four areas: 1) standards coordination; 2) 

testing and utilities; 3) pilots; and 4) innovation. 

Move Health Data Forward 

Challenge

Challenge to stimulate consumer-mediated exchange; will help create API solutions to help 

individuals securely and electronically authorize the movement of health data to destinations of 

their choice. 

Consumer Health Data 

Aggregator Challenge

Challenge to spur the development of third-party consumer-facing applications that use open, 

standardized APIs to help consumers aggregate their data in one place. 

HEART Working Group A working group aimed at developing a set of privacy and security specifications that enable a 

consumer to control the authorization of access to RESTful health-related data sharing APIs, and 

to facilitate the development of interoperable implementations of these specifications by others. 

Appendix II: ONC Efforts to Further Patients’

Ability to Access Longitudinal Electronic 

Health Information
Appendix II: ONC Efforts to Further Patients’ 

Ability to Access Longitudinal Electronic 

Health Information

Page 43 GAO-17-305 Health Information Technology

Office of the National 

Coordinator for Health 

Information Technology (ONC) 

Effort

Description

Patient Engagement Playbook Tool for health care providers, practice staff, and hospital administrators who want to leverage 

health IT using patient portals to engage patients in their health and care. 

Model Privacy Notice Update Update of the 2011 Model Privacy Notice to be more broadly applicable (beyond personal health 

records). According to ONC, this update provides open–source content that technology 

developers can use to notify consumers of their privacy and security practices. 

2015 Edition Health IT 

Certification Criteria Final Rule

The 2015 Edition of ONC’s Health IT Certification Criteria supports the certification of health IT,

including APIs, to support patient access to health data and view, download, and transmit 

functions that continue to support patient access to their health information, including via both 

encrypted and unencrypted email transmission to any third party the patient chooses. 

Interoperability Pledge Commitment from health care industry to make electronic health records work better for 

consumers and providers.

Source: GAO analysis of information provided by ONC. | GAO-17-305
Appendix III: Comments from the Department 

of Health and Human Services

Page 44 GAO-17-305 Health Information Technology

Appendix III: Comments from the 

Department of Health and Human Services
Appendix III: Comments from the Department 

of Health and Human Services

Page 45 GAO-17-305 Health Information Technology
Appendix III: Comments from the Department 

of Health and Human Services

Page 46 GAO-17-305 Health Information Technology
Appendix III: Comments from the Department 

of Health and Human Services

Page 47 GAO-17-305 Health Information Technology
Appendix III: Comments from the Department 

of Health and Human Services

Page 48 GAO-17-305 Health Information Technology
Appendix IV: GAO Contact and Staff 

Acknowledgments

Page 49 GAO-17-305 Health Information Technology

Carolyn Yocom, (202) 512-7114 or yocomc@gao.gov

In addition to the contact named above, Tom Conahan, Assistant 

Director; Andrea E. Richardson, Analyst-in-Charge; Marisa Beatley; A. 

Elizabeth Dobrenz; and Courtney Liesener made key contributions to this 

report. Also contributing were Jim Ashley, Carolyn Fitzgerald, Krister 

Friday, Monica Perez-Nelson, and Katie Singh.

Appendix IV: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(100333)
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to http://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, LinkedIn, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov and read The Watchblog.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.