The largest federal investments in health IT and patient electronic access to health information are the Medicare and Medicaid EHR Incentive Programs. These programs provide incentives to hospitals and health care professionals that are able to demonstrate meaningful use of a certified EHR system. Providers must attest that they have met certain measures in order to receive payment, with the required functions increasing in complexity as providers move through the stages of the program. Among the measures in the current programs are two that are specifically designed to capture the extent to which patients are able to electronically access their health information. Unless providers claimed an exclusion from reporting these measures, providers were required to successfully complete them in order to receive incentive payments for program year 2015. The measures are as follows: (1) Ability to electronically access health information. More than 50 percent of a providers’ patients must be provided timely access to view online, download, and transmit to a third party their health information, and (2) Actual electronic access. At least one of a provider’s patients must electronically view, download, or transmit to a third party their information during the 90-day reporting period. Providers participating in these EHR Programs must use certified EHR technology, which is technology that has been determined to conform to standards and certification criteria developed by ONC. These criteria do not specify a particular technical method for providing patients with access to their health information, but do specify parameters for accessing certain types of health information. According to ONC, many providers use some type of patient portal to provide access to these types of health information. A patient portal is a secure online website that gives patients 24-hour access to their personal health information and medical records from anywhere with an Internet connection. Patient portals are purchased by a provider and generally only include health information generated and made available by that individual provider. ONC released the most recent strategic plan for health IT, developed with input from federal and nonfederal stakeholders, in September 2015. This plan guides the actions of multiple federal agencies with regard to health IT. The plan outlines four primary goals, each with its own objectives for using health IT, to improve the health and well-being of individuals and communities. Two of these goals are to transform health care delivery and community health and advance person-centered and self-managed care. In addition to this strategic plan, in 2015 ONC developed, with input from federal and nonfederal stakeholders, a Shared Nationwide Interoperability Roadmap (which we refer to in this report as the Roadmap). The Roadmap proposes specific actions to advance the nation towards an interoperable health IT system that collectively improves health. The Roadmap includes the goals that patients can access their longitudinal electronic health information, contribute to this information, send and receive that information through a variety of technologies, and use that information to manage their health and participate in shared decision making with their health care providers. These Roadmap goals support the HHS strategic plan goals of advancing care and transforming health care delivery and community health. Performance measures assess performance via ongoing monitoring and reporting of program accomplishments, which include progress toward pre-established goals. Our previous work has found that performance measures can serve as an early warning system to management and as a vehicle for improving accountability to the public. We have also published guidance on assessing performance which states that it is important for performance measures to be tied to program goals and for agencies to ensure that their activities support their organizational missions and move them closer to accomplishing their strategic goals. In addition, our guidance to federal agencies on designing evaluations suggests that performance measures should include both process and outcome measures. (See table 1.) Outcome measures are particularly useful in assessing the status of program operations, identifying areas that need improvement, and ensuring accountability for end results. Furthermore, our guidance on assessing performance notes that leading organizations should not only establish performance measures but also use information from these performance measures to continuously improve processes, identify program priorities, and set improvement goals. Data from CMS show that most patients who received their health care from providers participating in the Medicare EHR Program had the ability to electronically access their health information. Information from our survey of providers and interviews with patients and providers show that this access is typically offered through patient portals and the type of information offered varies. In interviews, patients described the benefits and limitations of accessing their health information electronically. CMS data show that providers who participated in the 2015 Medicare EHR Program reported offering most of their patients the ability to electronically access their health information. In 2015, all participating hospitals and nearly all participating health care professionals reported offering electronic access to health information to 88 and 87 percent of their patients respectively, on average. This means that the providers gave the patients they saw or discharged all of the information necessary to electronically view, download, and transmit the patients’ health information, such as a website address, a username and password, and instructions for logging onto the website. Our survey of providers who participated in the 2014 Medicare EHR Program and interviews with providers further illustrate the extent to which providers offered their patients electronic access to their health information. Our survey found that nearly all providers routinely provided new patients with access to this information (92 percent of health care professionals and 91 percent of hospitals). Providers we interviewed also described circumstances in which a patient may not have been offered access. These circumstances included instances such as in emergency care, when offering electronic access may not be appropriate at the point of care, or for behavioral health data, when it might not be in the best interest of the patient to access the information. ONC has published information on the extent to which non-federal acute- care hospitals and office-based physicians provide their patients with access to their health information. ONC reported that in 2015, almost all hospitals (95 percent) offered patients the ability to electronically view their health information, and about 7 out of 10 (69 percent) hospitals provided their patients with the ability to view, download, and transmit their health information electronically. In addition, ONC recently reported that among office-based physicians, 63 percent provided patients with the ability to electronically view their health information. ONC reported in another data brief that in 2014, nearly 4 in 10 Americans were offered electronic access to their medical records. Our survey of providers who participated in the 2014 Medicare EHR Program found that most providers offered electronic access to patients through a patient portal, and our interviews suggest that patients often received access to a different portal for each provider. A patient portal is a secure website that allows patients to access information contained in their provider’s EHR system and is managed by the provider. EHR vendors and providers we interviewed noted that patients generally have to manage separate login information for each provider-specific portal. Many patients we interviewed confirmed this; for example, a number of patients we interviewed said that they had access to more than one portal, each of which contained their health information from a different care setting (e.g., hospital stays, general practitioner, and different specialists). The types of health information that providers made available to patients varied, but our survey of providers indicated that most routinely offer access to most types of patient information. The Medicare EHR Program requires that participants make certain types of information available, such as laboratory test results and current medications. According to our survey, an estimated 94 percent of hospitals and 77 percent of health care professionals routinely offered access to laboratory test results, which are required by the program. (See table 2.) Our survey also showed that fewer providers routinely offered access to certain information that is not required by the program and that they find less helpful for their patients to view. For example, 46 percent of hospitals and 54 percent of health care professionals reported routinely offering access to clinician notes, which are not required by the program. Additionally, our survey showed that fewer providers find it helpful for patients to view clinician notes and radiological images than for patients to view information such as laboratory results and current medications. Representatives from two hospitals we interviewed explained that their hospitals relied on a committee to decide what information to make available to patients through the portal and how soon after it is available to the provider to make it available to the patient. One EHR vendor we spoke with noted that the vendor automatically makes almost all information, including clinician notes, available to patients through the vendor’s patient portal by default, though many vendors allow providers to limit the types of information they routinely make available to patients. Patients we interviewed said that the type of information made available in their portals was incomplete and inconsistent across providers. Though many patients talked about accessing their lab results through their portal, multiple patients said that their results were not always available for them to view. For example, one patient said that sometimes her lab results are posted, and other times they are never made available to her, and she does not have a sense of when the results will be made available. Three patients expressed frustration that their vital signs information such as weight and blood pressure was not available through their portals, particularly since they knew that their providers collected this information during visits. Another patient said that she has observed a lot of variability in what information providers make available through their portals, with some doctors providing access to detailed information such as clinical notes and lab results, and others only making basic information available, such as appointment reminders and vital signs information collected during visits. During our interviews with 33 patients who have accessed their health information electronically, patients described numerous benefits from the ability to electronically access their health information. These benefits included the ability to communicate better with their health care providers, track health information over time, and share information with other providers. Multiple patients described circumstances in which they used information in their portal to improve their interactions with their provider and adhere to provider recommendations. For example, one patient described how he logged into the portal after a visit to review instructions from his provider that he had forgotten. Patients also noted that electronically accessing their health information made them feel empowered or more proactive to manage their health, particularly over time. For example, patients described using their electronic access to view specific test results over time to see whether their condition was changing, or to access diagnostic information that gave them the ability to do more research on their medical condition. One patient described using the information in her portal to notice a trend in her lab results and also learn of a condition she had of which her provider failed to inform her. Patients also described using their patient portals to share information with other providers. Multiple patients described printing out medical information from their portals, such as lab results, and bringing that information to appointments with other providers. Patients noted that portals make sharing health information very convenient. However, patients also described some limitations with their access, many of which were related to the functionality of the portal. Patients we interviewed stated that they were able to view their health information electronically, but many patients said that it was not clear that this information could be electronically downloaded or transmitted. Patients also expressed frustration with the amount of time and effort it took to set up electronic access through their providers, managing multiple passwords for their many portals, and understanding each portal’s user interface. Many patients said that the information itself was often incorrect or not presented in helpful ways, and some patients noted that there was no simple way to correct or denote incorrect information within the portal. For example, one patient said that another person’s information was included in her record, and it took multiple requests to her provider to remove this information from her record. Another patient was frustrated that information about his weight that was captured in his yearly physical was not available in the portal in a way that would allow him to track his weight over time. Multiple patients said that an overall limitation is that they could not aggregate all of their health information into a single longitudinal health record. While there are health IT products available to help patients and providers aggregate information, they are not in high demand. For example patient-purchased personal health records (PHR) can enable patients to aggregate electronic information from disparate sources into a single record. Health IT developers said that there are PHR products available for patients who attempt to generate such a record, but our survey and patient interviews indicate that these products are not widely used. Health IT developers noted that these products have limited functionality because they or the users (e.g., patients) cannot access information stored in EHR systems, and one developer noted that a lack of standardization limits the ability to present information in a meaningful way. Patients we interviewed generally stated that they were not using these products, and health IT developers agreed that consumer demand is low. Additionally, relatively few hospitals and health care professionals we surveyed reported having the capability to submit information to PHR products. Provider-purchased products can also help patients and providers aggregate longitudinal health information, according to health IT product developers and EHR vendors we interviewed. For example, one health IT developer explained that providers can currently purchase their product to display information from multiple EHR systems in a single portal; this product would need to be purchased separately from the EHR system and would require additional configuration. However, according to our survey, we estimate that most providers offer patient portals that are packaged with their EHR system. One EHR vendor representative said that the company was currently in the process of developing a product that will enable patients to access information from multiple providers using their EHR system. However, that product has not been released for provider and patient use. Providers participating in the Medicare EHR Program in 2015 reported that relatively few patients electronically accessed their health information when it was made available to them. In other words, few of these patients logged into a patient portal and viewed, downloaded, or transmitted their health information. Our analysis of 2015 Medicare EHR Program data collected by CMS showed that among participating hospitals, 15 percent of their patients electronically accessed their available health information; among physicians and other health care professionals, this percentage was twice as much, with about a third (30 percent) of their patients accessing their available health information. (See fig. 1.) Examining access rates by provider characteristics, our analysis shows that some types of non-hospital based providers reported relatively low percentages of patients accessing their health information electronically in 2015. Analyzing 2015 Medicare EHR Program data supplemented with other HHS data, we found that among non-hospital based providers participating in the 2015 program: health care professionals located in areas with a higher (i.e., above the national median) percentage of residents in poverty and located in rural areas reported lower levels of electronic access to health information, compared with professionals in lower-poverty areas and professionals located in urban areas; health care professionals with 50 or fewer group practice members reported notably lower levels of electronic access to health information compared with professionals with larger numbers of group practice members; and health care professionals other than general or specialty practitioners—including chiropractors, dentists, and podiatrists— reported notably lower levels of electronic access to health information compared with professionals in general practice or specialty practice. (See fig. 2.) Examining access rates by age, our analysis of data from the 2015 Medicare EHR Program and data from HRSA’s Area Health Resources File indicates that the level of electronic access to health information reported by both hospitals and health care professionals was lower among those located in areas with a higher percentage of the population over age 65. (See fig. 3.) The findings from our analysis of access rates by patient age are consistent with other evidence suggesting that older patients may be less likely to access their health information electronically compared with younger patients. Providers we interviewed and who responded to our survey, as well as health IT developers we interviewed, said that a patient’s age affected the extent to which she electronically accesses her health information. Multiple providers who responded to our survey and that we interviewed conveyed that, in their experience, older patients are less likely to electronically access their information. Providers and health IT developers noted that younger patients and those with chronic conditions are most likely to want electronic access to their health information. Some providers we surveyed and interviewed attributed the lack of interest in accessing health information electronically among older patients to a decreased likelihood of having access to a computer or web- enabled device. One provider stated that his hospital serves a large elderly population and that this was the biggest challenge to meeting the requirement under the 2014 Medicare EHR Program that over 5 percent of patients access their health information electronically. A recent data analysis by ONC found no differences in rates of access to or the viewing of online medical records by age, but the analysis did find that individuals between the ages of 50 and 59 had significantly higher rates of electronically communicating with health care providers, looking up test results online, and using smartphone health applications compared with individuals 70 years or older. More generally, a 2013 survey conducted by the Pew Research Center found that adults age 65 or older were most likely to say that they never go online. Age is not the only determinant as to whether patients electronically access their health information. According to studies we reviewed, patients may not access their health information frequently because they do not have a reason to do so. In 2015, ONC reported that for 2013 and 2014, about three-quarters of surveyed individuals who reported that they did not access their medical records online indicated that they did not do so because they did not have a need to use the information. Similarly, another study found that most patients who report rarely or never accessing electronic health information say that they do not have a need to do so. According to patients we interviewed, patients who electronically access their health information typically do so before or after a health care encounter. For example, patients we interviewed said that they accessed information in their portal to review information before or after an encounter with a provider—for example, to review lab test results, communicate with their providers about a recent appointment, or share information between providers during visits. About half of the patients we interviewed also described using portals offered by their providers to access “convenience features” related to receiving health care, such as features used to see appointment reminders, request medication refills, message their provider, or schedule an appointment. Similarly, one of the studies we reviewed found that consumers expressed preferences to use online access to their health information primarily for needs that occur before or after a health care encounter (e.g., to view recently completed lab work or notes from a recent physician visit) or because they are accessing convenience features offered in their provider’s portal, such as online appointment scheduling or to request medication refills. In our survey of 2014 Medicare EHR Program participants, providers reported using a variety of outreach strategies and other efforts to encourage their patients to access the health information made available to them. These methods include promoting the use of patient portals and providing prizes and other incentives to access the portals. (See table 3.) Providers we interviewed similarly reported undertaking a variety of efforts to encourage patients to electronically access their health information. For example, a hospital representative stated that to increase patient access, staff members tell patients about the portal and take steps to register patients for the portal at every interaction. Another hospital representative explained that hospital staff individually assist patients and even help patients obtain a private e-mail address to register for the portal, if necessary. Yet another hospital representative said that the hospital staff wore buttons instructing patients to ask staff about the portal, and the hospital also installed billboards to remind patients to ask staff about the portal. Despite these efforts, this hospital representative said that they struggled to meet the patient electronic access requirements under the Medicare EHR program. Our interviews with the 33 patients and analysis of Medicare EHR program data and our survey data indicate that the type of portal that providers use may influence the extent to which patients access their available health information. In particular, patients we interviewed noted that they sometimes experienced technical difficulties when attempting to access information through the portal or were confused by the portal’s user interfaces. For example, patients noted that they were sometimes unable to access information in their portals due to the sites being down for maintenance or that their portals were not optimized for viewing on a mobile device, which limited their ability to use the portal. Several patients also expressed frustration with the user interface of the portal offered by their providers, noting that it was difficult to navigate and find the information they wanted. About two-thirds of the providers we surveyed reported taking steps to improve their patient portal’s usability or design. Our provider survey data indicate that most providers offer patient portals that are packaged with their EHR system and therefore provided by the same vendor. One vendor we interviewed noted that it allows for some customization for each customer. We viewed demonstrations of three EHR systems’ patient portals, and observed that the portal design does vary by vendor. For example, the portals we viewed had differences in their interfaces, including where to access health information and how tabs were labeled. Our analysis of Medicare EHR Program data from ONC and CMS confirms that the type of portal itself may affect the extent to which patients access their available health information; the average percentage of patients that accessed their available health information varied depending on the provider’s reported EHR vendor. (See figs. 4 and 5.) HHS officials said that two agencies, CMS and ONC, have programs or other efforts aimed at increasing the ability of patients to electronically access their health information, including the ability to access longitudinal health information and aggregate it in a single location. In the case of CMS, agency officials told us that the Medicare and Medicaid EHR Programs have made a significant contribution towards achieving these goals. The two programs require participating hospitals and health care professionals to provide electronic access to health information to a specified portion of patients. According to CMS officials, the programs support HHS’s strategic goals to improve health care through the meaningful use of health information technology. In the case of ONC, agency officials identified multiple efforts they are undertaking to increase patients’ ability to electronically access their health information, including longitudinal health information. Some examples of these efforts include the following: Patient Engagement Playbook. The playbook is a tool developed by ONC to assist providers in engaging patients with health IT by, for example, using patient portals to engage patients in their health and care. Blue Button Initiative. This initiative includes three distinct efforts – a connector, a voluntary pledge program, and a research project. The connector is a website that helps patients locate their health information online and assists in the development of apps and tools to help consumers understand their health information. The voluntary pledge encourages public and private organizations—such as providers, hospitals, technology companies, and non-profit organizations—to commit to making health information available to patients electronically and to encourage patient access. The research effort is designed to understand the unmet needs and challenges facing stakeholders. Health IT Certification Standards and Certification Criteria. These standards and criteria identify certain vocabularies and structured formats that must be included in certified EHR systems and other EHR technology that providers are required to use in order to participate in the EHR programs. Consumer Health Data Aggregator Challenge. By awarding private sector innovation, this challenge aims to spur the development of third-party consumer-facing applications that use open, standardized application programming interfaces to help consumers aggregate their data in one place. (See Appendix II for a list of ONC’s programs and efforts most directly related to increasing patients’ ability to electronically access their health information.) Both CMS and ONC officials told us that their efforts aim to increase the extent to which patients can electronically access their health information. Officials said that their efforts are guided by goals such as the Roadmap’s long-term milestone of enabling patients to access longitudinal health information, contribute to their electronic health information (e.g., send data from wearable devices to their electronic health record), and direct their health information into any location of their choice (e.g., to a PHR application purchased by the patient that aggregates all their health information in a single location). According to ONC officials, the agency’s efforts all support HHS’s Federal Health IT Strategic Plan as well as ONC’s Roadmap, which establishes several milestones for the agency’s ongoing efforts to increase patients’ ability to access their health information electronically. These milestones are the following: 1) a majority of individuals are able to securely access their electronic health information and direct it to the destination of their choice (to be achieved between 2015 and 2017); 2) individuals regularly access and contribute to their longitudinal electronic health information via health IT, send and receive that information through a variety of emerging technologies, and use that information to manage their health and participate in shared decision- making with their care, support, and service teams (to be achieved between 2018 and 2020); and 3) individuals are able to seamlessly integrate and compile longitudinal electronic health information across online tools, mobile platforms, and devices to participate in shared decision-making with their care, support, and service teams (to be achieved between 2021 and 2024). HHS does not have information on the effectiveness of CMS’s and ONC’s efforts to increase the ability of patients to access their health information electronically. Although ONC measures some progress related to these efforts and the Medicare EHR Program, ONC does not directly measure the impact of these efforts on increasing patients’ electronic access to health information. In the case of CMS, officials told us that while they track the number of providers that participate in the Medicare EHR Program, the agency does not directly measure the extent to which the program specifically affects patients’ ability to access their health information electronically. However, HHS officials stated that they do monitor the program by seeking public comments during the rulemaking process and by publicly reporting statistics. Officials told us that ONC collaborates with CMS to monitor and review the EHR Programs and has used the results of these analyses to modify the programs over time. ONC officials told us there is a data use agreement in place that allows ONC to analyze Medicare EHR Program data. Additionally, ONC commissions evaluations of programs initiated under the HITECH Act, including the Medicare EHR Program. While ONC’s data analyses and commissioned evaluation provide information concerning patient access to electronic health information and patient engagement, these efforts do not measure the impact of the Medicare EHR Program on patients’ ability to access their health information electronically. In the case of ONC, ONC measures a range of outcomes associated with its multiple efforts, but the office does not measure the extent to which its individual efforts are having an effect on patients’ ability to access their health information electronically—by determining, for example, if providers that participate in these initiatives have higher rates of patient access. ONC officials stated that they use metrics as a means of assessing whether the technologies and resources made available through ONC’s efforts are being utilized. For example, ONC officials told us they count the number of website visits to the Patient Engagement Playbook page, the number of providers and other stakeholders who have pledged to make electronic health information available to their patients through ONC’s Blue Button Initiative, and the number of times patients access educational videos about their right to access their health information online. According to officials, ONC also uses nationally representative surveys of hospitals, other providers, and patients that are fielded by various organizations to measure the extent to which patients access their health information electronically; however, the surveys cannot be used to measure whether, or to what extent, ONC’s efforts most directly related to patient access are achieving their intended effects. ONC’s survey data identify, for example, how many patients reported being able to electronically view, download, or transmit their health information as well as if patients sent their health information to an app, mobile device, or PHR. The survey of patients provides information on how patients are accessing their health information and what they do with that information once accessed. For example, the survey asks patients whether they have attempted to electronically send their health information to another electronic location such as a PHR application. Finally, the survey also asks patients about the extent to which they experience any challenges when electronically accessing their health information. ONC officials told us that they plan to conduct a consumer survey with different questions in 2017; however, ONC has not finalized the questions for this survey. According to ONC officials, these surveys help the agency understand other factors, such as how broadband access and language influence patient access and whether progress is being made generally towards the Roadmap goal of increasing patients’ ability to access their health information electronically. HHS lacks information on the effectiveness of CMS’s and ONC’s efforts because it has not developed outcome measures. For example, ONC cannot determine if patient electronic access is higher for participants in the Blue Button Initiative compared with non-participants or if providers who use the Patient Engagement Playbook achieve more patient electronic access than non-users. In our prior work we have identified the use of outcome measures as a leading principle for measuring performance. Guidance for federal agencies based on these principles calls for federal agencies to include outcome measures that address the status of program operations, identify areas that need improvement, ensure accountability for end results, and measure progress towards agency strategic goals—in this case, HHS’s goals related to increasing patients’ ability to access their health information electronically. Without outcome-focused performance measures, HHS cannot determine whether, or to what extent, each of its efforts are contributing to the department’s overall goals, or if these efforts need to be modified in any way. Through CMS’s Medicare and Medicaid EHR Programs and ONC’s multiple individual initiatives, HHS supports a wide range of efforts intended to increase patients’ electronic access to their health information. HHS’s investment in these efforts has been significant— since 2009 HHS has spent over $35 billion on the development and adoption of health information technology. CMS’s and ONC’s efforts aim to encourage the use of technologies that allow patients to electronically access their longitudinal health information, contribute to that information, and direct it to any location of their choice. While HHS’s investment in health information technology is significant, HHS lacks the ability to determine whether, or to what extent, CMS’s and ONC’s efforts are helping HHS achieve its goals. ONC is largely responsible for measuring the nation’s progress towards increasing patients’ electronic access to health information. However, ONC has not developed outcome measures to directly measure the effectiveness of its individual efforts, identify areas that need improvement, and ensure accountability for achieving results. Without such outcome-focused performance measures linked to relevant agency goals, ONC—and by extension, HHS—cannot determine whether, or to what extent, each of the programs and efforts is contributing to overall goals, or if these efforts need to be modified in any way. To help ensure that its efforts to increase patients’ electronic access to health information are successful, the Secretary of HHS should direct ONC to take two actions. First, develop performance measures to assess outcomes of key efforts related to patients’ electronic access to longitudinal health information. Such actions may include, for example, determining whether the number of providers that participate in these initiatives have higher rates of patient access to electronic health information. Second, use the information these performance measures provide to make program adjustments, as appropriate. Such actions may include, for example, assessing the status of program operations or identifying areas that need improvement in order to help achieve program goals related to increasing patients’ ability to access their health information electronically. We provided a draft of this report to HHS for its review and comment. HHS provided written comments, which are reprinted in appendix III. HHS also provided technical comments, which we incorporated as appropriate. In its written comments, HHS concurred with both of our recommendations. With regard to our first recommendation, which calls for HHS to develop performance measures to assess the outcomes of key efforts related to patients’ electronic access to longitudinal health information, HHS noted that ONC is committed to assessing the effects of health IT adoption and use. HHS detailed efforts on the part of ONC and CMS to assess progress in patients’ access to their electronic health information and said that the department has used these assessments to modify its programs for encouraging such use over time. HHS stated that there has been an increase in patients’ ability to electronically access and use their health information and noted that we said this in our report. With regard to our statement that ONC is primarily responsible for assessing the effects of the Medicare EHR Program, HHS raised concerns that this statement was misleading because assessing the impact of the program is a coordinated effort between ONC and CMS. In response, we changed our description of the roles of ONC and CMS to reflect HHS’s comment. While HHS has worked to assess the impact of its efforts, it agreed that ONC has not developed a specific means for measuring outcomes associated with ONC’s efforts aimed at furthering patients’ ability to electronically access their health information. HHS also noted that ONC is required by HITECH and the Medicare Access and Children’s Health Insurance Program Reauthorization Act to develop performance measures for the adoption of EHRs and related efforts to facilitate the electronic use and exchange of health information. HHS stated that these required performance measures involved nationwide surveys that go beyond the scope of the Medicare EHR Program data discussed in this report. Therefore, HHS stated that ONC would make every effort to develop performance measures for patient education and outreach initiatives but would have to balance these efforts with its efforts to develop measures for the adoption of EHRs, interoperability, and patient engagement nationwide. In concurring with our second recommendation, that ONC use the information the performance measures provide to make program adjustments, HHS stated that it is committed to using performance measures to guide program improvement. We are sending copies of this report to the appropriate congressional committees, the Secretary of Health and Human Services, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staffs have any questions about this report, please contact me at (202) 512-7114 or at yocomc@gao.gov. Contact points for our Office of Congressional Relations and Office of Public Affairs can be found on the last page of this report. Other major contributors to this report are listed in appendix IV. This appendix provides additional details regarding our analysis of Medicare EHR Program data and our nationally representative provider survey. The data analysis and the provider survey were used, in part, to describe the extent and type of electronic access to health information currently available to patients, the extent to which patients electronically access their health information, and the actions providers are taking to encourage such access. We analyzed data from the Centers for Medicare & Medicaid Services (CMS) as supplemented with other government data to (1) determine the number of providers—that is, hospitals and health care professionals (e.g., physicians)—that participated in the 2015 Medicare EHR Program; (2) determine the number of program participants who reported each of two measures related to patient electronic access to health information; (3) determine the extent to which program participants are offering patients, and patients are using, the ability to electronically access their health information; and (4) examine the characteristics of providers that were associated with higher or lower percentages of patients who actually accessed their available health information. We assessed the reliability of these data by (1) performing electronic testing of required data elements, (2) reviewing existing information about the data and the system that produced them, and (3) consulting agency officials who are knowledgeable about these data. We determined that these data were sufficiently reliable for the purposes of our reporting objectives. Number of providers that participated in the Medicare EHR Program. To determine the number of providers that participated in the Medicare EHR Program in 2015, we analyzed data extracted from CMS’s National Level Repository that represented all successful attestations. CMS collected these data from January 2016 to March 2016. We counted the number of unique providers that were included in the 2015 program data (whom we refer to as “participants”). Number of program participants who reported two measures related to patient electronic access to health information. To determine the number of participants who reported two measures related to patient electronic access to health information, we counted the number of unique providers who reported a number for (1) the percentage of patients who were offered the ability to electronically view, download, and transmit their health information, and (2) the percentage of patients who actually electronically viewed, downloaded, or transmitted their health information. The extent to which program participants are offering patients—and patients are using—the ability to electronically access their health information. To determine the extent to which program participants offered patients the ability to electronically access their health information, we computed the average of the reported percentages of patients who were offered the ability to view, download, and transmit their health information by their provider. To determine the extent to which program participants’ patients actually used the ability to electronically access their health information, we computed the average of the reported percentage of patients who actually viewed, downloaded, or transmitted their health information. To determine the extent to which program participants’ patients actually used the ability to electronically access their health information when it was available, we divided the number of patients who actually accessed their health information by the number of patients who were offered access for each participant, and computed the average. Characteristics of providers associated with higher or lower percentages of patients who actually accessed their available health information. To examine the characteristics of providers that were associated with higher or lower percentages of patients who actually accessed their available health information, we analyzed data on provider characteristics from CMS, the Office of the National Coordinator for Health Information Technology (ONC), the Health Resources and Services Administration (HRSA), and the U.S. Census Bureau. Each characteristic is divided into two or more categories. For example, the characteristic “geographic region” is divided into four categories— Midwest, Northeast, South, and West regions. As part of this analysis, we computed the average percentage of patients who actually accessed their available health information for providers within each characteristic category without controlling for other characteristics. We examined the following provider characteristics: Regional characteristics. We analyzed data on the following regional characteristics using providers’ business zip code: Metropolitan status. We used the 2015-2016 HRSA Area Health Resources File to determine whether providers were located in a metropolitan area—an area that has at least one urbanized area of 50,000 people, among other criteria. We then categorized providers in metropolitan areas as being located in urban areas and providers that were not as being in rural areas. Geographic region. We used information from the U.S. Census Bureau to identify the U.S. census region—Midwest, Northeast, South, or West—where providers were located or practiced. County residents living in poverty. We used information from the HRSA Area Health Resources File to calculate the 2014 national median percentage of counties’ residents living under the poverty line. We then categorized providers into “higher poverty” areas if they were located in a county above the national median percentage of residents living in poverty and “lower poverty” areas if they were located in a county below or equal to the median. County residents over age 65. We used information from the HRSA Area Health Resources File to estimate the 2014 national median of counties’ percentage of residents over age 65. We then categorized providers into “higher population over 65” areas if they were located in a county above the national median percentage of residents over age 65 and “lower population over age 65” areas if they were located in a county below or equal to the median. Hospital type. We analyzed data on the following categorizations of hospital type: Hospital classification. We determined whether hospitals were classified as acute care hospitals or critical access hospitals by using data from CMS’s Hospital Compare file. Ownership type. We used data on ownership type from CMS’s Hospital Compare file to create three categories of ownership: (a) for- profit, (b) nonprofit, and (c) government-owned. Health care professional characteristics. We analyzed data on the following categorizations of professional characteristics: Health care professional specialty. We obtained data on professionals’ primary specialty from CMS’s Physician Compare file. We then consolidated these specialties into the following three categories: (a) general practice physician, (b) specialty practice physician, and (c) other, which includes chiropractors, podiatrists, and dentists. Number of health care professionals in the practice. We estimated the number of professionals in each practice by using data from CMS’s Physician Compare file. We subsequently created three practice size categories: (a) practice of 1 to 10 professionals, (b) practice of 11 to 50 professionals, and (c) practice of 51 or more professionals. We surveyed a nationally representative sample of providers who participated in the 2014 Medicare EHR Program about how they are providing patients with the ability to electronically access health information. The survey was designed to collect information from providers related to patient electronic access to health information, including the methods and specific technology providers use to give patients electronic access to their health information, the types of health information providers make available through these technologies, and any methods providers use to encourage patient electronic access. The survey was also designed to capture providers’ perspectives on the benefits of patients having such electronic access to their health information—specifically whether providers saw it beneficial for patients to electronically view, download, or transmit certain types of health information. The target population for this survey was all hospitals and health care professionals who reported to the Medicare EHR Program that 5 percent or more of their patients viewed, downloaded, or transmitted their health information for the 2014 program year. Using 2014 program data provided to us by CMS, we identified 60,321 hospitals and health care professionals to be included in the population for this survey. We selected a stratified random sample of 1,867 hospitals and providers as described in table 4 below. We stratified the population by type (hospitals and health care professionals) and the reported percentage of patients who electronically accessed their health information in 2014. We computed the sample sizes separately for hospitals and professionals needed to achieve a precision of plus or minus 5 percentage points or fewer at the 95 percent confidence level. We then increased the sample sizes in each group for expected response rates of about 50 and 30 percent for hospitals and health care professionals, respectively. (See table 4.) A link to this web-based survey was emailed to these 1,867 providers via the email addresses included in the program data provided by CMS. We received valid responses from 428 (23 percent) out of the 1,867 hospitals and health care professionals selected in our stratified random sample. The weighted response rate, which accounts for the differential sampling fractions within strata, is 21 percent for the full sample, 28 percent for hospitals, and 20 percent for health care professionals. (See table 5.) We conducted an analysis of our survey results to identify potential sources of nonresponse bias using two methods. First, we examined the response propensity of the sampled hospitals and health care professionals by several demographic characteristics. These characteristics included region, metropolitan status, specialty type, size of practice, hospital type, and hospital ownership type. Second, we compared weighted estimates from respondents and nonrespondents to known population values for measures that are related to the survey outcomes for which we were most interested. We conducted statistical tests of differences, at the 95 percent confidence level, between estimates and known population values, and between respondents and nonrespondents. These analyses were conducted separately for hospitals and health care professionals. Based on this analysis, we did not observe significant differences in response propensities or between known population values and estimates for nearly all of the characteristics we examined. However, we did observe significant differences by ownership type for hospitals and by region for health care professionals. Specifically, we found that proprietary and physician-owned hospitals were significantly under- represented by our respondents. Additionally, we found that professionals in the Northeast and South were significantly under-represented, while professionals in the Midwest and West were over-represented by our respondents. To ensure that the survey results appropriately represented the population of 60,321 hospitals and health care professionals, we weighted the results from the 428 respondents by the inverse of the probability of selection (base weight) and a nonresponse adjustment factor to account for nonresponse and the differences in response propensities we identified. The nonresponse adjustment factor was calculated using weighting class adjustments where adjustment cells were based on strata, hospital ownership type, and region. We repeated the nonresponse bias analysis using the adjusted weights and found no significant differences with known population and the weighed estimates for all of the characteristics we examined. This provided us with evidence that the nonresponse weighting class adjustments help mitigate any potential nonresponse bias introduced by the differences in response propensities we identified. Based on the results of this nonresponse bias analysis and the weighting adjustments, we determined that weighted estimates generated from these survey results are generalizable to the population of hospitals and health care professionals and are sufficiently reliable for the purposes of this report. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval (e.g., plus or minus 7 percentage points). This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. As a result, we are 95 percent confident that each of the confidence intervals in this report will include the true values in the study population. Voluntary pledge where organizations, such as providers and hospitals, commit to advance efforts to increase patient access to and use of their health data. Website to help consumers locate where to find health information online and assist the development of apps and tools to helps consumers understand and use their health information. Research designed to understand the experience of stakeholders surrounding patient access to their own health information and use of electronic health information. The research focuses on how empathy can help to understand the unmet needs and challenges facing stakeholders. Educational video for consumers about their right to access their health information. Intervention aimed at increasing the percentage of patients who enroll in online portals to view, download, and transmit their health records and communicate online with their clinicians. Four videos that show how health information technology is being used for patient engagement, access, and care coordination. Interactive document that walks providers through strategies they can use to engage patients with the use of health IT. Tool that identifies key considerations for adopting health information exchange based on personal health records. Infographic regarding an individual’s right to access their medical records (developed and published in conjunction with the Department of Health and Human Services (HHS) Office for Civil Rights). Easy–to–understand videos for consumers, captioned in English and Spanish, about individuals’ right to access their health information under HIPAA, addressing issues including fees and requesting that health information be sent to a third party (developed and published in conjunction with the HHS Office for Civil Rights). Published report on gaps in legal oversight between the collection of electronic health information regulated by HIPAA and not regulated by HIPAA so that consumers can be better aware of the privacy and security conditions of how they manage their digital health. Task force to identify priority recommendations for ONC that will help enable consumers to leverage API technology to access patient data, while ensuring the appropriate level of privacy and security protection. Provides internal and external stakeholders with common connection points to ONC’s standards and technology efforts. Tech Lab is organized around four areas: 1) standards coordination; 2) testing and utilities; 3) pilots; and 4) innovation. Challenge to stimulate consumer-mediated exchange; will help create API solutions to help individuals securely and electronically authorize the movement of health data to destinations of their choice. Challenge to spur the development of third-party consumer-facing applications that use open, standardized APIs to help consumers aggregate their data in one place. A working group aimed at developing a set of privacy and security specifications that enable a consumer to control the authorization of access to RESTful health-related data sharing APIs, and to facilitate the development of interoperable implementations of these specifications by others. Tool for health care providers, practice staff, and hospital administrators who want to leverage health IT using patient portals to engage patients in their health and care. Update of the 2011 Model Privacy Notice to be more broadly applicable (beyond personal health records). According to ONC, this update provides open–source content that technology developers can use to notify consumers of their privacy and security practices. The 2015 Edition of ONC’s Health IT Certification Criteria supports the certification of health IT, including APIs, to support patient access to health data and view, download, and transmit functions that continue to support patient access to their health information, including via both encrypted and unencrypted email transmission to any third party the patient chooses. Commitment from health care industry to make electronic health records work better for consumers and providers. In addition to the contact named above, Tom Conahan, Assistant Director; Andrea E. Richardson, Analyst-in-Charge; Marisa Beatley; A. Elizabeth Dobrenz; and Courtney Liesener made key contributions to this report. Also contributing were Jim Ashley, Carolyn Fitzgerald, Krister Friday, Monica Perez-Nelson, and Katie Singh.

Summary: HHS's goal is that all Americans will be able to electronically access their longitudinal health information, that is, their health information over time. HHS's efforts to achieve this goal include the Medicare EHR Program and other efforts to encourage providers to make patient health information available and for patients to access such information. GAO was asked to review the state of patients' electronic access to their health information. This report (1) describes the electronic access to health information available to patients, and patients' views of this access, (2) describes the extent to which patients electronically access their health information, and actions providers reported taking to encourage such access, and (3) evaluates HHS's efforts to advance patients' ability to electronically access their health information. GAO analyzed data from HHS and other sources; reviewed applicable strategic planning documents; surveyed a generalizable sample of providers that participated in the Medicare EHR program; and interviewed HHS officials and a nongeneralizable sample of patients, providers, and health information technology product developers. Since 2009, the Department of Health and Human Services (HHS) has invested over $35 billion in health information technology, including efforts to enhance patient access to and use of electronic health information. One of the largest programs is the Centers for Medicare & Medicaid Services' (CMS) Medicare Electronic Health Record Incentive Program (Medicare EHR Program), which, among other things, encourages providers to make electronic health information available to patients. Program data for 2015 show that health care providers that participated in the program (3,218 hospitals and 194,200 health care professionals such as physicians) offered most of their patients the ability to electronically access health information. Patients generally described this access as beneficial, but noted limitations such as the inability to aggregate their longitudinal health information from multiple sources into a single record. Data from the 2015 Medicare EHR Program show that relatively few patients electronically access their health information when offered the ability to do so. Patients GAO interviewed described primarily accessing health information before or after a health care encounter, such as reviewing the results of a laboratory test or sharing information with another provider. While HHS has multiple efforts to enhance patients' ability to access their electronic health information, it lacks information on the effectiveness of these efforts. The Office of the National Coordinator for Health Information Technology (ONC) within HHS collaborates with CMS to assess CMS's Medicare EHR Program as well as its own efforts to enhance patient access to and use of electronic health information. However, ONC has not developed outcome measures for these efforts consistent with leading principles for measuring performance. Without such measures, HHS lacks critical information necessary to determine whether each of its efforts are contributing to the department's overall goals, or if these efforts need to be modified in any way. GAO recommends that HHS 1) develop performance measures to assess outcomes of key efforts related to patients' electronic access to longitudinal health information, and 2) use the information from these measures to help achieve program goals. HHS concurred with the recommendations.