Electronic Health Record Programs: Participation Has Increased, but Action Needed to Achieve Goals, Including Improved Quality of Care
Released: 2014-03-06T00:00:00

Letter

Background

Incentive Payments and Penalties

Through use of incentive payments and penalties, the EHR programs are intended to help address the significant barriers to the adoption and use of EHRs, such as cost, and the technical challenges associated with their use. Beginning in 2011, the first year of the Medicare and Medicaid EHR programs, the programs have provided incentive payments to participating providers, that is, eligible providers that met program requirements.applying a penalty to hospitals and professionals that do not meet the Medicare EHR program requirements. There is no statutory end-point provided for the penalties. The Medicaid EHR program does not impose penalties on Medicaid providers that do not meet the Medicaid EHR program’s requirements by a specific date; however, if Medicaid providers also treat Medicare patients, they are required to meet the Medicare EHR program’s requirements from 2015 onward to avoid penalties from the Medicare EHR program. (See fig. 1.)

Beginning in 2015, CMS is generally required to begin In the Medicaid EHR program, professionals cannot earn more Beginning in 2015, CMS will apply penalties to Medicare providers if they did not meet Medicare EHR program requirements. For most hospitals, penalties will be applied by reducing the Medicare Inpatient Prospective Payment System payment rate increase by 25 percent each year, with a However, for critical maximum cumulative reduction of 75 percent.access hospitals, the penalty is applicable to the hospital’s Medicare reimbursement for inpatient services. For professionals, penalties will be assessed by reducing the reimbursement that the provider would ordinarily receive for furnishing Medicare Part B services by 1 percent for each year the professional did not meet the requirements, with a cumulative penalty of up to 3 percent per year. CMS may exempt hospitals and professionals from penalties if it determines that complying with the program requirements would result in a significant hardship to the provider; such exemptions are to be granted on a case-by-case basis subject to annual renewal. Examples of permissible hardship circumstances include lacking necessary infrastructure, such as internet broadband; facing unforeseen circumstances such as a natural disaster; or, for professionals, lacking patient interaction.

Medicare EHR Program and Medicaid EHR Program Requirements

CMS has established requirements for the EHR programs that specify the providers eligible to participate—that is, providers that are permitted to earn incentive payments or that may be subject to penalties. For example, providers must be a permissible provider type, such as an acute care hospital or a doctor of medicine and, for the Medicaid EHR program, providers must generally meet a patient volume requirement to be eligible.

In addition to meeting program eligibility requirements, to receive incentive payments or avoid penalties, eligible providers must also satisfy reporting requirements by submitting information to CMS for the Medicare EHR program, to the states for the Medicaid EHR program, or to both. The reporting requirements generally incorporate the three statutory criteria for “meaningful use” established in HITECH—(1) demonstrate use of certified EHR technology in a meaningful manner; (2) demonstrate that certified EHR technology is connected in a manner that provides for the electronic exchange of health information; and (3) submit information on CQMs using certified EHR technology. To receive incentive payments from the Medicare EHR program, the information reported by providers must satisfy all three statutory criteria; that is, providers must “demonstrate meaningful use.” However, to receive incentive payments from the Medicaid EHR program in their first year of participation, providers need not satisfy the three statutory criteria. Instead, they must only report that they adopted, implemented, or upgraded to certified EHR technology. In subsequent years, though, they must demonstrate meaningful use to receive incentive payments.

Certified EHR technology. Certified EHR technology is technology that has been determined by ONC-authorized organizations—referred to as certification bodies and testing laboratories—to conform to the standards and certification criteria developed by ONC. ONC has changed the certification criteria since the EHR programs began. Specifically, providers who participated in the EHR programs from 2011 through 2013 were generally required to use 2011-edition EHRs and all providers who participate in the EHR programs from 2014 through 2016—regardless of whether the provider participated in the program in earlier years—are required to use 2014-edition EHRs. another set of certification criteria for EHRs that providers would be required to use beginning in 2017.

In this report, when we use the term EHR, we are generally referring to certified EHR technology. different years. This approach is consistent with the HITECH Act, which directed the Secretary of HHS to seek to improve health care quality and the use of EHRs by requiring more stringent requirements over time. According to CMS, using the staged approach is intended to help overcome two important deficiencies—the lack of widespread use of capable EHRs and the lack of an HIT infrastructure—that existed when the requirements were being developed, with the hope that those deficiencies would be addressed over time as a result of the EHR programs. The stage that is applicable to each provider in each year is based on the first year in which the provider demonstrated meaningful use in the Medicare EHR program or the Medicaid EHR program. For example, a Medicare professional that first demonstrated meaningful use in 2011 must report the Stage 1 meaningful use measures for 2011, 2012, and 2013 to receive incentive payments for those years and must report the Stage 2 meaningful use measures beginning in 2014 to receive The Stage 3 requirements have not an incentive payment for that year.yet been developed but are expected to be finalized in 2015 and to apply beginning in 2017. (See table 1.)

The meaningful use measures in Stage 1 are intended to promote the electronic capture of health information in a structured format and to encourage providers to use that information to track key clinical conditions. The Stage 2 meaningful use measures were chosen to encourage continuous quality improvement at the point of care and are more stringent than the Stage 1 measures. For example, measures that were formerly optional are now mandatory, and some measures require that the provider perform the action for a greater percentage of patients or implement more interventions. Additionally, ONC and CMS have stated that Stage 2 places a stronger emphasis on electronic health information exchange with other providers compared to Stage 1.intention to develop even more stringent meaningful use measures for Stage 3 that would promote activities that can lead to improved health outcomes, such as using decision support tools or providing patient access to self-management tools.

Table 2 provides information on the number and type of measures that providers must report for Stage 1 and Stage 2.

Clinical quality measures (CQM). Health care quality measures, also known as CQMs, can be used to assess provider performance and to drive quality improvement and accountability. For example, one CQM for professionals measures the use of high-risk medications in the elderly. CQMs are composed of a number of clinical data elements, or pieces of data, that must be collected in order to determine performance on any given measure. Using those data elements, a score for the measure can then be calculated. In order to demonstrate meaningful use in the Medicare or Medicaid EHR programs, providers are required to collect CQMs electronically using EHRs and report those data to CMS (Medicare), the states (Medicaid), or both. However, historically, CQMs have been collected through other data collection methods, such as through a detailed, manual review of paper medical records, a process referred to as chart abstraction. Most of the CQMs identified by CMS for use in the EHR programs were originally developed for paper medical records or other data collection methods and later modified so they could be collected using EHRs.

The CQMs that providers must collect using EHRs and report to demonstrate meaningful use are determined by the edition of certified EHRs providers must use. Specifically, using their 2011-edition certified EHRs that applied from 2011 to 2013, hospitals were required to report all 15 CQMs identified by CMS, and professionals were required to report 6 CQMs from a list of 44 measures. EHRs that will apply from 2014 to 2016, hospitals are required to report 16 CQMs from a list of 29 measures, and professionals are required to report 9 CQMs from a list of 64 measures. Medicare providers are being encouraged to electronically report the results of CQMs to CMS, but they have the option of reporting CQMs by manually entering information into CMS’s web-based tools, a process known as attestation.

Professionals had to report 3 core or alternative core measures and had to report an additional 3 measures from the list of measures.

HHS’s Quality Measurement Activities

In addition to the EHR programs, HHS requires providers to collect and report CQMs for a number of other programs in order to measure health care quality, some of which provide financial incentives to health care providers to help achieve the goals of improved quality and efficiency. For example, CMS is to impose penalties on providers who do not satisfactorily report quality data to the Hospital Inpatient Quality Reporting (IQR) program and to the Physician Quality Reporting System (PQRS). The Patient Protection and Affordable Care Act directed HHS to implement new programs that will use CQMs, including pay-for- performance programs such as the Physician Value Based Payment Modifier program, which bases Medicare payments in part on CQM results, and the Medicare Shared Savings Program, which rewards Accountable Care Organizations that lower growth in health care costs while meeting performance standards.

Across its programs, HHS has a goal to use CQMs that are broad based, patient centered, and prioritized based on their potential to achieve population-wide improvements. In order to do this in its programs, CMS has set a goal of collecting and submitting data using EHRs, which HHS officials have indicated can be used to collect CQMs as a byproduct of the routine delivery of health care. To minimize the burden on providers of collecting and reporting similar CQMs for different CMS programs using differing methods, CMS has modified some of its programs’ requirements such that providers may submit data collected through EHRs as part of the Medicare or Medicaid EHR programs instead of separately collecting and submitting data through other methods such as chart abstraction. For example,

Hospitals electronically reporting to CMS one-quarter of CQM data collected in 2014 using 2014-edition EHRs for 16 CQMs will satisfy the IQR program’s four-quarter reporting requirement for those 16 measures for 2016 payment determinations and the Medicare EHR program’s CQM reporting requirement for 2014.

Professionals electronically reporting one year of CQM data generated from 2014-edition EHRs to CMS for PQRS in 2014 will satisfy the CQM reporting requirements for both PQRS and the Medicare EHR program. CMS will use those data for professionals in practices of 10 or more individuals to generate the Physician Value Based Payment Modifier that will be applied to payments made under the Medicare physician fee schedule starting January 1, 2016.

Performance Assessment

GPRAMA requires federal agencies to prepare performance reports and establish performance measures to assess performance of federal programs. Performance measures assess performance via ongoing monitoring and reporting of program accomplishments, which include progress toward pre-established goals. Our previous work notes that performance measures can serve as an early warning system to management and as a vehicle for improving accountability to the public. We have also published guidance on assessing performance that incorporates GPRAMA criteria and which states that it is important for performance measures to be tied to program goals and for agencies to ensure their activities support their organizational missions and move them closer to accomplishing their strategic goals. In addition, our guidance to federal agencies on designing evaluations suggests that performance measures should include both process and outcome measures (see table 3). Outcome measures are particularly useful in assessing the status of program operations, identifying areas that need improvement, and ensuring accountability for end results. Furthermore, our guidance on assessing performance notes that leading organizations not only establish performance measures but also use information from these performance measures to continuously improve processes, identify program priorities, and set improvement goals.

Participation in the EHR Programs Has Increased, but Program Changes Make It Difficult to Estimate Future Participation

Participation in CMS’s EHR programs increased substantially from 2011 to 2012, but some providers who participated in 2011 did not continue in 2012. It is difficult to estimate future participation in the EHR programs because of various program changes, including the planned increase in stringency of the meaningful use measures, the introduction of penalties for some providers in 2015, CMS’s efforts to increase participation among certain providers, and changes to eligibility requirements.

Participation in 2012 Increased Even Though Some Providers Did Not Continue after 2011

Participation in CMS’s EHR programs increased substantially from 2011 to 2012. For hospitals, participation increased from 45 percent of those eligible for 2011 to 64 percent of those eligible for 2012. For professionals, participation increased from 21 percent of those eligible for 2011 to 48 percent of those eligible for 2012. (See fig. 2.)

The amount CMS paid in incentive payments to providers for participating in the Medicare EHR program, Medicaid EHR program, or both programs increased from $5.2 billion for 2011 to $9.5 billion for 2012. In total, CMS paid $14.7 billion in incentive payments to providers for 2011 and 2012. Of this total, hospitals received nearly $8.6 billion and professionals received almost $6.1 billion. Incentive payments for hospitals and professionals under the Medicare EHR program amounted to $8.8 billion, while incentive payments for hospitals and professionals under the Medicaid EHR program were $5.9 billion. (See fig. 3.)

Although participation in the EHR programs has increased overall, a substantial percentage of providers that participated in the Medicare EHR program or Medicaid EHR program in 2011 did not participate in either program in 2012. (See fig. 4.) Specifically, within the 36 states that had completed their determinations of which providers would receive incentive payments for the 2012 Medicaid EHR program year, 61 percent of professionals and 36 percent of hospitals that participated in the Medicaid EHR program in 2011 did not continue in 2012. Sixteen percent of professionals and 10 percent of hospitals participating in the Medicare EHR program in 2011 did not continue to participate in 2012.

CMS officials are aware that some providers participating in 2011 did not continue in 2012 and told us that they are monitoring the issue and taking steps to reverse this trend. One CMS official told us there are various possible reasons Medicare and Medicaid providers did not continue to participate in the EHR programs. Noteworthy for the Medicaid EHR program, and in contrast to the Medicare EHR program, providers do not need to participate in consecutive years to maximize their incentive payments, and there are no penalties for not participating. Another possible reason providers did not continue to participate in the Medicaid EHR program in 2012 is that providers are not required to demonstrate meaningful use for their first year of participation, but must do so for their second year. One CMS official noted that a provider who received an incentive payment for adopting, implementing, or upgrading to a certified EHR could still be far from having the capability to demonstrate meaningful use. The CMS official also noted that the agency conducted a survey of a sample of providers to learn why they did not continue to participate in the EHR programs and found the following additional reasons: some providers did not realize they needed to participate in the program again; some providers did not know the deadline for submitting the required information to CMS for the Medicare EHR program, to the states for the Medicaid EHR program, or to both; some providers switched EHR vendors and were not ready in time to submit the required information; and some providers found it more difficult than anticipated to go from a 90-day reporting period to a full-year reporting period, as required for the second year of participation.

In an effort to reverse this trend, a CMS official told us CMS is working to ensure providers are well-informed regarding program requirements and data submission deadlines. For example, the official told us that CMS is emphasizing requirements and deadlines at meetings with providers, modifying educational materials, and reorganizing the agency’s website devoted to the EHR programs.

Program Changes, Including the Increasing Stringency of Requirements and the Implementation of Penalties, Make It Difficult to Estimate Future Participation

Various program changes may affect participation in the EHR programs in the future, making future participation difficult to estimate. Beginning in 2014, Stage 2 requirements for demonstrating meaningful use will be introduced for many providers participating in the EHR programs, requiring providers to demonstrate that they are using their systems in more advanced ways. The increased stringency of these Stage 2 requirements for demonstrating meaningful use may slow participation in the EHR programs. While CMS and ONC have developed guidance materials and trainings to help providers prepare for Stage 2, survey data and statements by organizations representing providers suggest that some providers may be much less capable of demonstrating Stage 2 meaningful use, compared to Stage 1 meaningful use, which might result in reduced participation.

Conversely, other factors may encourage future participation. Penalties for Medicare providers not participating in the Medicare EHR program begin in 2015 and may motivate providers to begin participating in the program or to continue participating in the program. In addition, CMS and ONC have identified certain provider groups as having more difficulty participating in the program and are taking steps to increase participation among these providers. For example, CMS officials told us the agency has partnered with the American Association of Pediatricians to identify states in need of more outreach to encourage pediatricians to participate in the Medicaid EHR program, and ONC’s Regional Extension Center program has provided assistance to providers such as professionals in solo practices or rural areas to help them participate in the EHR programs. As we previously reported, participation in the Medicare EHR program among rural hospitals and professionals in solo practices increased faster from 2011 to 2012 than participation by hospitals and professionals overall. Furthermore, CMS has made changes to certain program eligibility requirements that may increase the number of providers participating. For example, beginning in 2014, professionals that are hospital-based may receive incentive payments if they can demonstrate that they fund the acquisition, implementation, and maintenance of EHRs in lieu of using a hospital’s EHRs.these professionals were ineligible to participate in the EHR programs.

Providers Exceeded Meaningful Use Requirements for Most Stage 1 Measures, but May Face Challenges with Certain More Stringent Stage 2 Measures

The meaningful use measures reported by providers for 2011 and 2012 indicate that providers used their certified EHR systems more often than required and suggest that many providers who have already reported certain Stage 1 measures will be able to meet most of the more stringent reporting thresholds for similar measures in Stage 2. (See app. IV, table 6.) For 2011 and 2012, Medicare providers that received EHR incentive payments consistently exceeded most meaningful use measure reporting thresholds—that is, a specified percentage of patients or actions required to satisfy the measure—established by CMS and ONC for the Of the 14 hospital meaningful use measures that had EHR programs.thresholds, on average, Medicare hospitals exceeded thresholds by at least 10 percentage points for all measures and by considerably more for many measures. For example, hospitals are required to use computerized provider order entry for more than 30 percent of patients with at least one medication for Stage 1. For both 2011 and 2012, Medicare hospitals reported using computerized provider order entry for medication orders for over 84 percent of patients—about 2.8 times the required threshold for Stage 1. Medicare professionals also greatly exceeded the reporting thresholds. We did not conduct an analysis of meaningful use measures reported by Medicaid providers, but there is some evidence that Medicaid professionals also consistently exceeded required thresholds for 2012. While providers that have not participated in the EHR programs may have different experiences, the fact that Medicare providers and Medicaid professionals that participated in 2011 or 2012—the first two years of Stage 1—generally exceeded most reporting thresholds by wide margins for the measures they reported suggests that they will be able to meet most of the more stringent reporting thresholds in Stage 2. For example, in Stage 2, providers are required to use their certified EHRs to record and chart changes in vital signs for more than 80 percent of patients. Medicare hospitals and professionals exceeded this threshold for both 2011 and 2012, reporting that they met this measure for over 90 percent of patients.

In addition to consistently exceeding most required reporting thresholds, other analyses identify some measures that were frequently reported by (See app. IV, table 7.) For example, close to providers in Stage 1.100 percent of providers for 2011 and 2012 reported the mandatory Stage 1 measure to record smoking status for patients 13 years and older without claiming allowable exemptions. Certain optional measures, including one related to the electronic exchange of health information, were also frequently reported. For example, over 80 percent of hospitals reported incorporating clinical lab test results into EHRs as structured data.

Other measures not related to the electronic exchange of information may also be problematic for providers. For example, fewer than 25 percent of Medicare professionals in 2011 and 2012 reported sending patient reminders for preventive or follow-up care. This measure is optional for professionals in Stage 1 and mandatory for professionals in Stage 2. addition, these measures become more stringent in Stage 2, requiring ongoing submission of these data rather than just a one-time test of the ability to submit those data, which Stage 1 required.

Perform medication reconciliation for patients received from another setting of care or provider of care. Fewer than 30 percent of Medicare hospitals reported this measure for 2011 and 2012. This measure is optional in Stage 1 and mandatory in Stage 2.

Provide patients with their health information electronically. Less than 35 percent of Medicare hospitals and professionals for both 2011 and 2012 reported the mandatory measure to provide patients with an electronic copy of their health information. Data submitted by the providers that reported this measure indicate that very few patients— often fewer than 4 patients per year for hospitals and 12 patients per year for professionals—requested this electronic information. Additional evidence suggests that this measure was reported by a similar proportion of Medicaid professionals for 2012.measure becomes more stringent, by requiring that 5 percent of patients electronically view, download, or transmit their health information to a third party, in addition to requiring hospitals and professionals to provide patients with online access to their health information.

In Stage 2, the Other information also indicates that providers may face challenges with many of the same exchange-related Stage 2 meaningful use measures identified above. An analysis of the American Hospital Association’s 2012 survey of hospitals about technical capabilities related to meaningful use indicates that the least commonly implemented Stage 2 capabilities by acute care hospitals were those related to the electronic exchange of information.electronic data on reportable lab results to public health agencies; submitting electronic syndromic surveillance data to public health agencies; providing an electronic summary of care document for each transition of care or referral; and providing patients with electronic access to their health information by enabling them to view online, download, and transmit such information. These capabilities are related to optional Stage 1 measures but will be required in Stage 2 or become more complex in Stage 2. An analysis of the National Center for Health Statistics’ 2012 National Ambulatory Medical Care Survey shows similar results for physicians. Among the capabilities related to meaningful use measures, office-based physicians were least likely to have routinely submitted electronic data to immunization registries or information systems, or to have routinely communicated with patients using secure messaging. Stage 2 requires professionals to report measures that capture these capabilities, whereas reporting on the first capability was optional in Stage 1 and the second was not a Stage 1 measure.

Such capabilities included the following: submitting Consistent with indications from the survey data, stakeholder organizations have also noted concern that providers may have difficulty satisfying certain Stage 2 meaningful use measures due to the changes made to them for Stage 2. Table 4 lists meaningful use measures that organizations identified as challenging—two of which relate to the electronic exchange of information—and includes the requirements for satisfying each measure in Stage 1 and in Stage 2.

A CMS official noted that although providers have expressed concern about their ability to meet the Stage 2 requirements, similar concerns existed prior to Stage 1 regarding the thresholds established for certain measures and the data suggest that providers have exceeded thresholds established by CMS for the Stage 1 requirements. This official also explained efforts to help providers to prepare to satisfy the Stage 2 meaningful use measures. CMS has been modifying its materials and holding training sessions and discussions with various provider groups. In anticipation of obstacles to the exchange of health information and in order to accelerate electronic exchange even among providers not eligible for incentive payments from the EHR programs, ONC and CMS issued a request for information in March 2013 to, among other things, solicit feedback on the challenges to the electronic exchange of health information and on possible mechanisms to overcome those challenges. Using the information collected, CMS and ONC issued a strategy in August 2013 for how they expect to advance health information exchange, which includes various steps the agencies plan to take under three broad categories—accelerating health information exchange, advancing standards and interoperability, and patient engagement.Ongoing work is examining the key challenges to the electronic exchange of health information that have been reported by providers and stakeholders and HHS’s efforts to address those challenges.

HHS Lacks a Comprehensive Strategy to Help Ensure Reliability of EHR Data Collected to Improve Quality of Care

The lack of a comprehensive strategy limits HHS’s ability to ensure that the department can reliably use the CQMs collected in certified EHRs for quality measurement activities. According to internal control standards, an agency must have relevant and reliable information to help it achieve its mission and goals. In order to drive quality improvement and accountability and to effectively utilize CQM data collected in EHRs for HHS’s quality measurement activities, the data collected must be reliable, that is, both accurate and complete. Data that are not reliable limit the credibility of the information and may present a risk to individuals or entities making decisions based on the data, such as to a patient choosing a hospital for treatment, or to CMS when applying a payment modifier resulting in an increase or reduction to a physician’s reimbursement.

CMS and ONC have made changes to the specifications, certification criteria, and certification program to address some of the reliability concerns with the CQMs that providers were generally required to collect and report from 2011 to 2013 to demonstrate meaningful use (that is, using 2011-edition EHRs). These changes are incorporated into EHRs that will be used by participating providers in all stages of meaningful use from 2014 to 2016 (that is, 2014-edition EHRs). The changes made as a result of the concerns identified include the following.

Specifications. CMS made changes to CQM specifications—that is, the documentation that describes to providers, technology developers, and others how data on CQMs were to be collected through EHRs. These changes were made to correct previously identified errors and to help ensure the feasibility of implementing CQMs in the electronic format—that is, to ensure that the required data are readily available or can be collected in EHRs without undue burden. Problems with feasibility can compromise the accuracy or completeness of the data.

Certification criteria. To address providers’ concern that their 2011- edition EHRs did not collect all of the data elements necessary to calculate the CQMs, ONC modified the CQM-related certification criteria.

Specifically, 2014-edition EHRs must collect each of the data elements associated with a specific CQM to be certified for that measure.

Certification program. The program for certifying 2011-edition EHRs did not test whether the EHR system correctly collected and calculated the CQMs. However, beginning with the certification of 2014-edition EHRs, the certification bodies and testing laboratories will test whether EHRs accurately collect and calculate CQMs. These tests will be performed using a certification testing tool developed by ONC, known as Cypress. Further, beginning in 2014, the certification bodies will be required by ONC to conduct specific annual surveillance activities to determine whether 2014-edition EHRs are functioning as intended after being implemented by providers. CQMs have been identified as a priority area for these surveillance activities, which may help to inform CMS and ONC on whether the changes they have made to the specifications, certification criteria, and certification program are improving the reliability of CQMs, as intended, and whether further changes are needed.

CMS and ONC officials indicated the changes made thus far should help to address reliability concerns, but provider experience with 2014-edition EHRs will be an important gauge of the effectiveness of the changes. Furthermore, there are still issues that could result in reliability problems, hindering CMS’s ability to use the data for its quality measurement activities. For example, after EHRs are certified as 2014-edition EHRs, the agencies do not require them to be recertified following updates to the specifications, which may be made annually or more frequently, or following updates to the certification testing tool.

Testing bodies use the certification testing tool—Cypress—to test that EHRs accurately export, calculate, and electronically submit the data. This tool includes synthetic patient data used in the testing process. Vendors also have access to the testing tool prior to submitting their EHR products for certification. providers may report CQMs based on and tested to different requirements. CMS has taken some steps to mitigate the potential effect of these issues on the reliability of CQM data submitted by professionals who electronically submit CQM data by requiring them to report CQMs collected using EHR technology based on and tested to the most recent version of the electronic specifications. However, this variability may limit the ability to reliably compare quality across other providers, including hospitals, and undermines efforts CMS officials said they are taking to ensure that the CQMs lead to standardized, consistent data reporting. Another issue that may affect reliability is that many providers reported at least one CQM based on few patients, which may result in data that are not statistically reliable and therefore should not be used in quality measurement. We analyzed data submitted by providers to demonstrate meaningful use under the Medicare EHR program for 2011 and 2012 and found that 90 percent of hospitals and 50 percent of professionals reported at least one CQM based on few patients for 2012, a slight increase compared to 2011, when 86 percent of hospitals and 46 percent of professionals reported at least one CQM based on few patients.

In addition, stakeholder organizations continue to have concerns about reliability. CHIME members indicated that they do not think the specifications required to generate CQMs using 2014-edition EHRs will substantially improve the reliability of the data collected in Stage 2. They noted that the collection of CQM data includes data elements not currently captured by EHRs, and the data elements are not automatically part of most clinicians’ workflow (that is, incorporated into the tasks performed during the delivery of routine care). Due to these obstacles, CHIME members are concerned that clinicians will not consistently collect the required data elements, ultimately leading to problems with CQM reliability and comparability. CMS and ONC are making changes to the process used to develop new measures, which includes greater consideration of clinician workflow and provider burden. However, new CQMs will not be available until the beginning of Stage 3 in 2017.

Another concern expressed by stakeholder organizations such as the American Health Information Management Association, the American Hospital Association, the American Medical Association, the Healthcare Information and Management Systems Society, and The Joint Commission is that the CQMs generated from EHRs—which are based on electronically specified measures and are extracted automatically from EHRs—do not produce results that are comparable to corresponding measures based on data obtained through other methods, such as through manual abstraction of patient medical records by trained professionals. In its fiscal year 2014 Inpatient Prospective Payment System final rule, CMS noted that it intends to use the electronic CQM data submitted voluntarily by hospitals to assess the differences in performance rates from CQMs collected using EHRs and CQMs collected through chart-abstraction. However, CMS did not indicate when this study would be completed or describe how it would be designed. Designing the study to control for any other systematic differences between providers that use EHRs and other providers could have an important effect on CMS’s ability to understand the extent and implications of potential differences in performance rates.

Although CMS officials told us that they were beginning to develop an approach to validate CQM data generated from EHRs, HHS has not yet developed a comprehensive strategy to address concerns with the reliability of CQMs collected using certified EHRs. Consistent with GPRAMA and our prior work, which identifies several important practices that can guide agencies in planning and implementing an effective government program, a comprehensive strategy would establish objectives, steps to achieve results, priorities, and milestones and identify the steps necessary to achieve desired results. Addressing the concerns is important to ensure that the CQM data can be reliably used for CMS’s quality measurement activities as well as providers’ quality improvement activities. For example, CMS plans to use CQMs collected and submitted via EHRs by certain professionals in 2014 to determine which providers will be subject to payment modifications under the Value Based Payment Modifier program in 2016. For hospitals, CMS officials told us that as part of their approach to validating data, they plan to use electronically submitted CQMs to develop monitoring processes to help inform them about the consistency and reliability of the data reported. Also for hospitals, CMS officials told us that they expect to introduce a process that is as robust as the one that is currently used for the IQR program under which a CMS contractor conducts an independent analysis to verify that the data submitted accurately reflect the information in the medical record. CMS officials told us that the agency’s plans for validating CQM data are preliminary, but the agency has stated that it intends to develop and propose a strategy for hospital data in 2014 as part of the agency’s annual rulemaking process. While the IQR program may provide a model for validating data collected by hospitals using certified EHRs, no comparable model or process exists that could be considered when developing a validation strategy for data collected by professionals using certified EHRs. Until HHS establishes and implements a comprehensive strategy to ensure the reliability of CQMs collected using certified EHRs, it will be unclear whether the department’s plans are sufficient to address the concerns, and therefore it will be uncertain when the CQM data can be reliably used to help assess provider performance, improve quality, and adjust provider payments.

Performance Measures for the EHR Programs Do Not Assess Outcomes Such As Improved Quality, Efficiency, and Patient Safety

HHS, CMS, and ONC have established some performance measures for the EHR programs that are tied to strategic goals, but have not established performance measures for the goals most relevant to the intended outcomes of the EHR programs—that is, goals related to improving health care quality, efficiency, and patient safety. Establishing performance measures tied to goals is important for ensuring that agencies’ activities move them closer to accomplishing these goals, according to our guidance to federal agencies on effectively implementing GPRAMA, which describes leading practices for how federal agencies should assess their performance. Our guidance also notes that agencies should use information that performance measures provide to improve processes and set improvement goals. However, as the agencies have not established performance measures to assess outcomes, they are therefore unable to use the information such measures would provide.

In addition, the Centers for Disease Control and Prevention has 3 performance measures related to the meaningful use of EHRs intended to measure the readiness of public health agencies to receive electronically submitted data as part of the EHR programs. The Centers for Disease Control and Prevention provides direct assistance to state public health agencies to support electronic health information systems to receive data from providers.

GAO/GGD-96-118 (Washington, D.C.: June 1, 1996). the percentage of nonfederal acute care hospitals that are electronically sharing summary of care records with providers outside their organization. (See app. V, table 8 for a list of the department- and agency-level strategic goals and 26 associated performance measures for this category.)

For these established performance measures, the agencies have developed corresponding performance targets and track results against those targets. For example, ONC has set a fiscal year 2014 target that 65 percent of nonfederal acute hospitals electronically share summary of care records with providers outside of their organization. These performance targets are publicly released alongside the performance measures and actual results in the annual ONC and CMS budget justifications and the HHS online performance appendix. The agencies told us they use this performance information to set the broader strategic direction for the agency and may also use the information to make changes to the programs to improve performance.

However, although HHS, CMS, and ONC have established important performance measures for the goals related to adoption and meaningful use of EHRs, they have not established measures linked to the second category of goals, which would help them to track program outcomes such as health care quality, efficiency, and patient safety. (See app. V, table 9 for a list of the department- and agency-level strategic goals for this category.) Outcome measures would enable the agencies to measure whether the EHR programs’ intended effects are being achieved, rather than just the number of providers that satisfied the requirements necessary to demonstrate meaningful use. ONC’s strategic plan states that the meaningful use of EHRs—which, as explained above, the agencies track using several performance measures—is a necessary step to improve health care quality, efficiency, and patient safety. However, meaningful use is not sufficient to achieve these outcomes on its own. The plan notes that the EHR programs are an HIT investment made in lockstep with various other reform efforts such as patient- centered medical homes and that EHRs will contribute to improvements in health care quality, efficiency, and patient safety when used in support of these other reform efforts. Therefore, outcome measures—and the subsequent use of the information they provide—could be beneficial to evaluating how the EHR programs interact with other reform efforts. Furthermore, ONC has acknowledged the need to develop performance measures to track patient safety, in part due to concerns that EHRs could have some unintended consequences that cause patient harm. However, ONC has not yet developed these potential measures.

Without performance measures linked to the goals of improved health care quality, efficiency, and patient safety, CMS and ONC are limited in their abilities to determine the effectiveness of the EHR programs in improving outcomes. CMS and ONC may lack critical information necessary to establish program priorities and subsequently make program adjustments based on progress toward these outcomes. CMS and ONC officials have indicated that they do not expect to observe progress toward achieving the intended outcomes of improved health care quality, efficiency, and patient safety until at least Stage 3 of the EHR programs, which is not expected until 2017. However, establishing outcome-oriented performance measures before results are expected is a valuable practice to ensure that the agencies can make program adjustments as needed and are prepared to monitor outcomes and to establish baseline values, which can be useful for developing performance targets and assessing progress toward goals. Moreover, consistent with our guidance, outcome measures could be useful to guide the development and prioritization of Stage 3 requirements, which the agencies anticipate undertaking in 2014.

While the agencies have not established outcome-related performance measures, agency officials told us they are conducting some evaluations to assess the effect of the EHR programs on outcomes. For example, ONC officials told us that CMS, ONC, and the Agency for Healthcare Research and Quality were working on a study to assess the effect of electronic prescribing on quality and efficiency using Medicare Part D claims, specifically, to determine whether electronic prescribing for diabetes patients is associated with a reduced risk of adverse drug events. (See app. VI for a summary of program evaluations assessing the EHR programs’ effects on outcomes.) However, these program evaluations—in contrast to performance measures—capture only a specific period of time, and, as a result, do not allow for ongoing monitoring of the EHR programs’ effectiveness.

Conclusions

The EHR programs were intended to be long-term solutions—with multiple stages implemented over several years—to help address longstanding, significant barriers to greater use of EHRs as tools for improving outcomes such as health care quality, efficiency, and patient safety. The first stage of the EHR programs, from 2011 to 2012, has shown sizeable increases in the number of providers who have adopted and meaningfully used EHRs. This early measure of implementation still leaves substantial room to further increase meaningful use among certain providers. Implementation of the subsequent stages, which will increasingly emphasize the use of more advanced features of EHRs, will place more demands on providers through more stringent requirements, but could also increase the potential for achieving better outcomes. Because of the programs’ design to evolve over multiple years and stages, HHS has opportunities to make adjustments and closely monitor the programs to ensure that they remain on track to achieve their intended results.

One key feature of EHRs that shows a need for adjustment relates to the CQMs. We describe several issues regarding the reliability of CQM data—for example, providers may be reporting CQMs based on and tested to different specifications. The reliability of CQM data collected using certified EHRs is especially important due to the potential risk of making decisions—such as paying providers for performance—based on unreliable CQM data. CMS and ONC lack a comprehensive strategy that would allow them to ensure the reliability of CQM data collected using EHRs, in accordance with the internal control standard requiring an agency to have relevant and reliable information to achieve its mission and goals. Without a comprehensive strategy, the agencies may be limited in their abilities to reliably compare quality across providers or use CQM data collected through EHRs for a variety of quality measurement and pay-for-performance activities, thus restricting efforts to improve quality and efficiency.

Another area that merits attention relates to HHS’s ability to ensure that the EHR programs are on track to meet their goals. While the EHR programs are ultimately intended to improve outcomes such as health care quality, efficiency, and patient safety, the agencies have not established performance measures for monitoring progress toward achieving these improvements. Although HHS expects that EHRs can help achieve improved outcomes as well as support various other health care reform efforts that are also intended to improve care, that result is not yet assured. As established by our guidance to federal agencies, monitoring progress toward specific outcomes could allow HHS to make adjustments to the EHR programs throughout their implementation, such as adjusting priorities or requirements. Without such monitoring, the agencies will not be well positioned to understand how challenges affecting key parts of the programs, such as ensuring the reliability of CQM data or increasing providers’ abilities to exchange health information, might be affecting the programs’ overall ability to achieve their goals, reducing HHS’s ability to make course corrections.

Recommendations for Executive Action

To more effectively use CQMs to assess provider performance, the Secretary of Health and Human Services should direct CMS and ONC to take the following action:

Develop a comprehensive strategy for ensuring that CQM data collected and reported using certified EHR technology are reliable, including testing for and mitigation of reliability issues arising from variance in certified EHR systems tested to different CQM specifications.

To ensure that CMS and ONC can effectively monitor the effect of the EHR programs and progress made toward goals, the Secretary of Health and Human Services should direct the agencies to take the following two actions:

Develop performance measures to assess outcomes of the EHR programs—including any effects on health care quality, efficiency, and patient safety and other health care reform efforts that are intended to work toward similar outcomes—and

Use the information these performance measures provide to make program adjustments, as appropriate, to better achieve program goals.

Agency Comments and Our Evaluation

HHS provided written comments on a draft of this report, which are reprinted in appendix VII, and one technical comment. HHS agreed that data reliability and performance monitoring are important, but neither agreed nor disagreed with our recommendations. Regarding our first recommendation, HHS stated that the reliability and validity of CQM data are of paramount importance. HHS also stated that ONC and CMS work together to continually refine strategies to improve the reliability and efficiency of collecting, analyzing, and sharing CQM data collected using certified EHR technology. HHS did not, however, identify any specific actions the agencies might take to improve CQM reliability or whether those actions would include developing a comprehensive strategy. HHS also noted the importance of the certification testing tool, known as Cypress, which it stated ensures the reliability of CQM data collected using 2014-edition EHRs. While the addition of testing to ensure that 2014-edition EHRs accurately collect and calculate CQMs is an improvement over the process used to certify 2011-edition EHRs—during which such tests were not conducted—our report raises other reliability issues that would not be addressed by the certification testing tool alone. Regarding our second and third recommendations, HHS agreed that outcome-oriented performance measures that link participation in the EHR programs with discrete improvements in health care quality, efficiency, and safety would be useful for evaluating the extent to which improvements are achieved. HHS also noted that ONC oversees evaluation activities that have provided interim program results and conducts policy research and analytic projects that will enable the development of outcome-oriented performance measures. However, HHS did not provide any details on the timing or content of outcome-oriented performance measures that CMS and ONC might develop.

We are sending copies of this report to the Secretary of Health and Human Services, the Administrator of CMS, the National Coordinator for Health Information Technology, and other interested parties. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-7114 or at kohnl@gao.gov. Contact points for our Office of Congressional Relations and Office of Public Affairs can be found on the last page of this report. Other major contributors to this report are listed in appendix VIII.

Appendix I: Effect of the HITECH Act on Health Insurance Premiums

The Health Information Technology for Economic and Clinical Health (HITECH) Act mandates us to report on, among other things, the impact of the HITECH Act on health insurance premiums. In 2009, the Congressional Budget Office stated an expectation that nationwide adoption of health information technology such as electronic health records (EHR) would reduce total spending on health care and predicted that lower health care costs for private payers would result in lower health insurance premiums in the private sector.

To describe the effect of the HITECH Act on health insurance premiums, we contacted representatives from America’s Health Insurance Plans, the American Academy of Actuaries, the Blue Cross Blue Shield Association, and four health insurance companies. None of the seven organizations we spoke to had done any research to look at the impact of the HITECH Act on health insurance premiums. Three organizations also indicated that the HITECH Act’s effect on health insurance premiums would be difficult to isolate from the various other drivers of health insurance premium costs. For example, one organization noted that the health insurance industry is also in the midst of implementing the Patient Protection and Affordable Care Act, suggesting the reforms arising from that legislation are also affecting health insurance premium costs.

Five of the seven organizations offered anecdotes or speculation as to the potential effect of the HITECH Act—and particularly the adoption and meaningful use of EHRs as encouraged by the EHR programs—on health insurance premiums. One organization told us that there is evidence that widely implemented payment and delivery reform—particularly the patient-centered medical home, of which EHRs are an important element—has led to reductions in health care utilization and, in some cases, generated savings, which may eventually affect health insurance premiums. Another organization said that implementing the HITECH Act and the Patient Protection and Affordable Care Act has increased administrative costs and expressed concern that if the organization’s administrative costs increase due to changes in requirements, they may rise above the 15 to 20 percent level allowed under medical loss ratio requirements.

Appendix II: Scope and Methodology

This appendix provides additional details regarding our analysis of (1) participation in the Medicare and Medicaid Electronic Health Record (EHR) programs; (2) meaningful use measures and clinical quality measures (CQM) Medicare providers reported to the Centers for Medicare & Medicaid Services (CMS) to demonstrate meaningful use; and (3) 2012 survey data collected by the National Center for Health Statistics.

To ensure the reliability of the data, we interviewed officials from CMS, reviewed relevant documentation, and conducted electronic testing to identify missing data and obvious errors. On the basis of these activities, we determined that the data were sufficiently reliable for our analysis.

Analysis of participation in the Medicare and Medicaid EHR programs. We conducted several analyses to report on participation in the Medicare and Medicaid EHR programs for the 2011 and 2012 program years. Specifically, we analyzed data that CMS collected in the National Level Repository through October 23, 2013. As a result, we generally included full-year information for both years in our analysis of the Medicare EHR program.complete information for the Medicaid EHR program for the 2012 program year because, according to CMS, 14 states and the District of Columbia had not yet completed their determinations of which hospitals and professionals had met all the requirements to receive incentive payments for 2012. We used these data to conduct the following analyses.

However, our analysis does not contain

We estimated the percentage of eligible providers awarded an incentive payment from either the Medicare or the Medicaid EHR program. We determined the number of providers that were awarded a Medicare or Medicaid (or, in the case of some hospitals, both) EHR incentive payment by counting the number of providers that had an incentive payment disbursed to them for 2011 and for 2012.

Using a similar approach, we determined the number of providers that met the participation requirements of the Medicare Advantage EHR program for both years. divided the number of providers awarded incentive payments from either program by the total number of eligible providers CMS estimated in its Stage 1 final rule. Specifically, CMS estimated that 5,013 hospitals and 521,600 professionals were eligible for the Medicare or Medicaid EHR program for 2011 and 5,013 hospitals and 527,200 professionals for 2012.

We determined the total amount of Medicare and Medicaid EHR incentive payments awarded to providers each year by summing the Medicare and Medicaid EHR incentive payments awarded to providers for each program year.

We estimated the percentage of providers that were awarded a Medicare or Medicaid (or, in the case of some hospitals, both) EHR incentive payment for 2011 but not for 2012 by determining the number of providers that had a Medicare or Medicaid EHR incentive payment disbursed to them for 2011 but not for 2012. For the Medicaid EHR program, we limited this analysis to providers that were awarded incentive payments from 36 states that, according to CMS, had completed their determinations of which providers would receive incentive payments for the 2011 and 2012 Medicaid EHR program years as of October 23, 2013.

Analysis of meaningful use measures and CQMs Medicare providers reported to CMS to demonstrate meaningful use. We conducted several analyses of meaningful use measures and CQMs providers reported to CMS to demonstrate meaningful use under the Medicare EHR program for 2011 and 2012. We analyzed data that CMS collected in the National Level Repository through October 23, 2013. We did not analyze meaningful use measure or CQM data submitted by hospitals or professionals who received incentive payments from the Medicare Advantage EHR program or CQM data submitted by providers to one of the electronic reporting pilots that were available for the 2012 program year.program year.

For each of the following analyses, we compared reporting by

We determined the extent to which providers exceeded the reporting thresholds CMS established for a subset of the meaningful use measures (14 hospital measures and 16 professional measures) by determining providers’ average reporting rate for each applicable measure and comparing this rate to the threshold CMS established for the measures.

We determined the frequency with which providers reported meaningful use measures without claiming exemptions, by calculating the percentage of providers that reported meaningful use measures but excluding from the calculation those providers that reported to CMS that a measure was not relevant to them. A provider may claim exemptions from reporting some measures if the provider meets certain criteria specific to the individual measures. For example, the measure may not be relevant to the provider’s patient populations or clinical practices, the provider may have conducted too few actions to be measured, or the provider may not have been able to perform the action. The agency allows providers to claim exemptions from reporting certain meaningful use measures to help ensure that providers with all types of patient populations and clinical practices could potentially demonstrate meaningful use. For 2011 and 2012, hospitals were permitted to claim exemptions from reporting 7 meaningful use measures and professionals from reporting 14 meaningful use measures.

We determined the extent to which providers had few patients who could be included in the calculation of at least one CQM.capture a small number of patients may be unreliable measures of quality because relatively small changes in the number of patients who experienced the care processes or outcomes targeted by the measure can generate large shifts in the calculated percentage for the measure. CMS has recognized in other programs that including a small number of patients in the calculation of a CQM is a reliability issue. For example, on the agency’s Hospital Compare website, which publicly reports CQMs by hospital, CMS indicates whether the number of patients included in a particular measure calculation was based on fewer than 25 patients and was thus too small to reliably tell how well the hospital was performing. For our analysis, we identified CQMs as unreliable if fewer than 7 patients met inclusion criteria for the calculation the provider reported for a 90-day reporting period—that is, during the provider’s first year demonstrating meaningful use. For providers whose second year demonstrating meaningful use was 2012, the reporting period is a full year. Therefore, for these providers we identified CQMs as unreliable if fewer than 25 patients met the inclusion criteria for the calculation.

Analysis of 2012 survey data collected by the National Center for Health Statistics. We analyzed output of the National Ambulatory Medical Care Survey, an annual, nationally representative survey of office-based physicians conducted by the National Center for Health Statistics in the Centers for Disease Control and Prevention. The survey captures information about computerized capabilities of office-based physicians’ EHRs—which are related to meaningful use measures— including whether those capabilities are routinely used. We analyzed output from the 2012 survey obtained from the National Center for Health Statistics. We limited our analysis to office-based physicians who reported using an EHR or an electronic medical record system and who reported having patient care revenue from Medicare or Medicaid (69.0 percent of weighted responses). We then identified the computerized capabilities that were least likely to have been routinely used and determined, using a crosswalk of meaningful use measures and survey responses available from the Office of the National Coordinator for Health Information Technology (ONC), whether those capabilities were comparable to Stage 2 meaningful use measures. We also determined whether the computerized capabilities that were least likely to have been routinely used were related to electronic health information, based on information obtained from CMS and ONC.

Appendix III: Stage 1 and Stage 2 Meaningful Use Measures

In general, to receive incentive payments from the Medicare or Medicaid Electronic Health Record (EHR) programs in Stage 1 or Stage 2, hospitals must report on a total of 19, and professionals must report on a total of 20, measures regarding their use of certified EHR technology, known as meaningful use measures. For certain meaningful use measures, a provider may report to the Centers for Medicare & Medicaid Services (CMS) that the measures are not relevant if the provider meets certain criteria specific to the individual measures; this is referred to as claiming an exemption. For example, the measure may not be relevant to the provider’s patient population or clinical practice, the provider may have conducted too few actions to be measured, or the provider may not have been able to perform the action. Table 5 lists the meaningful use measures for each stage and provider type and identifies whether the measure was mandatory or optional and whether it was identified by the Office of the National Coordinator for Health Information Technology (ONC) and CMS as being related to the electronic exchange of health information.

Appendix IV: Meaningful Use Measure Reporting for the Medicare EHR Program, 2011 and 2012

In general, to receive incentive payments from the Medicare Electronic Health Record (EHR) program for 2011 or 2012, hospitals had to report on a total of 19, and professionals had to report on a total of 20, Stage 1 meaningful use measures. Among those measures reported, 14 were mandatory for hospitals and 15 were mandatory for professionals. The remaining measures were selected by providers from a list of optional measures. For certain meaningful use measures, providers must meet or exceed reporting thresholds—that is, a specified percentage of patients or actions required to satisfy the measure—established by the Centers for Medicare & Medicaid Services (CMS) and the Office of the National Coordinator for Health Information Technology (ONC) for the EHR programs. Table 6 displays the average percentage reported by providers for those measures that require providers to meet or exceed certain established thresholds. Table 7 displays the percentage of providers that reported certain measures for 2011 and 2012 but excludes from the calculation providers that reported to CMS that a measure was not relevant to them; this is referred to as claiming an exemption. Specifically, table 7 displays the percentage of providers that reported Stage 1 mandatory meaningful use measures that allow for exemptions and Stage 1 optional meaningful use measures. For 2011 and 2012, hospitals were allowed to claim exemptions from reporting 7 meaningful use measures, and professionals were allowed to claim exemptions from reporting 14 meaningful use measures.

Appendix V: Strategic Goals and Associated Performance Measures Relevant to the EHR Programs

We reviewed Department of Health and Human Services (HHS), Centers for Medicare & Medicaid Services (CMS), and Office of the National Coordinator for Health Information Technology (ONC) strategic goals and categorized them into one of two major categories particularly relevant to the Electronic Health Record (EHR) programs—(1) adoption and meaningful use of EHRs; and (2) improving quality, efficiency, and patient The first category of goals is focused on program processes, and safety.the second is focused on program outcomes. For the first category— adoption and meaningful use of EHRs—HHS, CMS, and ONC have established 26 performance measures. See table 8 for the department- and agency-level strategic goals and performance measures that fall into this category.

Note that ONC has three other goals that are relevant to the EHR programs but did not fall into the two categories we focused on and are thus not included in the table. Similarly to the measures identified in table 8, the performance measures associated with those additional goals enable the agency to track processes but not outcomes.

Appendix VI: Program Evaluations Assessing the EHR Programs’ Effects on Outcomes

Centers for Medicare & Medicaid Services (CMS) and Office of the National Coordinator for Health Information Technology (ONC) officials told us they are conducting some evaluations to assess the effect of the Electronic Health Record (EHR) programs on outcomes. See table 10 for a summary of these evaluations according to agency officials.

Appendix VII: Comments from the Department of Health and Human Services

Appendix VIII: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Will Simerl, Assistant Director; Julianne Flowers; Melanie Krause; Shannon Legeer; Hannah Marston Minter; Monica Perez-Nelson; Roseanne Price; and Rebecca Rust Williamson made key contributions to this report.