Electronic Health Record Programs: Participation Has Increased, but Action Needed to Achieve Goals, Including Improved Quality of Care
Released: 2014-03-06T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: The Health Information Technology for Economic and Clinical Health (HITECH) Act established the EHR programs, which provide incentive payments for—and later are expected to apply penalties to—certain providers, such as hospitals and professionals, to encourage them to demonstrate meaningful use of certified EHR technology and meet other program requirements. For example, one measure of meaningful use requires providers to implement checks for potential drug interactions with patients' other drugs and allergies.

As mandated by the HITECH Act, GAO (1) assessed the extent of current and expected participation in the EHR programs, (2) examined information reported by providers and others to measure meaningful use in the EHR programs, (3) evaluated HHS efforts to ensure that EHR data can be reliably used to measure quality of care, and (4) evaluated HHS efforts to assess the effect of the EHR programs on program goals related to adoption and meaningful use of EHRs and improved outcomes. GAO analyzed data from CMS and other sources; reviewed applicable statutes, regulations, and guidance; and interviewed officials from HHS and stakeholder groups.

section_title: What GAO Found
paragraphs: Based on the number of providers awarded incentive payments, participation in the Department of Health and Human Services' (HHS) Medicare and Medicaid Electronic Health Record (EHR) programs increased substantially from their first year in 2011 to 2012. For hospitals, participation increased from 45 percent of those eligible for 2011 to 64 percent of those eligible for 2012. For professionals, such as physicians, participation increased from 21 percent of those eligible for 2011 to 48 percent of those eligible for 2012. While increases occurred, a substantial percentage of providers that participated in 2011 did not participate in 2012. Officials who oversee the programs at the Centers for Medicare & Medicaid Services (CMS) noted there could be several reasons for this, such as challenges in demonstrating meaningful use, and are monitoring the issue. Various program changes make future participation difficult to estimate. For example, increased stringency of requirements for the programs' second phase beginning in 2014—Stage 2—may slow participation, while the introduction of penalties in 2015 for some providers may motivate participation.

Reporting on meaningful use for 2011 and 2012 indicates that providers who have already participated in the programs' first phase—Stage 1—used their certified EHR systems more often than required. For example, for both 2011 and 2012, Medicare hospitals reported using computerized provider order entry for over 84 percent of patients—in excess of the required threshold for Stage 1 of 30 percent. However, some meaningful use measures may be more challenging for providers, including measures involving the electronic exchange of information. For example, less than 15 percent of professionals reported on an optional Stage 1 measure to provide a summary of care document at each care transition or referral, which is mandatory in Stage 2. A CMS official said the agency is taking steps to help providers prepare for Stage 2 meaningful use measures.

The lack of a comprehensive strategy limits HHS's ability to ensure the department can reliably use the clinical quality measures (CQM) collected in certified EHRs for quality measurement activities. Reliability issues persist, although CMS and HHS's Office of the National Coordinator for Health Information Technology (ONC) have made efforts to address concerns. For example, different providers may report CQMs based on and tested to different requirements depending on whether their EHRs have incorporated technical updates. Without a comprehensive strategy, efforts to address reliability issues (in accordance with the internal control standard requiring relevant and reliable information) and improve quality and efficiency may be limited.

Consistent with law and GAO guidance on assessing agency performance, HHS, CMS, and ONC have established some performance measures for the EHR programs that are tied to strategic goals regarding adoption and meaningful use of EHRs; however, they have not established measures that would help them to track progress toward program outcomes such as health care quality, efficiency, and patient safety. Although HHS expects that the use of EHRs can help achieve improved outcomes and support other efforts that are also intended to improve care, that result is not yet assured. CMS and ONC may lack critical information necessary to establish program priorities and subsequently make program adjustments based on progress toward outcomes.

section_title: What GAO Recommends
paragraphs: GAO recommends that HHS develop a comprehensive strategy to better ensure the reliability of CQM data collected using EHRs and develop and use outcome-oriented performance measures to monitor progress toward goals. HHS agreed data reliability and performance monitoring are important but neither agreed nor disagreed with GAO's recommendations.

section_title: Letter
paragraphs: The Medicare and Medicaid Electronic Health Record (EHR) programs  began in 2011 with a goal of promoting the adoption and meaningful use  of certified EHR technology. As of 2009, before the programs began,  studies estimated that 78 percent of office-based physicians and   91 percent of hospitals had not adopted EHRs. Such technology has the  potential to improve the quality of care patients receive and to reduce  health care costs, if the technology is used in a way that improves  providers’ and patients’ access to critical information. The Medicare and  Medicaid EHR programs are the largest of the activities, in terms of  potential federal expenditures, and are funded by the Health Information  Technology for Economic and Clinical Health (HITECH) Act, which was  enacted as part of the American Recovery and Reinvestment Act of 2009  and provides funding for various activities to promote the adoption and  meaningful use of certified EHR technology. The Congressional Budget  Office estimated total spending for these activities to be $30 billion from  2011 through 2019.

To increase the adoption and meaningful use of certified EHR technology,  the Medicare and Medicaid EHR programs provide incentive payments  for providers—that is, certain types of hospitals, such as critical access  hospitals, and certain types of professionals, such as physicians and  nurse practitioners. Additionally, beginning in 2015, the Medicare EHR  program is to begin applying a payment adjustment, referred to in this  report as a penalty, for hospitals and professionals that treat Medicare  patients but do not meet the Medicare EHR program requirements. In  establishing the EHR programs, Congress defined “a meaningful EHR  user” as a hospital or professional that meets the following three criteria:  (1) demonstrates use of certified EHR technology in a meaningful  manner; (2) demonstrates that certified EHR technology is connected in a  manner that provides for the electronic exchange of health information to  improve the quality of health care, such as promoting care coordination;  and (3) uses certified EHR technology to submit information on clinical   The EHR programs were  quality measures (CQM) and other measures.designed to last for multiple years to increase, over time, the widespread  adoption and meaningful use of EHRs, which is viewed by the  Department of Health and Human Services (HHS) and others as a  necessary step toward transforming health care into a system that can  achieve goals of improved quality, efficiency, and patient safety. For  example, use of EHRs could improve efficiency by reducing the  duplication of diagnostic tests, or improve patient safety by preventing  medical errors resulting from, for example, incomplete medical histories  for new patients. Additionally, CQM data collected by providers using  EHRs could be used by HHS to measure and improve health care quality  by, for example, designing programs that compensate providers for  meeting quality and efficiency targets—that is, pay for performance.

Within HHS, the Centers for Medicare & Medicaid Services (CMS) and  the Office of the National Coordinator for Health Information Technology  (ONC) have developed the EHR programs’ requirements. CMS is  responsible for administering the Medicare EHR program and overseeing  and funding most of the Medicaid EHR program, which is administered by  the states and U.S. insular areas. ONC is responsible for overseeing the  certification of EHR technology, including establishing technical standards  and certification criteria for it. Additionally, ONC is charged with  formulating the federal government’s health information technology (HIT)  strategy and coordinating federal HIT policies, programs, and  investments.

Pub.L. No. 111-5, § 13424(e), 123 Stat. 278-279. efforts to oversee the Medicare EHR program during its first year as well  as challenges encountered by providers and strategies they used to  participate in the program. We recommended that CMS take steps to  enhance its processes to verify that providers met the requirements to  receive incentive payments. On behalf of CMS, HHS agreed with most of  our recommendations. In July 2012 we reported information on providers  that were awarded Medicare EHR incentive payments for 2011, including  the number of award recipients and their characteristics. In December  2012 we reported information on providers that were awarded Medicaid  EHR incentive payments for 2011, including the number of award  recipients and their characteristics. In October 2013 we reported  information on providers that were awarded Medicare EHR incentive  payments for 2011 and 2012, including the number of award recipients  and their characteristics. Ongoing work is examining the key challenges  to the electronic exchange of health information that have been reported  by providers and stakeholders and HHS’s efforts to address those  challenges. In this report, we (1) assess the extent of current and  expected participation in the EHR programs; (2) examine information  reported by providers and others to measure meaningful use in the EHR  programs; (3) evaluate HHS’s efforts to ensure that EHR data can be  reliably used to measure quality of care; and (4) evaluate HHS’s efforts to  assess the effect of the EHR programs on program goals related to  adoption and meaningful use of EHRs and improved outcomes such as  quality, efficiency, and patient safety. In addition, the HITECH Act  specified that we report on the effect of the act on, among other things,  health insurance premiums, and such information is provided in   appendix I.

To assess the extent of current participation in the Medicare and  Medicaid EHR programs, we analyzed data related to the 2011 and 2012  program years that CMS collected in its National Level Repository  through October 23, 2013. Using these data, we estimated the percentage of eligible providers awarded a Medicare or  Medicaid (or both) EHR incentive payment for 2011 and for 2012; determined the total amount of Medicare and Medicaid EHR incentive  payments awarded to providers each year; and estimated the percentage of providers that were awarded a Medicare  or Medicaid (or, in the case of some hospitals, both) EHR incentive  payment for 2011 but not for 2012.

To assess the extent of expected participation in the EHR programs, we  interviewed CMS officials regarding agency activities to increase  participation and efforts to ensure that providers continue to participate in  the EHR programs after their first year. We also interviewed individuals  from organizations representing stakeholders, including the American  Hospital Association and the College of Healthcare Information  Management Executives (CHIME), whose members include health care  chief information officers; and reviewed an analysis of 2012 survey data  of hospitals collected by the American Hospital Association. In addition,  we reviewed applicable regulations and other relevant agency  documents.

To examine information reported by providers and others to measure  meaningful use in the EHR programs, we analyzed data CMS collected in  its National Level Repository through October 23, 2013. Specifically, we  analyzed data that providers reported to CMS for the 2011 and 2012  program years to demonstrate meaningful use under the Medicare EHR  program—known as meaningful use measures. We analyzed the CMS  data to identify  the extent to which providers exceeded reporting thresholds, which  are specified percentages of patients or actions that providers must  meet or exceed to satisfy the requirements of a subset of the  meaningful use measures (14 hospital measures and 16 professional  measures); and  the frequency with which providers reported meaningful use measures  without claiming exemptions, which are allowed under the program for  some measures if providers meet certain criteria specific to the  individual measures (e.g., the measure is not relevant to their patient  populations or clinical practices, the providers conducted too few  actions to be measured, or the providers were not able to perform the  action).

As part of our analysis, we also reviewed a CMS analysis of meaningful  use measure data submitted by professionals to states to demonstrate  meaningful use in the Medicaid EHR program for the 2012 program year,  reviewed an analysis of 2012 survey data of hospitals collected by the  American Hospital Association, and analyzed 2012 survey data collected  by the National Center for Health Statistics in the Centers for Disease  Control and Prevention from office-based physicians. We identified  meaningful use measures related to electronic health information  exchange based on information obtained from officials from CMS and  ONC. We interviewed officials from CMS, ONC, the American Hospital  Association, the American Medical Association, and CHIME, and  reviewed letters submitted by stakeholder organizations to HHS related to  the EHR programs. In addition, we reviewed regulations and other  relevant agency documentation.

To evaluate HHS’s efforts to ensure that EHR data can be reliably used to  measure quality of care, we reviewed letters submitted by stakeholder  organizations to HHS related to the EHR programs; reviewed the EHR  programs’ requirements and agency documentation describing HHS’s  quality measurement activities, including the Inpatient Prospective  Payment System and the Physician Fee Schedule regulations; reviewed  other materials related to quality measurement; and interviewed officials  from CMS, ONC, the American Hospital Association, and CHIME. We  also reviewed relevant internal control standards regarding information  used by agencies and compared them against HHS activities. We  analyzed CQM data that providers reported to CMS to demonstrate  meaningful use under the Medicare EHR program for the 2011 and 2012  program years and that are captured in CMS’s National Level Repository.  Specifically, we analyzed the extent to which Medicare providers had few  patients who could be included in the calculation of at least one clinical  quality measure.

To ensure the reliability of the various data we analyzed, we interviewed  officials from CMS, reviewed relevant documentation, and conducted  electronic testing to identify missing data and obvious errors. On the basis  of these activities, we determined that the data were sufficiently reliable  for our analysis. (For more information on our data analysis, see app. II.)

To evaluate HHS’s efforts to assess the effect of the EHR programs on  program goals, we reviewed HHS’s Fiscal Years 2010-2015 Strategic  Plan, ONC’s 2011-2015 Federal Health Information Technology Strategic  Plan, and the EHR programs’ requirements to identify department and  agency goals particularly relevant to the EHR programs. We also  reviewed HHS’s Fiscal Year 2014 Annual Performance Report and  Performance Plan, CMS’s Fiscal Year 2014 Budget Justification, and  ONC’s Fiscal Year 2014 Budget Justification to identify department and  agency performance measures relevant to the EHR programs. In  addition, we interviewed CMS and ONC officials regarding performance  measures and program evaluations relevant to the EHR programs. We  assessed this evidence against relevant criteria from the Government  Performance and Results Act Modernization Act of 2010 (GPRAMA) as  incorporated in our guidance on assessing performance and our guidance  on designing evaluations.

We conducted this performance audit from February 2013 through March  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Incentive Payments and Penalties
paragraphs: Through use of incentive payments and penalties, the EHR programs are  intended to help address the significant barriers to the adoption and use  of EHRs, such as cost, and the technical challenges associated with their  use. Beginning in 2011, the first year of the Medicare and Medicaid EHR  programs, the programs have provided incentive payments to  participating providers, that is, eligible providers that met program  requirements.applying a penalty to hospitals and professionals that do not meet the  Medicare EHR program requirements. There is no statutory end-point  provided for the penalties. The Medicaid EHR program does not impose  penalties on Medicaid providers that do not meet the Medicaid EHR  program’s requirements by a specific date; however, if Medicaid providers  also treat Medicare patients, they are required to meet the Medicare EHR  program’s requirements from 2015 onward to avoid penalties from the  Medicare EHR program. (See fig. 1.)

Beginning in 2015, CMS is generally required to begin   In the Medicaid EHR program, professionals cannot earn more  Beginning in 2015, CMS will apply penalties to Medicare providers if they  did not meet Medicare EHR program requirements.  For most hospitals,  penalties will be applied by reducing the Medicare Inpatient Prospective  Payment System payment rate increase by 25 percent each year, with a   However, for critical  maximum cumulative reduction of 75 percent.access hospitals, the penalty is applicable to the hospital’s Medicare  reimbursement for inpatient services. For professionals, penalties will be  assessed by reducing the reimbursement that the provider would  ordinarily receive for furnishing Medicare Part B services by 1 percent for  each year the professional did not meet the requirements, with a  cumulative penalty of up to 3 percent per year. CMS may exempt  hospitals and professionals from penalties if it determines that complying  with the program requirements would result in a significant hardship to the  provider; such exemptions are to be granted on a case-by-case basis  subject to annual renewal. Examples of permissible hardship  circumstances include lacking necessary infrastructure, such as internet  broadband; facing unforeseen circumstances such as a natural disaster;  or, for professionals, lacking patient interaction.

section_title: Medicare EHR Program and Medicaid EHR Program Requirements
paragraphs: CMS has established requirements for the EHR programs that specify the  providers eligible to participate—that is, providers that are permitted to  earn incentive payments or that may be subject to penalties. For  example, providers must be a permissible provider type, such as an acute  care hospital or a doctor of medicine and, for the Medicaid EHR program,  providers must generally meet a patient volume requirement to be  eligible.

In addition to meeting program eligibility requirements, to receive  incentive payments or avoid penalties, eligible providers must also satisfy  reporting requirements by submitting information to CMS for the Medicare  EHR program, to the states for the Medicaid EHR program, or to both.  The reporting requirements generally incorporate the three statutory  criteria for “meaningful use” established in HITECH—(1) demonstrate use  of certified EHR technology in a meaningful manner; (2) demonstrate that  certified EHR technology is connected in a manner that provides for the  electronic exchange of health information; and (3) submit information on  CQMs using certified EHR technology. To receive incentive payments  from the Medicare EHR program, the information reported by providers  must satisfy all three statutory criteria; that is, providers must  “demonstrate meaningful use.” However, to receive incentive payments  from the Medicaid EHR program in their first year of participation,  providers need not satisfy the three statutory criteria. Instead, they must  only report that they adopted, implemented, or upgraded to certified EHR  technology. In subsequent years, though, they must demonstrate  meaningful use to receive incentive payments.

Certified EHR technology. Certified EHR technology is technology that  has been determined by ONC-authorized organizations—referred to as  certification bodies and testing laboratories—to conform to the standards  and certification criteria developed by ONC. ONC has changed the  certification criteria since the EHR programs began. Specifically,  providers who participated in the EHR programs from 2011 through 2013  were generally required to use 2011-edition EHRs and all providers who  participate in the EHR programs from 2014 through 2016—regardless of  whether the provider participated in the program in earlier years—are  required to use 2014-edition EHRs. another set of certification criteria for EHRs that providers would be  required to use beginning in 2017.

In this report, when we use the term EHR, we are generally referring to certified EHR  technology. different years. This approach is consistent with the HITECH Act, which  directed the Secretary of HHS to seek to improve health care quality and  the use of EHRs by requiring more stringent requirements over time.  According to CMS, using the staged approach is intended to help  overcome two important deficiencies—the lack of widespread use of  capable EHRs and the lack of an HIT infrastructure—that existed when  the requirements were being developed, with the hope that those  deficiencies would be addressed over time as a result of the EHR  programs. The stage that is applicable to each provider in each year is  based on the first year in which the provider demonstrated meaningful  use in the Medicare EHR program or the Medicaid EHR program. For  example, a Medicare professional that first demonstrated meaningful use  in 2011 must report the Stage 1 meaningful use measures for 2011,  2012, and 2013 to receive incentive payments for those years and must  report the Stage 2 meaningful use measures beginning in 2014 to receive   The Stage 3 requirements have not  an incentive payment for that year.yet been developed but are expected to be finalized in 2015 and to apply  beginning in 2017. (See table 1.)

The meaningful use measures in Stage 1 are intended to promote the  electronic capture of health information in a structured format and to  encourage providers to use that information to track key clinical  conditions. The Stage 2 meaningful use measures were chosen to  encourage continuous quality improvement at the point of care and are  more stringent than the Stage 1 measures. For example, measures that  were formerly optional are now mandatory, and some measures require  that the provider perform the action for a greater percentage of patients or  implement more interventions. Additionally, ONC and CMS have stated  that Stage 2 places a stronger emphasis on electronic health information  exchange with other providers compared to Stage 1.intention to develop even more stringent meaningful use measures for  Stage 3 that would promote activities that can lead to improved health  outcomes, such as using decision support tools or providing patient  access to self-management tools.

Table 2 provides information on the number and type of measures that  providers must report for Stage 1 and Stage 2.

Clinical quality measures (CQM). Health care quality measures, also  known as CQMs, can be used to assess provider performance and to  drive quality improvement and accountability. For example, one CQM for  professionals measures the use of high-risk medications in the elderly.  CQMs are composed of a number of clinical data elements, or pieces of  data, that must be collected in order to determine performance on any  given measure. Using those data elements, a score for the measure can  then be calculated. In order to demonstrate meaningful use in the Medicare or Medicaid EHR programs, providers are required to collect  CQMs electronically using EHRs and report those data to CMS  (Medicare), the states (Medicaid), or both. However, historically, CQMs  have been collected through other data collection methods, such as  through a detailed, manual review of paper medical records, a process  referred to as chart abstraction. Most of the CQMs identified by CMS for  use in the EHR programs were originally developed for paper medical  records or other data collection methods and later modified so they could  be collected using EHRs.

The CQMs that providers must collect using EHRs and report to  demonstrate meaningful use are determined by the edition of certified  EHRs providers must use. Specifically, using their 2011-edition certified  EHRs that applied from 2011 to 2013, hospitals were required to report all  15 CQMs identified by CMS, and professionals were required to report   6 CQMs from a list of 44 measures. EHRs that will apply from 2014 to 2016, hospitals are required to report  16 CQMs from a list of 29 measures, and professionals are required to  report 9 CQMs from a list of 64 measures. Medicare providers are being  encouraged to electronically report the results of CQMs to CMS, but they  have the option of reporting CQMs by manually entering information into  CMS’s web-based tools, a process known as attestation.

Professionals had to report 3 core or alternative core measures and had to report an  additional 3 measures from the list of measures.

section_title: HHS’s Quality Measurement Activities
paragraphs: In addition to the EHR programs, HHS requires providers to collect and  report CQMs for a number of other programs in order to measure health  care quality, some of which provide financial incentives to health care  providers to help achieve the goals of improved quality and efficiency. For  example, CMS is to impose penalties on providers who do not  satisfactorily report quality data to the Hospital Inpatient Quality Reporting  (IQR) program and to the Physician Quality Reporting System (PQRS).  The Patient Protection and Affordable Care Act directed HHS to  implement new programs that will use CQMs, including pay-for- performance programs such as the Physician Value Based Payment  Modifier program, which bases Medicare payments in part on CQM  results, and the Medicare Shared Savings Program, which rewards  Accountable Care Organizations that lower growth in health care costs  while meeting performance standards.

Across its programs, HHS has a goal to use CQMs that are broad based,  patient centered, and prioritized based on their potential to achieve  population-wide improvements. In order to do this in its programs, CMS  has set a goal of collecting and submitting data using EHRs, which HHS  officials have indicated can be used to collect CQMs as a byproduct of  the routine delivery of health care. To minimize the burden on providers of  collecting and reporting similar CQMs for different CMS programs using  differing methods, CMS has modified some of its programs’ requirements  such that providers may submit data collected through EHRs as part of  the Medicare or Medicaid EHR programs instead of separately collecting  and submitting data through other methods such as chart abstraction. For  example,

Hospitals electronically reporting to CMS one-quarter of CQM data  collected in 2014 using 2014-edition EHRs for 16 CQMs will satisfy  the IQR program’s four-quarter reporting requirement for those 16  measures for 2016 payment determinations and the Medicare EHR  program’s CQM reporting requirement for 2014.

Professionals electronically reporting one year of CQM data  generated from 2014-edition EHRs to CMS for PQRS in 2014 will  satisfy the CQM reporting requirements for both PQRS and the  Medicare EHR program. CMS will use those data for professionals  in practices of 10 or more individuals to generate the Physician Value  Based Payment Modifier that will be applied to payments made under  the Medicare physician fee schedule starting January 1, 2016.

section_title: Performance Assessment
paragraphs: GPRAMA requires federal agencies to prepare performance reports and  establish performance measures to assess performance of federal  programs. Performance measures assess performance via ongoing  monitoring and reporting of program accomplishments, which include  progress toward pre-established goals. Our previous work notes that  performance measures can serve as an early warning system to  management and as a vehicle for improving accountability to the public.  We have also published guidance on assessing performance that  incorporates GPRAMA criteria and which states that it is important for  performance measures to be tied to program goals and for agencies to  ensure their activities support their organizational missions and move  them closer to accomplishing their strategic goals. In addition, our  guidance to federal agencies on designing evaluations suggests that  performance measures should include both process and outcome  measures (see table 3). Outcome measures are particularly useful in  assessing the status of program operations, identifying areas that need  improvement, and ensuring accountability for end results. Furthermore,  our guidance on assessing performance notes that leading organizations  not only establish performance measures but also use information from  these performance measures to continuously improve processes, identify  program priorities, and set improvement goals.

section_title: Participation in the EHR Programs Has Increased, but Program Changes Make It Difficult to Estimate Future Participation
paragraphs: Participation in CMS’s EHR programs increased substantially from 2011  to 2012, but some providers who participated in 2011 did not continue in  2012. It is difficult to estimate future participation in the EHR programs  because of various program changes, including the planned increase in  stringency of the meaningful use measures, the introduction of penalties  for some providers in 2015, CMS’s efforts to increase participation among  certain providers, and changes to eligibility requirements.
subsections: section_title: Participation in 2012 Increased Even Though Some Providers Did Not Continue after 2011
paragraphs: Participation in CMS’s EHR programs increased substantially from 2011  to 2012. For hospitals, participation increased from 45 percent of those  eligible for 2011 to 64 percent of those eligible for 2012. For  professionals, participation increased from 21 percent of those eligible for  2011 to 48 percent of those eligible for 2012. (See fig. 2.)

The amount CMS paid in incentive payments to providers for participating  in the Medicare EHR program, Medicaid EHR program, or both programs  increased from $5.2 billion for 2011 to $9.5 billion for 2012. In total,  CMS paid $14.7 billion in incentive payments to providers for 2011 and  2012. Of this total, hospitals received nearly $8.6 billion and  professionals received almost $6.1 billion. Incentive payments for  hospitals and professionals under the Medicare EHR program amounted  to $8.8 billion, while incentive payments for hospitals and professionals  under the Medicaid EHR program were $5.9 billion. (See fig. 3.)

Although participation in the EHR programs has increased overall, a  substantial percentage of providers that participated in the Medicare EHR  program or Medicaid EHR program in 2011 did not participate in either  program in 2012. (See fig. 4.) Specifically, within the 36 states that had  completed their determinations of which providers would receive incentive  payments for the 2012 Medicaid EHR program year, 61 percent of  professionals and 36 percent of hospitals that participated in the Medicaid  EHR program in 2011 did not continue in 2012. Sixteen percent of  professionals and 10 percent of hospitals participating in the Medicare  EHR program in 2011 did not continue to participate in 2012.

CMS officials are aware that some providers participating in 2011 did not  continue in 2012 and told us that they are monitoring the issue and taking  steps to reverse this trend. One CMS official told us there are various  possible reasons Medicare and Medicaid providers did not continue to  participate in the EHR programs. Noteworthy for the Medicaid EHR  program, and in contrast to the Medicare EHR program, providers do not  need to participate in consecutive years to maximize their incentive  payments, and there are no penalties for not participating. Another  possible reason providers did not continue to participate in the Medicaid  EHR program in 2012 is that providers are not required to demonstrate  meaningful use for their first year of participation, but must do so for their  second year. One CMS official noted that a provider who received an  incentive payment for adopting, implementing, or upgrading to a certified  EHR could still be far from having the capability to demonstrate  meaningful use. The CMS official also noted that the agency conducted a  survey of a sample of providers to learn why they did not continue to  participate in the EHR programs and found the following additional  reasons:  some providers did not realize they needed to participate in the  program again;  some providers did not know the deadline for submitting the required  information to CMS for the Medicare EHR program, to the states for  the Medicaid EHR program, or to both;  some providers switched EHR vendors and were not ready in time to  submit the required information; and  some providers found it more difficult than anticipated to go from a   90-day reporting period to a full-year reporting period, as required for  the second year of participation.

In an effort to reverse this trend, a CMS official told us CMS is working to  ensure providers are well-informed regarding program requirements and  data submission deadlines. For example, the official told us that CMS is  emphasizing requirements and deadlines at meetings with providers,  modifying educational materials, and reorganizing the agency’s website  devoted to the EHR programs.

section_title: Program Changes, Including the Increasing Stringency of Requirements and the Implementation of Penalties, Make It Difficult to Estimate Future Participation
paragraphs: Various program changes may affect participation in the EHR programs in  the future, making future participation difficult to estimate. Beginning in  2014, Stage 2 requirements for demonstrating meaningful use will be  introduced for many providers participating in the EHR programs,  requiring providers to demonstrate that they are using their systems in  more advanced ways. The increased stringency of these Stage 2  requirements for demonstrating meaningful use may slow participation in  the EHR programs. While CMS and ONC have developed guidance  materials and trainings to help providers prepare for Stage 2, survey data  and statements by organizations representing providers suggest that  some providers may be much less capable of demonstrating Stage 2  meaningful use, compared to Stage 1 meaningful use, which might result  in reduced participation.

Conversely, other factors may encourage future participation. Penalties  for Medicare providers not participating in the Medicare EHR program  begin in 2015 and may motivate providers to begin participating in the  program or to continue participating in the program. In addition, CMS and ONC have identified certain provider groups as having more difficulty  participating in the program and are taking steps to increase participation  among these providers. For example, CMS officials told us the agency  has partnered with the American Association of Pediatricians to identify  states in need of more outreach to encourage pediatricians to participate  in the Medicaid EHR program, and ONC’s Regional Extension Center  program has provided assistance to providers such as professionals in  solo practices or rural areas to help them participate in the EHR  programs. As we previously reported, participation in the Medicare EHR  program among rural hospitals and professionals in solo practices  increased faster from 2011 to 2012 than participation by hospitals and  professionals overall. Furthermore, CMS has made changes to certain  program eligibility requirements that may increase the number of  providers participating. For example, beginning in 2014, professionals  that are hospital-based may receive incentive payments if they can  demonstrate that they fund the acquisition, implementation, and  maintenance of EHRs in lieu of using a hospital’s EHRs.these professionals were ineligible to participate in the EHR programs.

section_title: Providers Exceeded Meaningful Use Requirements for Most Stage 1 Measures, but May Face Challenges with Certain More Stringent Stage 2 Measures
paragraphs: The meaningful use measures reported by providers for 2011 and 2012  indicate that providers used their certified EHR systems more often than  required and suggest that many providers who have already reported  certain Stage 1 measures will be able to meet most of the more stringent  reporting thresholds for similar measures in Stage 2. (See app. IV,   table 6.) For 2011 and 2012, Medicare providers that received EHR  incentive payments consistently exceeded most meaningful use measure  reporting thresholds—that is, a specified percentage of patients or actions  required to satisfy the measure—established by CMS and ONC for the   Of the 14 hospital meaningful use measures that had  EHR programs.thresholds, on average, Medicare hospitals exceeded thresholds by at  least 10 percentage points for all measures and by considerably more for  many measures. For example, hospitals are required to use computerized  provider order entry for more than 30 percent of patients with at least one  medication for Stage 1. For both 2011 and 2012, Medicare hospitals  reported using computerized provider order entry for medication orders  for over 84 percent of patients—about 2.8 times the required threshold for  Stage 1. Medicare professionals also greatly exceeded the reporting  thresholds. We did not conduct an analysis of meaningful use measures  reported by Medicaid providers, but there is some evidence that Medicaid  professionals also consistently exceeded required thresholds for 2012.  While providers that have not participated in the EHR programs may have  different experiences, the fact that Medicare providers and Medicaid  professionals that participated in 2011 or 2012—the first two years of  Stage 1—generally exceeded most reporting thresholds by wide margins  for the measures they reported suggests that they will be able to meet  most of the more stringent reporting thresholds in Stage 2. For example,  in Stage 2, providers are required to use their certified EHRs to record  and chart changes in vital signs for more than 80 percent of patients.  Medicare hospitals and professionals exceeded this threshold for both  2011 and 2012, reporting that they met this measure for over 90 percent  of patients.

In addition to consistently exceeding most required reporting thresholds,  other analyses identify some measures that were frequently reported by   (See app. IV, table 7.) For example, close to   providers in Stage 1.100 percent of providers for 2011 and 2012 reported the mandatory  Stage 1 measure to record smoking status for patients 13 years and older  without claiming allowable exemptions. Certain optional measures,  including one related to the electronic exchange of health information,  were also frequently reported. For example, over 80 percent of hospitals  reported incorporating clinical lab test results into EHRs as structured  data.

Other measures not related to the electronic exchange of information may also be  problematic for providers. For example, fewer than 25 percent of Medicare professionals  in 2011 and 2012 reported sending patient reminders for preventive or follow-up care. This  measure is optional for professionals in Stage 1 and mandatory for professionals in   Stage 2. addition, these measures become more stringent in Stage 2, requiring  ongoing submission of these data rather than just a one-time test of  the ability to submit those data, which Stage 1 required.

Perform medication reconciliation for patients received from  another setting of care or provider of care. Fewer than 30 percent  of Medicare hospitals reported this measure for 2011 and 2012. This  measure is optional in Stage 1 and mandatory in Stage 2.

Provide patients with their health information electronically. Less  than 35 percent of Medicare hospitals and professionals for both 2011  and 2012 reported the mandatory measure to provide patients with an  electronic copy of their health information. Data submitted by the  providers that reported this measure indicate that very few patients— often fewer than 4 patients per year for hospitals and 12 patients per  year for professionals—requested this electronic information.  Additional evidence suggests that this measure was reported by a  similar proportion of Medicaid professionals for 2012.measure becomes more stringent, by requiring that 5 percent of  patients electronically view, download, or transmit their health  information to a third party, in addition to requiring hospitals and  professionals to provide patients with online access to their health  information.

In Stage 2, the  Other information also indicates that providers may face challenges with  many of the same exchange-related Stage 2 meaningful use measures  identified above. An analysis of the American Hospital Association’s 2012  survey of hospitals about technical capabilities related to meaningful use  indicates that the least commonly implemented Stage 2 capabilities by  acute care hospitals were those related to the electronic exchange of  information.electronic data on reportable lab results to public health agencies;  submitting electronic syndromic surveillance data to public health  agencies; providing an electronic summary of care document for each  transition of care or referral; and providing patients with electronic access  to their health information by enabling them to view online, download, and  transmit such information. These capabilities are related to optional   Stage 1 measures but will be required in Stage 2 or become more  complex in Stage 2. An analysis of the National Center for Health  Statistics’ 2012 National Ambulatory Medical Care Survey shows similar  results for physicians. Among the capabilities related to meaningful use  measures, office-based physicians were least likely to have routinely  submitted electronic data to immunization registries or information  systems, or to have routinely communicated with patients using secure  messaging. Stage 2 requires professionals to report measures that  capture these capabilities, whereas reporting on the first capability was  optional in Stage 1 and the second was not a Stage 1 measure.

Such capabilities included the following: submitting  Consistent with indications from the survey data, stakeholder  organizations have also noted concern that providers may have difficulty  satisfying certain Stage 2 meaningful use measures due to the changes  made to them for Stage 2. Table 4 lists meaningful use measures that  organizations identified as challenging—two of which relate to the  electronic exchange of information—and includes the requirements for  satisfying each measure in Stage 1 and in Stage 2.

A CMS official noted that although providers have expressed concern  about their ability to meet the Stage 2 requirements, similar concerns  existed prior to Stage 1 regarding the thresholds established for certain  measures and the data suggest that providers have exceeded thresholds  established by CMS for the Stage 1 requirements. This official also  explained efforts to help providers to prepare to satisfy the Stage 2  meaningful use measures. CMS has been modifying its materials and  holding training sessions and discussions with various provider groups. In  anticipation of obstacles to the exchange of health information and in  order to accelerate electronic exchange even among providers not  eligible for incentive payments from the EHR programs, ONC and CMS  issued a request for information in March 2013 to, among other things,  solicit feedback on the challenges to the electronic exchange of health  information and on possible mechanisms to overcome those challenges.  Using the information collected, CMS and ONC issued a strategy in  August 2013 for how they expect to advance health information  exchange, which includes various steps the agencies plan to take under  three broad categories—accelerating health information exchange,  advancing standards and interoperability, and patient engagement.Ongoing work is examining the key challenges to the electronic exchange  of health information that have been reported by providers and  stakeholders and HHS’s efforts to address those challenges.

section_title: HHS Lacks a Comprehensive Strategy to Help Ensure Reliability of EHR Data Collected to Improve Quality of Care
paragraphs: The lack of a comprehensive strategy limits HHS’s ability to ensure that  the department can reliably use the CQMs collected in certified EHRs for  quality measurement activities. According to internal control standards, an  agency must have relevant and reliable information to help it achieve its  mission and goals. In order to drive quality improvement and  accountability and to effectively utilize CQM data collected in EHRs for  HHS’s quality measurement activities, the data collected must be reliable,  that is, both accurate and complete. Data that are not reliable limit the  credibility of the information and may present a risk to individuals or  entities making decisions based on the data, such as to a patient  choosing a hospital for treatment, or to CMS when applying a payment  modifier resulting in an increase or reduction to a physician’s  reimbursement.

CMS and ONC have made changes to the specifications, certification  criteria, and certification program to address some of the reliability  concerns with the CQMs that providers were generally required to collect  and report from 2011 to 2013 to demonstrate meaningful use (that is,  using 2011-edition EHRs). These changes are incorporated into EHRs  that will be used by participating providers in all stages of meaningful use  from 2014 to 2016 (that is, 2014-edition EHRs). The changes made as a  result of the concerns identified include the following.

Specifications. CMS made changes to CQM specifications—that is, the  documentation that describes to providers, technology developers, and  others how data on CQMs were to be collected through EHRs. These  changes were made to correct previously identified errors and to help  ensure the feasibility of implementing CQMs in the electronic format—that  is, to ensure that the required data are readily available or can be  collected in EHRs without undue burden. Problems with feasibility can  compromise the accuracy or completeness of the data.

Certification criteria. To address providers’ concern that their 2011- edition EHRs did not collect all of the data elements necessary to  calculate the CQMs, ONC modified the CQM-related certification criteria.

Specifically, 2014-edition EHRs must collect each of the data elements  associated with a specific CQM to be certified for that measure.

Certification program. The program for certifying 2011-edition EHRs did  not test whether the EHR system correctly collected and calculated the  CQMs. However, beginning with the certification of 2014-edition EHRs,  the certification bodies and testing laboratories will test whether EHRs  accurately collect and calculate CQMs. These tests will be performed  using a certification testing tool developed by ONC, known as Cypress. Further, beginning in 2014, the certification bodies will be required by  ONC to conduct specific annual surveillance activities to determine  whether 2014-edition EHRs are functioning as intended after being  implemented by providers. CQMs have been identified as a priority area  for these surveillance activities, which may help to inform CMS and ONC  on whether the changes they have made to the specifications,  certification criteria, and certification program are improving the reliability  of CQMs, as intended, and whether further changes are needed.

CMS and ONC officials indicated the changes made thus far should help  to address reliability concerns, but provider experience with 2014-edition  EHRs will be an important gauge of the effectiveness of the changes.  Furthermore, there are still issues that could result in reliability problems,  hindering CMS’s ability to use the data for its quality measurement  activities. For example, after EHRs are certified as 2014-edition EHRs,  the agencies do not require them to be recertified following updates to the  specifications, which may be made annually or more frequently, or  following updates to the certification testing tool.

Testing bodies use the certification testing tool—Cypress—to test that EHRs accurately  export, calculate, and electronically submit the data. This tool includes synthetic patient  data used in the testing process. Vendors also have access to the testing tool prior to  submitting their EHR products for certification. providers may report CQMs based on and tested to different  requirements. CMS has taken some steps to mitigate the potential effect  of these issues on the reliability of CQM data submitted by professionals  who electronically submit CQM data by requiring them to report CQMs  collected using EHR technology based on and tested to the most recent  version of the electronic specifications. However, this variability may limit  the ability to reliably compare quality across other providers, including  hospitals, and undermines efforts CMS officials said they are taking to  ensure that the CQMs lead to standardized, consistent data reporting.  Another issue that may affect reliability is that many providers reported at  least one CQM based on few patients, which may result in data that are  not statistically reliable and therefore should not be used in quality  measurement. We analyzed data submitted by providers to demonstrate  meaningful use under the Medicare EHR program for 2011 and 2012 and  found that 90 percent of hospitals and 50 percent of professionals  reported at least one CQM based on few patients for 2012, a slight  increase compared to 2011, when 86 percent of hospitals and 46 percent  of professionals reported at least one CQM based on few patients.

In addition, stakeholder organizations continue to have concerns about  reliability. CHIME members indicated that they do not think the  specifications required to generate CQMs using 2014-edition EHRs will  substantially improve the reliability of the data collected in Stage 2. They  noted that the collection of CQM data includes data elements not  currently captured by EHRs, and the data elements are not automatically  part of most clinicians’ workflow (that is, incorporated into the tasks  performed during the delivery of routine care). Due to these obstacles,  CHIME members are concerned that clinicians will not consistently collect  the required data elements, ultimately leading to problems with CQM  reliability and comparability. CMS and ONC are making changes to the  process used to develop new measures, which includes greater  consideration of clinician workflow and provider burden. However, new  CQMs will not be available until the beginning of Stage 3 in 2017.

Another concern expressed by stakeholder organizations such as the  American Health Information Management Association, the American  Hospital Association, the American Medical Association, the Healthcare  Information and Management Systems Society, and The Joint  Commission is that the CQMs generated from EHRs—which are based  on electronically specified measures and are extracted automatically from  EHRs—do not produce results that are comparable to corresponding  measures based on data obtained through other methods, such as  through manual abstraction of patient medical records by trained  professionals. In its fiscal year 2014 Inpatient Prospective Payment  System final rule, CMS noted that it intends to use the electronic CQM  data submitted voluntarily by hospitals to assess the differences in  performance rates from CQMs collected using EHRs and CQMs collected  through chart-abstraction. However, CMS did not indicate when this  study would be completed or describe how it would be designed.  Designing the study to control for any other systematic differences  between providers that use EHRs and other providers could have an  important effect on CMS’s ability to understand the extent and  implications of potential differences in performance rates.

Although CMS officials told us that they were beginning to develop an  approach to validate CQM data generated from EHRs, HHS has not yet  developed a comprehensive strategy to address concerns with the  reliability of CQMs collected using certified EHRs. Consistent with  GPRAMA and our prior work, which identifies several important practices  that can guide agencies in planning and implementing an effective  government program, a comprehensive strategy would establish  objectives, steps to achieve results, priorities, and milestones and identify  the steps necessary to achieve desired results. Addressing the concerns  is important to ensure that the CQM data can be reliably used for CMS’s  quality measurement activities as well as providers’ quality improvement  activities. For example, CMS plans to use CQMs collected and submitted  via EHRs by certain professionals in 2014 to determine which providers  will be subject to payment modifications under the Value Based Payment  Modifier program in 2016. For hospitals, CMS officials told us that as part  of their approach to validating data, they plan to use electronically  submitted CQMs to develop monitoring processes to help inform them  about the consistency and reliability of the data reported. Also for  hospitals, CMS officials told us that they expect to introduce a process  that is as robust as the one that is currently used for the IQR program  under which a CMS contractor conducts an independent analysis to verify  that the data submitted accurately reflect the information in the medical  record. CMS officials told us that the agency’s plans for validating CQM  data are preliminary, but the agency has stated that it intends to develop  and propose a strategy for hospital data in 2014 as part of the agency’s  annual rulemaking process. While the IQR program may provide a model  for validating data collected by hospitals using certified EHRs, no  comparable model or process exists that could be considered when  developing a validation strategy for data collected by professionals using  certified EHRs. Until HHS establishes and implements a comprehensive  strategy to ensure the reliability of CQMs collected using certified EHRs, it  will be unclear whether the department’s plans are sufficient to address  the concerns, and therefore it will be uncertain when the CQM data can  be reliably used to help assess provider performance, improve quality,  and adjust provider payments.

section_title: Performance Measures for the EHR Programs Do Not Assess Outcomes Such As Improved Quality, Efficiency, and Patient Safety
paragraphs: HHS, CMS, and ONC have established some performance measures for  the EHR programs that are tied to strategic goals, but have not  established performance measures for the goals most relevant to the  intended outcomes of the EHR programs—that is, goals related to  improving health care quality, efficiency, and patient safety. Establishing  performance measures tied to goals is important for ensuring that  agencies’ activities move them closer to accomplishing these goals,  according to our guidance to federal agencies on effectively implementing  GPRAMA, which describes leading practices for how federal agencies  should assess their performance. Our guidance also notes that  agencies should use information that performance measures provide to  improve processes and set improvement goals. However, as the  agencies have not established performance measures to assess  outcomes, they are therefore unable to use the information such  measures would provide.

In addition, the Centers for Disease Control and Prevention has 3 performance  measures related to the meaningful use of EHRs intended to measure the readiness of  public health agencies to receive electronically submitted data as part of the EHR  programs. The Centers for Disease Control and Prevention provides direct assistance to  state public health agencies to support electronic health information systems to receive  data from providers.

GAO/GGD-96-118 (Washington, D.C.: June 1, 1996). the percentage of nonfederal acute care hospitals that are electronically  sharing summary of care records with providers outside their  organization. (See app. V, table 8 for a list of the department- and  agency-level strategic goals and 26 associated performance measures  for this category.)

For these established performance measures, the agencies have  developed corresponding performance targets and track results against  those targets. For example, ONC has set a fiscal year 2014 target that   65 percent of nonfederal acute hospitals electronically share summary of  care records with providers outside of their organization. These  performance targets are publicly released alongside the performance  measures and actual results in the annual ONC and CMS budget  justifications and the HHS online performance appendix. The agencies  told us they use this performance information to set the broader strategic  direction for the agency and may also use the information to make  changes to the programs to improve performance.

However, although HHS, CMS, and ONC have established important  performance measures for the goals related to adoption and meaningful  use of EHRs, they have not established measures linked to the second  category of goals, which would help them to track program outcomes  such as health care quality, efficiency, and patient safety. (See app. V,  table 9 for a list of the department- and agency-level strategic goals for  this category.) Outcome measures would enable the agencies to measure  whether the EHR programs’ intended effects are being achieved, rather  than just the number of providers that satisfied the requirements  necessary to demonstrate meaningful use. ONC’s strategic plan states  that the meaningful use of EHRs—which, as explained above, the  agencies track using several performance measures—is a necessary  step to improve health care quality, efficiency, and patient safety.  However, meaningful use is not sufficient to achieve these outcomes on  its own. The plan notes that the EHR programs are an HIT investment  made in lockstep with various other reform efforts such as patient- centered medical homes and that EHRs will contribute to improvements  in health care quality, efficiency, and patient safety when used in support  of these other reform efforts. Therefore, outcome measures—and the  subsequent use of the information they provide—could be beneficial to  evaluating how the EHR programs interact with other reform efforts.  Furthermore, ONC has acknowledged the need to develop performance  measures to track patient safety, in part due to concerns that EHRs could  have some unintended consequences that cause patient harm. However,  ONC has not yet developed these potential measures.

Without performance measures linked to the goals of improved health  care quality, efficiency, and patient safety, CMS and ONC are limited in  their abilities to determine the effectiveness of the EHR programs in  improving outcomes. CMS and ONC may lack critical information  necessary to establish program priorities and subsequently make  program adjustments based on progress toward these outcomes. CMS  and ONC officials have indicated that they do not expect to observe  progress toward achieving the intended outcomes of improved health  care quality, efficiency, and patient safety until at least Stage 3 of the  EHR programs, which is not expected until 2017. However, establishing  outcome-oriented performance measures before results are expected is a  valuable practice to ensure that the agencies can make program  adjustments as needed and are prepared to monitor outcomes and to  establish baseline values, which can be useful for developing  performance targets and assessing progress toward goals. Moreover,  consistent with our guidance, outcome measures could be useful to guide  the development and prioritization of Stage 3 requirements, which the  agencies anticipate undertaking in 2014.

While the agencies have not established outcome-related performance  measures, agency officials told us they are conducting some evaluations  to assess the effect of the EHR programs on outcomes. For example,  ONC officials told us that CMS, ONC, and the Agency for Healthcare  Research and Quality were working on a study to assess the effect of  electronic prescribing on quality and efficiency using Medicare Part D  claims, specifically, to determine whether electronic prescribing for  diabetes patients is associated with a reduced risk of adverse drug  events. (See app. VI for a summary of program evaluations assessing  the EHR programs’ effects on outcomes.) However, these program  evaluations—in contrast to performance measures—capture only a  specific period of time, and, as a result, do not allow for ongoing  monitoring of the EHR programs’ effectiveness.

section_title: Conclusions
paragraphs: The EHR programs were intended to be long-term solutions—with  multiple stages implemented over several years—to help address  longstanding, significant barriers to greater use of EHRs as tools for  improving outcomes such as health care quality, efficiency, and patient  safety. The first stage of the EHR programs, from 2011 to 2012, has  shown sizeable increases in the number of providers who have adopted  and meaningfully used EHRs. This early measure of implementation still  leaves substantial room to further increase meaningful use among certain  providers. Implementation of the subsequent stages, which will  increasingly emphasize the use of more advanced features of EHRs, will  place more demands on providers through more stringent requirements,  but could also increase the potential for achieving better outcomes.  Because of the programs’ design to evolve over multiple years and  stages, HHS has opportunities to make adjustments and closely monitor  the programs to ensure that they remain on track to achieve their  intended results.

One key feature of EHRs that shows a need for adjustment relates to the  CQMs. We describe several issues regarding the reliability of CQM  data—for example, providers may be reporting CQMs based on and  tested to different specifications. The reliability of CQM data collected  using certified EHRs is especially important due to the potential risk of  making decisions—such as paying providers for performance—based on  unreliable CQM data. CMS and ONC lack a comprehensive strategy that  would allow them to ensure the reliability of CQM data collected using  EHRs, in accordance with the internal control standard requiring an  agency to have relevant and reliable information to achieve its mission  and goals. Without a comprehensive strategy, the agencies may be  limited in their abilities to reliably compare quality across providers or use  CQM data collected through EHRs for a variety of quality measurement  and pay-for-performance activities, thus restricting efforts to improve  quality and efficiency.

Another area that merits attention relates to HHS’s ability to ensure that  the EHR programs are on track to meet their goals. While the EHR  programs are ultimately intended to improve outcomes such as health  care quality, efficiency, and patient safety, the agencies have not  established performance measures for monitoring progress toward  achieving these improvements. Although HHS expects that EHRs can  help achieve improved outcomes as well as support various other health  care reform efforts that are also intended to improve care, that result is  not yet assured. As established by our guidance to federal agencies,  monitoring progress toward specific outcomes could allow HHS to make  adjustments to the EHR programs throughout their implementation, such  as adjusting priorities or requirements. Without such monitoring, the  agencies will not be well positioned to understand how challenges  affecting key parts of the programs, such as ensuring the reliability of  CQM data or increasing providers’ abilities to exchange health  information, might be affecting the programs’ overall ability to achieve  their goals, reducing HHS’s ability to make course corrections.

section_title: Recommendations for Executive Action
paragraphs: To more effectively use CQMs to assess provider performance, the  Secretary of Health and Human Services should direct CMS and ONC to  take the following action:

Develop a comprehensive strategy for ensuring that CQM data  collected and reported using certified EHR technology are reliable,  including testing for and mitigation of reliability issues arising from  variance in certified EHR systems tested to different CQM  specifications.

To ensure that CMS and ONC can effectively monitor the effect of the  EHR programs and progress made toward goals, the Secretary of Health  and Human Services should direct the agencies to take the following two  actions:

Develop performance measures to assess outcomes of the EHR  programs—including any effects on health care quality, efficiency, and  patient safety and other health care reform efforts that are intended to  work toward similar outcomes—and

Use the information these performance measures provide to make  program adjustments, as appropriate, to better achieve program  goals.

section_title: Agency Comments and Our Evaluation
paragraphs: HHS provided written comments on a draft of this report, which are  reprinted in appendix VII, and one technical comment. HHS agreed that  data reliability and performance monitoring are important, but neither  agreed nor disagreed with our recommendations. Regarding our first  recommendation, HHS stated that the reliability and validity of CQM data  are of paramount importance. HHS also stated that ONC and CMS work  together to continually refine strategies to improve the reliability and  efficiency of collecting, analyzing, and sharing CQM data collected using  certified EHR technology. HHS did not, however, identify any specific  actions the agencies might take to improve CQM reliability or whether  those actions would include developing a comprehensive strategy. HHS  also noted the importance of the certification testing tool, known as  Cypress, which it stated ensures the reliability of CQM data collected  using 2014-edition EHRs. While the addition of testing to ensure that  2014-edition EHRs accurately collect and calculate CQMs is an  improvement over the process used to certify 2011-edition EHRs—during  which such tests were not conducted—our report raises other reliability  issues that would not be addressed by the certification testing tool alone.  Regarding our second and third recommendations, HHS agreed that  outcome-oriented performance measures that link participation in the  EHR programs with discrete improvements in health care quality,  efficiency, and safety would be useful for evaluating the extent to which  improvements are achieved. HHS also noted that ONC oversees  evaluation activities that have provided interim program results and  conducts policy research and analytic projects that will enable the  development of outcome-oriented performance measures. However, HHS  did not provide any details on the timing or content of outcome-oriented  performance measures that CMS and ONC might develop.

We are sending copies of this report to the Secretary of Health and  Human Services, the Administrator of CMS, the National Coordinator for  Health Information Technology, and other interested parties. In addition,  the report will be available at no charge on GAO’s website at  http://www.gao.gov.

If you or your staffs have any questions about this report, please contact  me at (202) 512-7114 or at kohnl@gao.gov. Contact points for our Office  of Congressional Relations and Office of Public Affairs can be found on  the last page of this report. Other major contributors to this report are  listed in appendix VIII.

section_title: Appendix I: Effect of the HITECH Act on Health Insurance Premiums
paragraphs: The Health Information Technology for Economic and Clinical Health  (HITECH) Act mandates us to report on, among other things, the impact  of the HITECH Act on health insurance premiums. In 2009, the  Congressional Budget Office stated an expectation that nationwide  adoption of health information technology such as electronic health  records (EHR) would reduce total spending on health care and predicted  that lower health care costs for private payers would result in lower health  insurance premiums in the private sector.

To describe the effect of the HITECH Act on health insurance premiums,  we contacted representatives from America’s Health Insurance Plans, the  American Academy of Actuaries, the Blue Cross Blue Shield Association,  and four health insurance companies. None of the seven organizations  we spoke to had done any research to look at the impact of the HITECH  Act on health insurance premiums. Three organizations also indicated  that the HITECH Act’s effect on health insurance premiums would be  difficult to isolate from the various other drivers of health insurance  premium costs. For example, one organization noted that the health  insurance industry is also in the midst of implementing the Patient  Protection and Affordable Care Act, suggesting the reforms arising from  that legislation are also affecting health insurance premium costs.

Five of the seven organizations offered anecdotes or speculation as to  the potential effect of the HITECH Act—and particularly the adoption and  meaningful use of EHRs as encouraged by the EHR programs—on health  insurance premiums. One organization told us that there is evidence that  widely implemented payment and delivery reform—particularly the  patient-centered medical home, of which EHRs are an important  element—has led to reductions in health care utilization and, in some  cases, generated savings, which may eventually affect health insurance  premiums. Another organization said that implementing the HITECH Act  and the Patient Protection and Affordable Care Act has increased  administrative costs and expressed concern that if the organization’s  administrative costs increase due to changes in requirements, they may  rise above the 15 to 20 percent level allowed under medical loss ratio  requirements.

section_title: Appendix II: Scope and Methodology
paragraphs: This appendix provides additional details regarding our analysis of   (1) participation in the Medicare and Medicaid Electronic Health Record  (EHR) programs; (2) meaningful use measures and clinical quality  measures (CQM) Medicare providers reported to the Centers for  Medicare & Medicaid Services (CMS) to demonstrate meaningful use;  and (3) 2012 survey data collected by the National Center for Health  Statistics.

To ensure the reliability of the data, we interviewed officials from CMS,  reviewed relevant documentation, and conducted electronic testing to  identify missing data and obvious errors. On the basis of these activities,  we determined that the data were sufficiently reliable for our analysis.

Analysis of participation in the Medicare and Medicaid EHR  programs. We conducted several analyses to report on participation in  the Medicare and Medicaid EHR programs for the 2011 and 2012  program years. Specifically, we analyzed data that CMS collected in the  National Level Repository through October 23, 2013. As a result, we  generally included full-year information for both years in our analysis of  the Medicare EHR program.complete information for the Medicaid EHR program for the 2012 program  year because, according to CMS, 14 states and the District of Columbia  had not yet completed their determinations of which hospitals and  professionals had met all the requirements to receive incentive payments  for 2012. We used these data to conduct the following analyses.

However, our analysis does not contain

We estimated the percentage of eligible providers awarded an  incentive payment from either the Medicare or the Medicaid EHR  program. We determined the number of providers that were awarded  a Medicare or Medicaid (or, in the case of some hospitals, both) EHR  incentive payment by counting the number of providers that had an  incentive payment disbursed to them for 2011 and for 2012.

Using a similar approach, we determined the number of providers that met the  participation requirements of the Medicare Advantage EHR program for both years. divided the number of providers awarded incentive payments from  either program by the total number of eligible providers CMS  estimated in its Stage 1 final rule. Specifically, CMS estimated that  5,013 hospitals and 521,600 professionals were eligible for the  Medicare or Medicaid EHR program for 2011 and 5,013 hospitals and  527,200 professionals for 2012.

We determined the total amount of Medicare and Medicaid EHR  incentive payments awarded to providers each year by summing the  Medicare and Medicaid EHR incentive payments awarded to  providers for each program year.

We estimated the percentage of providers that were awarded a  Medicare or Medicaid (or, in the case of some hospitals, both) EHR  incentive payment for 2011 but not for 2012 by determining the  number of providers that had a Medicare or Medicaid EHR incentive  payment disbursed to them for 2011 but not for 2012. For the Medicaid EHR program, we limited this analysis to providers that were  awarded incentive payments from 36 states that, according to CMS,  had completed their determinations of which providers would receive  incentive payments for the 2011 and 2012 Medicaid EHR program  years as of October 23, 2013.

Analysis of meaningful use measures and CQMs Medicare providers  reported to CMS to demonstrate meaningful use. We conducted  several analyses of meaningful use measures and CQMs providers  reported to CMS to demonstrate meaningful use under the Medicare EHR  program for 2011 and 2012. We analyzed data that CMS collected in the  National Level Repository through October 23, 2013. We did not analyze  meaningful use measure or CQM data submitted by hospitals or  professionals who received incentive payments from the Medicare  Advantage EHR program or CQM data submitted by providers to one of  the electronic reporting pilots that were available for the 2012 program  year.program year.

For each of the following analyses, we compared reporting by

We determined the extent to which providers exceeded the reporting  thresholds CMS established for a subset of the meaningful use  measures (14 hospital measures and 16 professional measures) by  determining providers’ average reporting rate for each applicable  measure and comparing this rate to the threshold CMS established for  the measures.

We determined the frequency with which providers reported  meaningful use measures without claiming exemptions, by calculating  the percentage of providers that reported meaningful use measures  but excluding from the calculation those providers that reported to  CMS that a measure was not relevant to them. A provider may claim  exemptions from reporting some measures if the provider meets  certain criteria specific to the individual measures. For example, the  measure may not be relevant to the provider’s patient populations or  clinical practices, the provider may have conducted too few actions to  be measured, or the provider may not have been able to perform the  action. The agency allows providers to claim exemptions from  reporting certain meaningful use measures to help ensure that  providers with all types of patient populations and clinical practices  could potentially demonstrate meaningful use. For 2011 and 2012,  hospitals were permitted to claim exemptions from reporting 7  meaningful use measures and professionals from reporting 14  meaningful use measures.

We determined the extent to which providers had few patients who  could be included in the calculation of at least one CQM.capture a small number of patients may be unreliable measures of  quality because relatively small changes in the number of patients  who experienced the care processes or outcomes targeted by the  measure can generate large shifts in the calculated percentage for the  measure. CMS has recognized in other programs that including a  small number of patients in the calculation of a CQM is a reliability  issue. For example, on the agency’s Hospital Compare website, which  publicly reports CQMs by hospital, CMS indicates whether the number  of patients included in a particular measure calculation was based on  fewer than 25 patients and was thus too small to reliably tell how well  the hospital was performing. For our analysis, we identified CQMs as  unreliable if fewer than 7 patients met inclusion criteria for the  calculation the provider reported for a 90-day reporting period—that  is, during the provider’s first year demonstrating meaningful use. For  providers whose second year demonstrating meaningful use was  2012, the reporting period is a full year. Therefore, for these providers  we identified CQMs as unreliable if fewer than 25 patients met the  inclusion criteria for the calculation.

Analysis of 2012 survey data collected by the National Center for  Health Statistics. We analyzed output of the National Ambulatory  Medical Care Survey, an annual, nationally representative survey of  office-based physicians conducted by the National Center for Health  Statistics in the Centers for Disease Control and Prevention. The survey  captures information about computerized capabilities of office-based  physicians’ EHRs—which are related to meaningful use measures— including whether those capabilities are routinely used. We analyzed  output from the 2012 survey obtained from the National Center for Health  Statistics. We limited our analysis to office-based physicians who  reported using an EHR or an electronic medical record system and   who reported having patient care revenue from Medicare or Medicaid   (69.0 percent of weighted responses). We then identified the  computerized capabilities that were least likely to have been routinely  used and determined, using a crosswalk of meaningful use measures and  survey responses available from the Office of the National Coordinator for  Health Information Technology (ONC), whether those capabilities were  comparable to Stage 2 meaningful use measures. We also determined  whether the computerized capabilities that were least likely to have been  routinely used were related to electronic health information, based on  information obtained from CMS and ONC.

section_title: Appendix III: Stage 1 and Stage 2 Meaningful Use Measures
paragraphs: In general, to receive incentive payments from the Medicare or Medicaid  Electronic Health Record (EHR) programs in Stage 1 or Stage 2,  hospitals must report on a total of 19, and professionals must report on a  total of 20, measures regarding their use of certified EHR technology,  known as meaningful use measures. For certain meaningful use  measures, a provider may report to the Centers for Medicare & Medicaid  Services (CMS) that the measures are not relevant if the provider meets  certain criteria specific to the individual measures; this is referred to as  claiming an exemption. For example, the measure may not be relevant to  the provider’s patient population or clinical practice, the provider may  have conducted too few actions to be measured, or the provider may not  have been able to perform the action. Table 5 lists the meaningful use  measures for each stage and provider type and identifies whether the  measure was mandatory or optional and whether it was identified by the  Office of the National Coordinator for Health Information Technology  (ONC) and CMS as being related to the electronic exchange of health  information.

section_title: Appendix IV: Meaningful Use Measure Reporting for the Medicare EHR Program, 2011 and 2012
paragraphs: In general, to receive incentive payments from the Medicare Electronic  Health Record (EHR) program for 2011 or 2012, hospitals had to report  on a total of 19, and professionals had to report on a total of 20, Stage 1  meaningful use measures. Among those measures reported, 14 were  mandatory for hospitals and 15 were mandatory for professionals. The  remaining measures were selected by providers from a list of optional  measures. For certain meaningful use measures, providers must meet or  exceed reporting thresholds—that is, a specified percentage of patients or  actions required to satisfy the measure—established by the Centers for  Medicare & Medicaid Services (CMS) and the Office of the National  Coordinator for Health Information Technology (ONC) for the EHR  programs. Table 6 displays the average percentage reported by providers  for those measures that require providers to meet or exceed certain  established thresholds. Table 7 displays the percentage of providers that  reported certain measures for 2011 and 2012 but excludes from the  calculation providers that reported to CMS that a measure was not  relevant to them; this is referred to as claiming an exemption. Specifically,  table 7 displays the percentage of providers that reported Stage 1  mandatory meaningful use measures that allow for exemptions and Stage  1 optional meaningful use measures. For 2011 and 2012, hospitals were  allowed to claim exemptions from reporting 7 meaningful use measures,  and professionals were allowed to claim exemptions from reporting 14  meaningful use measures.

section_title: Appendix V: Strategic Goals and Associated Performance Measures Relevant to the EHR Programs
paragraphs: We reviewed Department of Health and Human Services (HHS), Centers  for Medicare & Medicaid Services (CMS), and Office of the National  Coordinator for Health Information Technology (ONC) strategic goals and  categorized them into one of two major categories particularly relevant to  the Electronic Health Record (EHR) programs—(1) adoption and  meaningful use of EHRs; and (2) improving quality, efficiency, and patient   The first category of goals is focused on program processes, and  safety.the second is focused on program outcomes. For the first category— adoption and meaningful use of EHRs—HHS, CMS, and ONC have  established 26 performance measures. See table 8 for the department-  and agency-level strategic goals and performance measures that fall into  this category.

Note that ONC has three other goals that are relevant to the EHR  programs but did not fall into the two categories we focused on and are  thus not included in the table. Similarly to the measures identified in   table 8, the performance measures associated with those additional goals  enable the agency to track processes but not outcomes.

section_title: Appendix VI: Program Evaluations Assessing the EHR Programs’ Effects on Outcomes
paragraphs: Centers for Medicare & Medicaid Services (CMS) and Office of the  National Coordinator for Health Information Technology (ONC) officials  told us they are conducting some evaluations to assess the effect of the  Electronic Health Record (EHR) programs on outcomes. See table 10 for  a summary of these evaluations according to agency officials.

section_title: Appendix VII: Comments from the Department of Health and Human Services

section_title: Appendix VIII: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Will Simerl, Assistant   Director; Julianne Flowers; Melanie Krause; Shannon Legeer;   Hannah Marston Minter; Monica Perez-Nelson; Roseanne Price;   and Rebecca Rust Williamson made key contributions to this report.


Question-summary annotations: questions: question: What are the indications of those who participated in the programs' first phase?
answer: Reporting on meaningful use for 2011 and 2012 indicates that providers who have already participated in the programs' first phase—Stage 1—used their certified EHR systems more often than required.
child_questions: question: To what extent did these providers exceed requirements?
answer: For example, for both 2011 and 2012, Medicare hospitals reported using computerized provider order entry for over 84 percent of patients—in excess of the required threshold for Stage 1 of 30 percent.

question: What are the drawbacks of stage 1 measures?
answer: However, some meaningful use measures may be more challenging for providers, including measures involving the electronic exchange of information. For example, less than 15 percent of professionals reported on an optional Stage 1 measure to provide a summary of care document at each care transition or referral, which is mandatory in Stage 2.
child_questions: question: How is CMS addressing the lack of provider preparedness?
answer: A CMS official said the agency is taking steps to help providers prepare for Stage 2 meaningful use measures.

questions: question: What is the effect of the lack of a comprehensive strategy?
answer: The lack of a comprehensive strategy limits HHS's ability to ensure the department can reliably use the clinical quality measures (CQM) collected in certified EHRs for quality measurement activities.
child_questions: question: What are some persisting issues and what have the respective departments done to address these issues?
answer: Reliability issues persist, although CMS and HHS's Office of the National Coordinator for Health Information Technology (ONC) have made efforts to address concerns.

question: What issues are present in reporting CQMs?
answer: For example, different providers may report CQMs based on and tested to different requirements depending on whether their EHRs have incorporated technical updates.

question: How efficient are the quality and efficiency?
answer: Without a comprehensive strategy, efforts to address reliability issues (in accordance with the internal control standard requiring relevant and reliable information) and improve quality and efficiency may be limited.

questions: question: What are the measures for the EHR program and a drawback?
answer: Consistent with law and GAO guidance on assessing agency performance, HHS, CMS, and ONC have established some performance measures for the EHR programs that are tied to strategic goals regarding adoption and meaningful use of EHRs; however, they have not established measures that would help them to track progress toward program outcomes such as health care quality, efficiency, and patient safety.
child_questions: question: Is the use of EHRs proved to improve outcomes?
answer: Although HHS expects that the use of EHRs can help achieve improved outcomes and support other efforts that are also intended to improve care, that result is not yet assured.

question: What are the issues facing CMS and ONC in regards to outcomes?
answer: CMS and ONC may lack critical information necessary to establish program priorities and subsequently make program adjustments based on progress toward outcomes.