HEALTH CARE 

QUALITY

HHS Should Set 

Priorities and 

Comprehensively 

Plan Its Efforts to 

Better Align Health 

Quality Measures

Report to Congressional Committees

October 2016

GAO-17-5

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-17-5, a report to 

congressional committees

October 2016

HEALTH CARE QUALITY

HHS Should Set Priorities and 

Comprehensively Plan Its Efforts to Better 

Align Health Quality Measures

What GAO Found

While the full extent of misalignment among health care quality measures is 

unknown, it can have adverse effects on providers and efforts to improve quality 

of care. Misalignment occurs when health care payers require providers to report 

on measures that focus on different quality issues or define the measures using 

different specifications. GAO identified three studies that provided some 

information on the extent of misalignment. For the most part, these studies 

examined the number of measures that were used in common, among a narrow 

selection of public and private payers, and found that with few exceptions, only a 

small proportion of measures were commonly used by these payers. The 

Department of Health and Human Services’ (HHS) Centers for Medicare & 

Medicaid Services (CMS) agrees that misalignment exists, and some experts 

note that it adds to providers’ administrative burden and often results in quality 

information that is not comparable. 

GAO’s interviews with HHS officials and experts indicate that three interrelated 

factors drive misalignment of health care quality measures, as described in the 

table.

Factors Driving Misalignment of Health Care Quality Measures

Factor Description

Dispersed decisionmaking

Among public and private payers and other stakeholders, each entity 

independently decides which quality measures it will use and which 

specifications should apply to those measures.

Variation in data 

collection and 

reporting systems

Payers may choose different measures, modify existing measures, or leave 

details about measure specifications up to providers in order to 

accommodate differences in data that providers collect and the systems they 

use to collect these data. 

Few meaningful 

measures

Although hundreds of quality measures have been developed, relatively few 

are measures that payers, providers, and other stakeholders agree to adopt, 

because few are viewed as leading to meaningful improvements in quality.

Source: GAO interviews with Department of Health and Human Services officials and experts. | GAO-17-5

HHS has various ongoing efforts that address different aspects of misalignment 

of quality measures and the three factors that drive it. For example, HHS has 

begun to address dispersed decision-making by negotiating with private payers 

to adopt a core set of measures. To address variation in data systems, HHS is 

taking steps to develop electronic quality measures—those that allow providers 

to report data electronically—and standardize the data collected under these 

measures. CMS has also taken steps to address the paucity of meaningful 

measures through efforts to develop new measures that focus on key quality 

concerns. However, HHS has not prioritized development of electronic quality 

measures specifically for the core measures CMS negotiated with private payers, 

which could delay the implementation of this alignment effort. Further, CMS has 

not comprehensively planned how to target the development of new, more 

meaningful measures that address misalignment, and it has not set timelines and 

methods to track its progress. Federal internal control standards and leading 

principles for planning call for agencies to prioritize their efforts and assess their 

progress in achieving their objectives. Without comprehensive planning, CMS

cannot ensure that it will achieve its objective of reducing misalignment. 

View GAO-17-5. For more information, contact 

A. Nicole Clowers at (202) 512-7114 or 

clowersa@gao.gov.

Why GAO Did This Study

Both the federal government and 

private payers, such as health plans, 

increasingly use quality measures to 

encourage providers to improve health 

care quality. In addition to its ongoing 

programs that use quality measures to 

assess provider performance, HHS

has proposed to begin implementing

the CMS Quality Payment Program, in 

January 2017. However, if measures 

are misaligned across these programs, 

the misalignment could create 

administrative burden for providers. 

The Medicare Access and CHIP 

Reauthorization Act of 2015 includes a 

provision for GAO to examine the use 

of quality measures across HHS 

programs and private payers, with a 

focus on reducing burden. In this 

report, GAO examined (1) what is 

known about the extent and effects, if 

any, of quality measure misalignment; 

(2) key factors that can contribute to 

misalignment; and (3) HHS’s efforts to 

address any misalignment. GAO 

conducted a literature review to identify 

related studies; reviewed HHS 

documents; and interviewed HHS 

officials and experts from 16 

organizations that represent a range of 

perspectives, including providers and 

payers. 

What GAO Recommends

GAO recommends that HHS (1) 

prioritize its development of electronic 

quality measures and related data 

elements for the core measures it and 

private payers have agreed to use, and 

(2) comprehensively plan, including 

setting timelines for, its efforts to 

develop more meaningful quality 

measures. HHS concurred with the 

recommendations.
Page i GAO-17-5 Alignment of Health Care Quality Measures

Letter 1

Background 5

While the Full Extent of Quality Measure Misalignment Is 

Unknown, Evidence Indicates That It Increases Provider 

Burden and Produces Information That Is Not Comparable 9

Dispersed Decision-Making, Variation in Data Collection and 

Reporting Systems, and Few Meaningful Measures Drive 

Misalignment 13

HHS Has Taken Steps to Address Quality Measure Misalignment 

but Lacks Comprehensive Planning to Guide Prioritization and 

Ensure Effective Implementation 16

Conclusions 26

Recommendations for Executive Action 27

Agency Comments and Our Evaluation 27

Appendix I Literature Review Methodology 30

Appendix II Comments from the Department of Health and Human Services 31

Appendix III GAO Contact and Staff Acknowledgments 35

Bibliography 36

Tables

Table 1: Examples of Data Elements Used in Measures for 

Controlling Blood Pressure and Variation between the 

Measures 7

Table 2: Department of Health and Human Services (HHS) Efforts 

to Address Quality Measure Misalignment 17

Figure

Figure 1: A General Overview of How to Calculate Physician 

Performance on Measure of Blood Pressure Control 6

Contents
Page ii GAO-17-5 Alignment of Health Care Quality Measures

Abbreviations 

CMS Centers for Medicare & Medicaid Services 

CQMC Core Quality Measures Collaborative

EHR electronic health record

GPRA Government Performance and Results Act of 1993

HHS Department of Health and Human Services

MACRA Medicare Access and CHIP Reauthorization Act of 2015

MIPS Merit-Based Incentive Payment System 

ONC Office of the National Coordinator for Health Information 

Technology

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-17-5 Alignment of Health Care Quality Measures

441 G St. N.W.

Washington, DC 20548

October 13, 2016

Congressional Committees

Both the federal government and private payers, such as health plans, 

increasingly use health care quality measures to encourage providers to 

improve health care quality. This often involves comparing the 

performance of physicians and other providers in order to hold them 

accountable for the health care they deliver and adjust their payments 

accordingly. For example, the Centers for Medicare & Medicaid Services 

(CMS) in the Department of Health and Human Services (HHS) has 

several programs and initiatives that provide financial incentives to 

physicians and other providers based on information they report on 

various health care quality measures. In addition, HHS proposed to begin 

implementation of the CMS Quality Payment Program—a new incentive 

payment program for physicians and other eligible providers—in January 

2017, in accordance with the Medicare Access and CHIP Reauthorization 

Act of 2015 (MACRA).1 This program will adjust physician and other 

provider payments through bonuses or penalties based in part on their

performance on a set of quality measures. At the same time, multiple 

private payers have expanded their use of different quality measures to 

assess physician and other provider quality and adjust their payments 

accordingly. The combination of these public and private efforts has led to 

physicians and other providers facing increased financial incentives to 

demonstrate high or improving performance across a growing list of 

diverse quality measures.

As payers increasingly rely on quality measures to inform their payments, 

concerns have been raised by Congress and organizations involved with 

quality measurement about the differences in the quality measures public 

and private payers require physicians and other providers to report, which 

we refer to as quality measure misalignment. In this report, we focus on 

two types of misalignment: (1) when different health care payers require 

providers to focus on different quality issues and, accordingly, require 

providers to report on different quality measures and (2) when different 

health care payers require providers to report on the same measure, but 

 1

Pub. L. No. 114-10, § 101, 129 Stat. 87, 89 (2015).

Letter
Page 2 GAO-17-5 Alignment of Health Care Quality Measures

set different specifications for that measure, such as different definitions 

of the measure’s target population. Both types of misalignment can create 

administrative burdens for providers if providers must report different, 

sometimes highly detailed, clinical information to different public and 

private payers. Both types of misalignment may also make it difficult for 

providers to improve the quality of care they provide if the misaligned 

measures produce inconsistent information on the areas where the 

providers should focus their improvement efforts.

MACRA includes a provision for us to examine the use of quality 

measures across HHS programs and private payers, including the 

administrative burden for providers. In this report we:

1. describe what is known about the extent and effects, if any, of health 

care quality measure misalignment;

2. describe key factors that can contribute to quality measure 

misalignment; and

3. evaluate HHS’s efforts to address quality measure misalignment.

To examine what is known about the extent and effects, if any, of health 

care quality measure misalignment, we conducted a literature review to 

identify relevant studies published in peer-reviewed journals, trade and 

association publications, conference papers, and government reports 

from January 2010 to February 2016.2 We also conducted a more general 

internet search. As a result, we identified and reviewed 13 relevant 

studies.3 (For more details about the methodology of our literature review, 

including our criteria for determining relevant studies, see app. I.) We 

examined the methodologies for each of these studies and interviewed 

some of their authors. We determined that the studies were sufficiently 

reliable for our purposes. We also interviewed HHS officials and a 

selection of experts from 16 organizations. We selected experts based on 

their relevant experience or professional qualifications to cover a range of 

stakeholder perspectives on quality measurement and misalignment, 

 2

We selected this timeframe to ensure that we captured literature that best reflects the 

current state of quality measurement efforts—particularly those that may affect measure 

alignment—since the enactment of the Patient Protection and Affordable Care Act in 

2010, which, among other things, included requirements for quality measurement. 

3

See the bibliography for a complete list of the studies we identified. 
Page 3 GAO-17-5 Alignment of Health Care Quality Measures

including the perspectives of providers, payers, consumers and 

purchasers, measure professionals, and researchers, such as the authors 

of two studies we identified related to the extent of misalignment.4 We 

synthesized the experts’ observations along with relevant literature and 

documents to describe what is known about the extent and effects of 

misalignment.

To examine the key factors that can contribute to quality measure 

misalignment, we used the same methodology as the one described 

above. Specifically, we reviewed the 13 relevant studies we identified and 

interviewed HHS officials and experts from the 16 selected organizations. 

We synthesized the information from these sources to describe the key 

factors contributing to the misalignment of quality measures.

To examine HHS’s efforts to address quality measure misalignment, we 

interviewed agency officials and reviewed agency documents. We 

interviewed relevant officials in CMS’s Center for Clinical Standards and 

Quality concerning their efforts to reduce quality measure misalignment in 

the context of their broader quality measurement efforts, including their 

efforts to implement the Quality Payment Program. In addition, we 

interviewed officials in HHS’s Office of the National Coordinator for Health 

Information Technology (ONC) regarding their efforts to promote the 

development of health information technology standards related to the 

development of electronic quality measures. In reviewing relevant HHS 

documents, we focused on documents outlining HHS’s plans for 

addressing measure misalignment in the context of the agency’s broader 

efforts to assess provider quality performance, including the CMS Quality 

Strategy, the CMS Quality Measure Development Plan, and the Blueprint 

for the CMS Measures Management System. We also interviewed and 

reviewed documents from experts that provided information related to 

 4

We interviewed experts from the following organizations: (1) providers—American 

Academy of Family Physicians, American Academy of Orthopaedic Surgeons, American 

College of Cardiology, American College of Physicians, American College of Surgeons, 

and LeadingAge; (2) payers—America’s Health Insurance Plans and the National 

Association of Medicaid Directors; (3) consumers and purchasers—National Partnership 

for Women & Families and the Pacific Business Group on Health; (4) measure 

professionals—National Committee for Quality Assurance, National Quality Forum, and 

the Network for Regional Healthcare Improvement; and (5) researchers—Bailit Health 

Purchasing LLC, the Health and Medicine Division of the National Academies of Sciences, 

Engineering, and Medicine (previously known as the Institute of Medicine), and the RAND 

Corporation.
Page 4 GAO-17-5 Alignment of Health Care Quality Measures

HHS’s efforts to reduce quality measure misalignment. In addition, we 

reviewed the relevant standards for internal control in the federal 

government and the relevant criteria from GAO’s body of work on 

effectively managing performance under the Government Performance 

and Results Act of 1993 (GPRA), as enhanced by the GPRA 

Modernization Act of 2010.5

We conducted this performance audit from September 2015 to October 

2016 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

 5

Internal control is a process effected by an entity’s oversight body, management, and 

other personnel that provides reasonable assurance that the objectives of an entity will be 

achieved. See GAO, Standards for Internal Control in the Federal Government, 

GAO-14-704G (Washington, D.C.: Sept. 2014). In past reports, we have identified best 

practices in planning. For example, see GAO, Executive Guide: Effectively Implementing 

the Government Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: 

June 1996), GAO, Combating Terrorism: Evaluation of Selected Characteristics in 

National Strategies Related to Terrorism, GAO-04-408T (Washington, D.C.: Feb. 3, 2004) 

and GAO, Veterans’ Health Care: Proper Plan Needed to Modernize System for Paying 

Community Providers, GAO-16-353 (Washington, D.C.: May 11, 2016).
Page 5 GAO-17-5 Alignment of Health Care Quality Measures

CMS and private payers use a variety of quality measures to assess 

different aspects of health care quality. Quality measures may be used to 

measure the performance of providers in various settings, including 

hospitals, physician offices, and nursing homes. Process measures 

assess the extent to which providers effectively implement clinical 

practices (or treatments) that have been shown to result in high-quality or 

efficient care, such as the percentage of patients with a myocardial 

infarction who receive an aspirin prescription on discharge. Others are 

outcome measures, which track the results of health care, such as 

mortality, infections, and patients’ experiences of that care.

A variety of different entities may develop health care quality measures, 

such as the Joint Commission, the National Committee for Quality 

Assurance, and various medical specialty societies.6 In some cases CMS 

contracts with entities for the development of measures for use in its 

quality programs and has established a Measures Management System 

Blueprint that lays out the steps measure developers should follow to first 

identify priority topics or conditions where new measures are needed, and 

then develop and test specific new measures to fill those identified gaps. 

According to CMS estimates, it can take 2 years or more to complete all 

of these steps.7 CMS and entities that develop measures may voluntarily 

submit them to the National Quality Forum, a nonprofit organization that 

evaluates and endorses measures—that is, determines which measures 

should be recognized as the best available for a given aspect of care. The 

National Quality Forum has endorsed over 700 quality measures.

Quality measures are composed of a number of clinical data elements, or 

pieces of data, that are needed to calculate providers’ performance on 

any given measure. Some measures are more complex and require more 

data elements. Historically, providers have collected data elements for 

quality measures through a detailed, manual review of paper medical 

records. More recently, a limited number of electronic quality measures 

have been developed to allow providers to report data elements 

 6

The Joint Commission is a nonprofit organization that accredits and certifies nearly 

21,000 health care organizations and programs in the United States, including hospitals. 

The National Committee for Quality Assurance is a nonprofit organization that accredits 

health plans and develops quality standards and performance measures for them. 

7

See Centers for Medicare & Medicaid Services, Blueprint for the CMS Measures 

Management System, version 12.0, May 2016, p. 23.

Background

What are some examples of process and 

outcome measures related to diabetes care?

Process measures:

Hemoglobin A1c testing

• What percentage of patients with diabetes 

had their hemoglobin A1c tested?

Diabetes eye exam

• What percentage of patients with a diagnosis 

of diabetes had an eye exam?

Outcome measures:

Hemoglobin A1c Poor Control

• What is the percentage of patients with 

diabetes that had poor hemoglobin A1c 

control (HbA1c > 9%)?

Diabetes low density lipoprotein 

(LDL-C) control

• What is the percentage of patients with 

diabetes whose LDL-C was adequately 

controlled (<100mg/dL)?

Source: GAO review of Centers for Medicare & Medicaid 

Services information. | GAO-17-5
Page 6 GAO-17-5 Alignment of Health Care Quality Measures

electronically using electronic health records (EHR).8 (See fig. 1 for a 

general overview of how a measure, related to blood pressure control, is 

calculated and table 1 for examples of how the use of data elements 

between two blood pressure control measures can vary.)

Figure 1: A General Overview of How to Calculate Physician Performance on 

Measure of Blood Pressure Control

 8

Electronic quality measures are also referred to as electronic clinical quality measures or 

eCQMs. 

Where do quality measures come from?

The variety of quality measures reflects the fact 

that many different organizations are involved in 

developing them. Some, such as the Joint 

Commission and the National Committee for 

Quality Assurance, develop measures to support 

their accreditation of hospitals and health plans, 

respectively. Other developers are various 

medical specialty societies that organize clinical 

data registries or other efforts to collect and 

analyze data on patient treatment and outcomes 

in specific clinical areas, such as cardiology or 

oncology. The organization that develops the 

measure is generally recognized as that 

measure’s steward, and it controls any official 

changes to the measure’s specifications over 

time.

Source: GAO review of Centers for Medicare & Medicaid 

Services and stakeholder information. | GAO-17-5
Page 7 GAO-17-5 Alignment of Health Care Quality Measures

Table 1: Examples of Data Elements Used in Measures for Controlling Blood Pressure and Variation between the Measures

Data elements How used in the measures Variation between two measures

Patient diagnoses To identify patients to be included or 

excluded based on diagnoses

Use of different patient diagnoses to identify patients 

who should be included or excluded, such as whether 

a patient has a diagnosis of diabetes or end stage 

renal disease 

Patient age To identify patients to be included or 

excluded based on age

Inclusion of patients that range from age 18 through 

75 compared to those aged 18 through 85

Blood pressure To identify blood pressure level for 

patients with adequately controlled blood 

pressure

Blood pressure under 140/90 millimeters of mercury 

(mm Hg) for patients aged 18 through 75 with 

diabetes compared to blood pressure under 150/90 

mm Hg for those aged 60 through 85 without diabetes 

Source: GAO analysis of information from the Department of Health and Human Services and the National Quality Forum. | GAO-17-5

A physician or other health care provider may be required to report 

multiple quality measures to multiple organizations, including CMS and 

multiple private payers. For example, a physician may participate in 

Medicare and a private health plan that each use different measures for 

assessing the care of diabetic patients. In another example, a physician 

may report similar measures to multiple payers that assess blood sugar 

levels among diabetic patients, but each measure may use a different 

threshold to determine which patients have their blood sugar levels under 

control.

Since the early 2000’s, CMS has launched a number of programs that 

offer financial incentives to providers receiving Medicare payments to 

report their performance on specified quality measures. Some of these 

programs, such as the Physician Quality Reporting System, are pay-forreporting programs, in which providers may receive a mix of bonuses and 

penalties for simply reporting their performance on quality measures. 

Others, such as the Value-based Modifier program for physicians, are 

pay-for-performance programs, in which the level of providers’ 

performance on the quality measures affects the amount of the payment 

they receive.9 In addition, the Medicare EHR program provides a mix of 

bonuses and penalties to encourage hospitals and physicians to acquire 

EHR systems that meet certain requirements and use them in specified 

ways. As part of this EHR program, providers are required to use their 

 9

The Value-based Modifier program uses the quality measures included in the Physician 

Quality Reporting System program.
Page 8 GAO-17-5 Alignment of Health Care Quality Measures

certified EHR technology to collect and report certain quality measures to 

CMS. CMS also uses quality measures as a basis for payment in various 

alternative payment models—such as accountable care organizations 

where CMS pays groups of providers based in part on the collective 

performance of those providers—rather than the fee-for-service 

traditionally paid in Medicare.

The proposed January 2017 implementation of CMS’s Quality Payment 

Program, in accordance with MACRA, will continue CMS’s efforts to link 

Medicare payments with physicians’ performance on various quality 

measures. Under this program, components of the previously separate 

Physician Quality Reporting System, Physician Value-based Payment 

Modifier program, and Medicare EHR incentive program will be merged 

into the Merit-Based Incentive Payment System (MIPS) so that payments 

for most physicians will reflect physician performance on both quality 

measures and EHR use. Although this new program will not start to affect 

physician payments until 2019, CMS has proposed that the 2019 

payment adjustments will be based on physician performance on quality 

measures assessed during 2017. The payment adjustments will increase 

over time, from 4 percent in 2019 to 9 percent in 2022 and after. Payment 

adjustments under the program will be budget neutral, which means that 

the total sum of bonuses and penalties will be equal. To prepare to 

implement the program in 2017, CMS issued a proposed rule for the 

Quality Payment Program in May 2016 that included a preliminary list of 

the quality measures from which physicians can choose when reporting to 

CMS.10 MACRA requires that CMS publish an annual list of final MIPS 

quality measures by no later than November 1 of the year prior to the 

MIPS performance period in which the measures will be used. MACRA 

also directed the Secretary of HHS to transfer $15 million from the 

Federal Supplementary Medical Trust Fund for each fiscal year from 2015 

through 2019 to support measure development related to implementation 

of the new physician Quality Payment Program.

 10Medicare Program; Merit-Based Incentive Payment System (MIPS) and Alternative 

Payment Model (APM) Incentive Under the Physician Fee Schedule, and Criteria for 

Physician-Focused Payment Models; Proposed Rule, 81 Fed. Reg. 28,161 (proposed May 

9, 2016).
Page 9 GAO-17-5 Alignment of Health Care Quality Measures

Our literature review and interviews with experts indicate that the full 

extent of quality measure misalignment is unknown. We identified three 

relevant studies, but these provide only indirect information on the extent 

of measure misalignment. For the most part, the studies examined the 

number of measures that were aligned, or used in common, among a 

narrow selection of public and private payers and did not identify the 

extent of misalignment among the measures. All three studies found that 

with few exceptions, only a small proportion of measures were used in 

common by public and private payers.

• A 2015 CMS report examined the number of aligned measures used 

among various CMS and state Medicaid programs.11 The study found 

that the percentage of aligned measures varied, ranging from 13 to 62 

percent, depending on the state Medicaid program type.12

• A 2013 study examined the number of aligned measures used among 

23 commercial health plans and in two Medicare programs, the 

Physician Quality Reporting System and the Medicare Shared 

Savings Program.13 The study found that of the 546 quality measures 

used by the commercial health plans, 26—or about 5 percent—of the 

measures were used by more than half of the plans.14 Further, 17 out 

of 301 Physician Quality Reporting System measures and 5 Medicare 

 11The report compared measures used by various CMS programs to those used by 

various state programs—including Medicaid—as well as to those used by Veterans Health 

Administration programs. Centers for Medicare & Medicaid Services, 2015 National 

Impact Assessment of the Centers for Medicare & Medicaid (CMS) Quality Measures 

Report, (Baltimore, Md.: March 2, 2015). 

12State Medicaid program types include Fee-for-Service, Managed Care Organizations, 

Accountable Care Organizations, and Dual Eligible programs, among others. See Centers 

for Medicare & Medicaid Services, 2015 National Impact Assessment, 101-102. 

13The Medicare Shared Savings Program allows providers that participate in an 

Accountable Care Organization to continue to receive traditional fee-for-service payments 

under Medicare and also be eligible for additional payments if they meet specified quality 

and savings requirements. See A. Higgins, G. Veselovskiy, and L. McKnown. “Provider 

Performance Measures in Private and Public Programs: Achieving Meaningful Alignment 

with Flexibility to Innovate,” Health Affairs, vol. 32, no. 8, (2013). 

14Higgins, Veselovskiy, and McKnown, “Provider Performance Measures,” 1455-1456. 

While the Full Extent 

of Quality Measure 

Misalignment Is 

Unknown, Evidence 

Indicates That It 

Increases Provider 

Burden and Produces 

Information That Is 

Not Comparable
Page 10 GAO-17-5 Alignment of Health Care Quality Measures

Shared Savings Program measures were identical to those commonly 

used by the commercial plans.15

• The third study, released in 2013, examined the number of measures 

that were aligned among 48 state and regional programs, including 

state Medicaid programs, commercial health plans, and regional 

collaboratives, among others.16 The study found these programs used 

509 measures, and that 20 percent of the measures were used by 

more than one program.17 Medicaid Managed Care Organizations 

were the only type of program to share more than half of their 

measures.18

Our interviews with the authors of two of these studies helped to explain 

why it is difficult to determine the extent of quality measure misalignment. 

According to these experts, limitations in data collection and analysis 

make it difficult to quantify the extent of measure misalignment. For 

example, one of the authors said that the research team was not always 

able to obtain complete measure specifications from publicly available 

sources or measure users. Further, the specifications they did receive 

were often several pages long, for even just one measure, making it 

challenging to identify differences between the measures. The author also 

told us that the study was limited due to restrictions in time and funding 

and asserted that this type of work would be even more challenging today 

given the increase in the number of quality measures in use since 2013, 

when the study was completed.

Although CMS conducted a limited analysis of the extent of alignment as 

part of its 2015 study and acknowledges that misalignment exists, it has 

 15The authors did not state the total number of Medicare Shared Savings Program 

measures. Higgins, Veselovskiy, and McKnown, “Provider Performance Measures,” 1458. 

16Kate Bazinsky and Michael Bailit, The Significant Lack of Alignment Across State and 

Regional Health Measure Sets, Health Care Performance Measurement Activity: An 

Analysis of 48 State and Regional Measure Sets, (Needham, Mass.: Bailit Health 

Purchasing, LLC, 2013). Kate Bazinsky and Michael Bailit, The Significant Lack of 

Alignment Across State and Regional Health Measure Sets: An Analysis of 48 State and 

Regional Measure Sets, Presentation, (September 10, 2013).

17Bazinsky and Bailit, Lack of Alignment, 3.

18The six Medicaid Managed Care Organizations included in the analysis shared 62 

percent of the measures used among these programs. Bazinsky and Bailit, Lack of 

Alignment, 4. 
Page 11 GAO-17-5 Alignment of Health Care Quality Measures

not attempted to quantify the extent of misalignment among its own 

programs or across public and private payers. CMS officials told us that 

they have not studied misalignment among their own programs because 

doing so would be resource-intensive, and they instead have focused 

resources on addressing misalignment.19 CMS noted in its 2015 report 

that it has not attempted to quantify the extent of misalignment across 

public and private payers because findings from other studies have 

provided evidence regarding such misalignment.20 Like the other studies 

we reviewed, these studies do not provide comprehensive information on 

the extent of misalignment. CMS officials told us that they felt the studies 

were sufficient to indicate that misalignment exists across public and 

private payers and needs to be addressed.

While the full extent of quality measure misalignment has not been 

quantified, evidence indicates that misalignment creates administrative 

burdens for health care providers and often results in quality information 

from different payers that is not comparable. This, in turn, diminishes 

providers’ ability to identify high-impact quality improvements or prioritize 

the resources dedicated to them. HHS officials and one expert we 

interviewed observed that misalignment between federal and private 

payers, more than misalignment among CMS programs, may contribute 

to both administrative burden and quality information that is not 

comparable. CMS officials stated that aligning the measures used in CMS 

programs with the measures used by private payers and other 

stakeholders has the potential to provide the largest benefit. The adverse 

effects of measure misalignment are described in more detail below.

According to a few of the experts we interviewed, quality measurement 

efforts require providers to expend staff time and incur administrative 

costs. Experts indicated that measure misalignment adds to this burden, 

with a few experts indicating that misalignment may be particularly 

burdensome among small provider practices. One expert told us that 

having measures with different specifications, such as different thresholds 

 19The National Quality Forum is conducting work under contract with HHS that aims to

gather more information on variation of measure specifications, including identifying how, 

where, and why variation in quality measure specifications is occurring as well as 

determining the implications of such variation, among other issues. 

20Centers for Medicare & Medicaid Services, 2015 National Impact Assessment, 88-89, 

98.

Administrative burden
Page 12 GAO-17-5 Alignment of Health Care Quality Measures

for blood sugar control levels, complicates the reporting process because 

providers need to compile information differently for different payers, 

which adds to providers’ workload and disrupts their clinical workflow. 

Furthermore, according to a few of the experts we interviewed, small 

provider practices may have more difficulty devoting resources to quality 

measurement. Specifically, one expert said that misalignment may have a 

greater impact on such practices as they may not have the technology 

infrastructure and administrative support to collect, report, and analyze 

quality measure data.

Two of the studies we reviewed identified costs associated with 

misalignment and quality measurement more generally. A 2016 study 

examined the cost of quality measurement and found that 46 percent of 

physician practices surveyed for the study reported that it was “a 

significant burden to deal with measures that were similar but not identical 

to each other.”21 While the study did not identify specifically how much of 

the cost of quality measurement is attributable to misalignment, the 

authors reported that physician practices spent 785.2 hours per physician 

per year on overall quality measurement efforts, with an average annual 

cost of $40,069 per physician.22 Similarly, a study published by the 

Institute of Medicine in 2015 noted that in a preliminary survey, 20 health 

care organizations reported that they may need 50 to 100 full-time 

equivalent employees, including physicians, at estimated costs ranging 

from $3.5 million to $12 million per year to support overall quality 

measurement efforts.23 One expert we interviewed added that if there was 

greater alignment of quality measures, hospitals and other providers 

could save a significant amount of money.

Some of the experts we interviewed also indicated that misaligned 

measures could result in providers receiving performance feedback from 

payers that is not comparable. For example, one expert told us that 

 21L.P. Casalino, D. Gans, R. Weber, et al., “US Physician Practices Spend More Than 

$15.4 Billion Annually to Report Quality Measures,” Health Affairs, vol. 35, no. 3, (2016).

22Casalino, Gans, Weber, et al., “Quality Measures,” 401-402. 

23Institute of Medicine of the National Academies, Vital Signs: Core Metrics for Health and 

Health Care Progress, (Washington, D.C.: National Academies Press, 2015), 91. In March 

2016, the Institute of Medicine was renamed the Health and Medicine Division of the 

National Academies of Sciences, Engineering, and Medicine. 

Quality information that is not 

comparable 
Page 13 GAO-17-5 Alignment of Health Care Quality Measures

different payers may use measures that address the same health care 

issue but have different specifications—such as a different threshold for 

determining a patient’s blood pressure level. The expert stated that with 

misalignment, a provider may receive feedback from one payer indicating 

that the provider performs adequately, while receiving feedback from 

another payer indicating poor performance.24 Some of the experts 

indicated that this conflicting feedback in turn diminishes providers’ ability 

to identify high-impact quality improvements or prioritize the resources 

dedicated to them. One of these experts told us that when providers 

receive this type of conflicting feedback from different payers, it makes it 

difficult to determine what, if any, changes they need to make to improve 

their performance. Further, according to one of the studies we reviewed, 

providers must report on large numbers of measures across different 

programs, resulting in “measure chaos” and inhibiting providers’ ability to 

make focused quality improvements.25

According to HHS officials and experts we interviewed, three interrelated 

factors drive the misalignment of health care quality measures: (1) 

dispersed decision-making among various public and private health care 

payers regarding measures, (2) variations in the data collection and 

reporting systems used by providers and payers, and (3) a paucity of 

meaningful measures on which stakeholders can agree to align.

 24One of the experts we interviewed told us that payers typically measure providers’ 

performance based only on patients who are covered by that payer, rather than the 

provider’s entire patient population. Therefore, when faced with conflicting feedback, a 

provider may not know if the differences are caused by misalignment, by the differing 

patient populations, or both.

25Bazinsky and Bailit, Lack of Alignment, 5. A few of the experts we interviewed told us 

that in some cases it may be reasonable for measures to differ from each other for various 

reasons, such as when a new or innovative measure is developed; when a different 

measure is needed to address a different population or priority at the regional, state, or 

local level; or when a measure is modified or replaced based on new scientific evidence. 

Dispersed DecisionMaking, Variation in 

Data Collection and 

Reporting Systems, 

and Few Meaningful 

Measures Drive 

Misalignment
Page 14 GAO-17-5 Alignment of Health Care Quality Measures

According to HHS officials and experts we interviewed, quality measure 

misalignment is driven, in part, by public and private payers and other 

stakeholders independently deciding which measures or measure 

specifications to use. For example, some experts we interviewed told us 

that payers may prioritize different measures because they serve different 

populations—for example, Medicare primarily serves people over age 65, 

while Medicaid and private payers serve populations of various ages. In 

addition, payers may prioritize certain measures to focus on frequent or 

costly episodes of care or to differentiate themselves in competitive 

markets. Alternatively, physicians provide input to CMS and other payers 

on which measures to use and may prioritize measures focused on 

improving care for specific conditions they treat in their practices. Further, 

some experts said that even if multiple payers agreed upon a core set of 

quality measures at the national level, separate decisions at the state or 

regional level to use different measures or different measure 

specifications compared to those in the core sets could lead to continued 

misalignment.

HHS officials and most of the experts we interviewed agreed that 

measure misalignment is in part driven by the different decisions payers 

make to accommodate variations in the quality data that physicians and 

other providers collect and in the systems they use to report the data. For 

example, they indicated that physicians may collect the data used to 

calculate and report measures using different EHR systems, paper 

records, or clinical data registries. To accommodate these differences, 

experts explained that payers may choose different measures, modify 

existing measures, or leave details about measure specifications up to 

providers. While these accommodations by payers may allow providers to 

report on measures using the data each has available, they can also 

contribute to misalignment and necessitate that providers contracting with 

several payers produce multiple reports with different measure 

requirements. Furthermore, some experts told us that without common 

measures or measure specifications, vendors that design EHR systems 

for providers have little incentive to standardize their systems to facilitate 

the alignment of data collection and reporting.

According to HHS officials and most of the experts we interviewed, 

measure misalignment is also driven, in part, by the lack of meaningful 

measures that payers, providers, and other stakeholders can agree 

should be commonly adopted. Some of these experts told us that 

physicians were reluctant to align with payers on measures that could not 

be meaningfully used to improve the quality of patient care. Despite the 

existence of hundreds of quality measures, experts clarified that there are 

Dispersed decision-making

Variation in data collection and 

reporting systems

Few meaningful measures
Page 15 GAO-17-5 Alignment of Health Care Quality Measures

few measures that can lead to meaningful improvements in the quality of 

care.26 HHS officials and experts we interviewed emphasized that certain 

types of measures are more likely to be meaningful, such as outcome 

measures. HHS officials and experts we spoke with also explained that 

the paucity of meaningful measures may be due to various challenges 

associated with developing such measures, including the lack of reliable 

data on which to base measures or the scarcity of resources needed for 

the development, testing, and validation of new measures.27

HHS officials and some of the experts we interviewed discussed the ways 

each of the three key factors driving misalignment are interrelated. For 

example, HHS officials and experts told us that without coordinated 

decisions on quality measure priorities and measure specifications, 

providers will continue to use incompatible systems or collect inconsistent 

quality data. Similarly, some experts told us that the paucity of meaningful 

measures is exacerbated by dispersed decision-making and different 

priorities among payers and other stakeholders.

 26Authors of one study we identified found that only 27 percent of physician practices 

participating in the study believed that current measures were moderately or strongly 

representative of the quality of care. Casalino, Gans, Weber, et al., “Quality Measures,” 

401-406.

27These experts discussed the methodological challenges associated with developing 

measures, particularly outcome measures, such as how to attribute health outcomes to a 

provider when several providers were involved in a patient’s care, or how quality 

measures account for other variables that can affect patients’ health that may not be in a 

provider’s control, such as age or socioeconomic status.
Page 16 GAO-17-5 Alignment of Health Care Quality Measures

HHS has several efforts underway that deal with different aspects of 

quality measure misalignment, including efforts to address the key factors 

that drive misalignment. However, in their efforts to develop electronic 

quality measures, HHS agencies have not prioritized developing such 

measures for the aligned core quality measures that have been adopted 

by both CMS and private payers. In addition, HHS has not conducted 

comprehensive planning for its efforts to address misalignment through 

the development of more meaningful quality measures, jeopardizing the 

effectiveness of those efforts.

Based on our reviews of relevant documents and interviews with agency 

officials, we found that several ongoing HHS efforts help to address 

misalignment and the factors that drive it. Some of these efforts, such as 

the Measurement Policy Council and Measure Applications Partnership, 

focus on aligning measures across HHS programs. Others focus on 

developing and aligning quality measures for use by both HHS and 

external groups, such as private health plans and medical specialty 

societies. For example, the CMS Quality Measure Development Plan 

includes CMS’s activities to promote more effective and efficient 

development of quality measures for its new Quality Payment Program by 

both HHS and external groups. In addition, the Core Quality Measures 

Collaborative (CQMC) focuses on aligning the measures used by CMS 

and private payers that assess physician quality of care. (See table 2 for 

more information on each effort, including which of the factors driving 

misalignment the effort aims to address.)

HHS Has Taken 

Steps to Address 

Quality Measure 

Misalignment but 

Lacks 

Comprehensive 

Planning to Guide 

Prioritization and 

Ensure Effective 

Implementation

Several Ongoing HHS 

Efforts Help to Address 

Misalignment and the 

Factors That Drive It
Page 17 GAO-17-5 Alignment of Health Care Quality Measures

Table 2: Department of Health and Human Services (HHS) Efforts to Address Quality Measure Misalignment

HHS effort Key activities

Factors driving quality measure 

misalignment that the effort aims 

to address

HHS 

Measurement 

Policy Council

Initiated in 2012, the Measurement Policy Council considers and strives to 

reach agreement across HHS agencies on selected measures to assess 

quality for selected conditions, such as hypertension. 

Dispersed decision-making

The Centers for Medicare & Medicaid Services (CMS), the Agency for 

Healthcare Research and Quality, and other HHS agencies have 

representation on the Measurement Policy Council.

Core Quality 

Measures 

Collaborative 

(CQMC)

Initiated in 2014, the CQMC provides a venue for CMS and health plans 

to negotiate sets of core measures on which they agree to focus their 

physician quality performance measurement for certain conditions. 

Dispersed decision-making

Physician specialty societies, employer groups, consumer groups, and 

regional collaboratives have been included in the negotiations.

CMS Quality 

Measure 

Development Plan

Issued in 2016, the Quality Measure Development Plan outlines the 

quality measurement activities that CMS has underway to implement the 

Quality Payment Program for physician payment reform. 

Variation in data systems

Few meaningful measures

The Quality Measure Development Plan includes activities to facilitate the 

development of new measures intended to fill gaps in existing measures. 

The Quality Measure Development Plan includes activities to facilitate 

development of electronic quality measures, which are measures whose 

specifications have been adapted to enable automated collection of data 

from electronic health records (EHRs).

National Quality 

Forum 

Measure 

Applications 

Partnership

Created in 1999, the National Quality Forum evaluates and endorses 

quality measures, applying criteria that include a focus on 

harmonization—the alignment of measures with respect to their 

specifications—and the selection of “best in class” measures. 

Few meaningful measures

Since 2011, under contract with HHS, the forum has convened the 

Measure Applications Partnership to obtain external stakeholder input 

annually on measures that CMS is considering for inclusion in its quality 

programs (prior to the Secretary’s ultimate decisions on which measures 

to include) and to identify priority areas for future measure development.

Office of the 

National 

Coordinator for 

Health Information 

Technology (ONC)

Created in 2004, ONC has promoted data interoperability and works, for 

example, with the National Library of Medicine to develop and share 

standardized data elements that contribute to the development of 

electronic quality measures.a

Variation in data systems

Source: GAO review of HHS and National Quality Forum documents and interviews with agency officials. | GAO-17-5

a

Standardized data elements are a type of information contained within a health information 

technology data system, such as an EHR, where each individual piece of information adheres to 

universal specifications for content, structure, and the format in which it is stored. Standardized data 

elements should be interchangeable without requiring translation or transformation from one quality 

measure or data system to another.

As shown in table 2, HHS’s multiple ongoing efforts address in some way 

each of the factors that drive quality measure misalignment. We describe 
Page 18 GAO-17-5 Alignment of Health Care Quality Measures

in more detail below how these efforts work within the context of HHS’s 

quality programs to address the factors driving misalignment.

HHS has taken steps to address misalignment caused by dispersed 

decision-making over quality measures, both among its own programs 

and across federal and private quality improvement efforts. From June 

2012 through January 2014, HHS’s Measurement Policy Council formally 

considered and approved seven sets of common measures, each 

focused on a different medical condition, for use across HHS agencies.28

Since January 2014, the Measurement Policy Council has not approved 

additional sets of measures because, according to HHS officials, the 

focus of their alignment efforts has shifted to the negotiations conducted 

through the CQMC with private health plans and other external 

stakeholders.

Through the CQMC, HHS has begun to address directly the complaints of 

physicians who currently report on an array of differing quality measures 

to multiple private health plans as well as Medicare. Although decisionmaking authority remains dispersed among payers, CQMC provides a 

venue for HHS and private payers to negotiate on new sets of aligned 

measures, which include the measures that they can all agree to use in 

their physician quality reporting programs. In February 2016, CQMC 

released its first seven sets of core measures, which focus on different 

primary care and specialty care conditions.29 CQMC stated that it intends 

to continually update these core measure sets as more meaningful 

measures are developed over time.

Because the CQMC has only recently released its first round of agreedupon core measure sets, it remains to be seen how quickly and 

completely they will be implemented. CMS reported that changes to the 

 28The Measurement Policy Council has decided on common measures across HHS 

programs for the following medical conditions: hypertension, smoking cessation, 

depression screening, hospital-acquired conditions, HIV/AIDS, perinatal, and obesity/body 

mass index.

29These seven core measure sets focus on primary care, cardiology, orthopedics, medical 

oncology, obstetrics and gynecology, gastroenterology, and HIV/hepatitis C. In addition to 

CMS officials and representatives of the private health plans, several medical specialty 

societies as well as representatives of regional health collaboratives, employers, and 

consumers took part in these negotiations.

Dispersed decision-making
Page 19 GAO-17-5 Alignment of Health Care Quality Measures

quality measures used in Medicare programs will continue to be made 

through formal rulemaking, and comparable changes in quality measures 

reported to private health plans will be made incrementally as the plans 

update or renew their contracts with different providers. While the CQMC 

has allowed participants to make certain decisions together, the 

negotiations for the initial core measure sets did not include all of the 

health plans, medical specialty groups, and other stakeholders involved in 

quality reporting programs. Moreover, limitations in available measures 

and the need for agreement across multiple stakeholders has, according 

to some experts we interviewed, led to the selection of several “lowest 

common denominator” measures that are not sufficiently meaningful. 

CMS and health plan officials acknowledge that the initial sets of core 

measures are just a first step, and the officials stress their interest in 

agreeing on more meaningful quality measures as such measures 

become available. Notably, each of the seven core measure sets include 

a list of specific topics for future measure development. According to 

CMS officials, these lists are intended in part to provide measure 

developers a guide as to what gaps in existing measures they should 

consider addressing in the future.

HHS has generally addressed variation in quality data collection and 

reporting systems primarily through its efforts to promote the use of EHRs 

for quality measurement. To reduce the burden of quality reporting for 

physicians and other providers, HHS has worked to develop both 

electronic quality measures and associated standardized data elements 

that will allow providers to use their EHRs to report on quality measures 

automatically to CMS. Implementing electronic quality measures with 

standardized data elements could in turn promote greater quality 

measure alignment to the extent that increased use of those measures 

leads more physicians to electronically report quality measures with 

uniform measure specifications. The Quality Measure Development Plan 

describes a number of CMS efforts that are intended to facilitate the 

development of electronic quality measures. CMS has developed 

specifications for 64 specific electronic quality measures for physicians as 

part of its EHR incentive program. However, HHS has noted that 

implementation of these electronic quality measures has been difficult in 

practice due in part, according to the National Quality Forum, to 

differences in design among EHR systems offered by different EHR 

vendors.

HHS is also working to simplify the task of developing and implementing 

electronic quality measures by promoting the development of 

standardized data elements across different EHR systems. This work is 

Variation in data collection and 

reporting systems
Page 20 GAO-17-5 Alignment of Health Care Quality Measures

led, for the most part, by ONC. ONC’s efforts have focused primarily on 

data elements used for purposes other than quality measurement, such 

as exchanging patient demographic and medical history information 

among providers. However, according to ONC officials, ONC also 

provides support and resources to facilitate the development of 

standardized data elements needed for the development of new 

electronic quality measures.

CMS has taken steps to address the paucity of meaningful measures 

through efforts to facilitate and streamline the development of new 

measures to fill identified gaps, which has the potential to allow for 

greater agreement on measures among CMS and private payers. Some 

of these efforts involve coordinating and sharing resources with measure 

developers working outside of government to develop new quality 

measures. In other cases, CMS itself leads and funds new measure 

development. The impending implementation of the new physician Quality 

Payment Program has made development of more meaningful quality 

measures to assess physician performance a particular priority for CMS.

The Quality Measure Development Plan outlines what CMS has done or 

plans to do to promote the development of new, more meaningful, quality 

measures for physicians under the Quality Payment Program and also to 

promote greater alignment across those measures. For example, the 

Quality Measure Development Plan describes CMS efforts to make 

development tools and measure testing resources available to private 

sector measure developers. It also describes efforts to encourage 

measure developers to coordinate their approach in designing new 

measures, such as by incorporating the same methodologies for patient 

risk adjustment into multiple measures. However, some of these activities 

are in early stages of implementation and others were launched in the last 

year or two, so it remains to be seen how much effect these activities will 

ultimately have on the development of new, more meaningful quality 

measures for the Quality Payment Program.

The Quality Measure Development Plan spells out a number of key 

considerations in selecting new measures for development. They include 

addressing an important medical condition or topic where there is 

demonstrated variation in the care offered by different providers—and 

therefore opportunity for improvement. To the extent that these new 

measures generate more useful information on how to improve care, they 

may also win stronger support from providers who have been skeptical of 

the benefit offered by previously available quality measures.

Few meaningful measures
Page 21 GAO-17-5 Alignment of Health Care Quality Measures

As new, more meaningful measures are developed and tested by CMS 

and private sector developers, CMS can promote alignment by 

incorporating them into its quality reporting and pay-for-performance 

programs for different types of providers, including physicians. Every year 

CMS goes through a multi-step process to consider changes to the 

measures used in those programs. Quality measure alignment is one of 

the key criteria that CMS applies in deciding which new measures to add 

and which measures used in prior years to drop. In making these 

decisions, CMS states that it aims to avoid duplication across quality. 

HHS contracts with the National Quality Forum to apply these criteria 

through the forum’s measure endorsement process and to conduct the 

annual Measure Applications Partnership review of measures under 

consideration for CMS quality programs. As part of this process, the 

Measure Applications Partnership seeks to identify measures that are the 

“best available”—that is, the most meaningful—when competing 

measures are under consideration. Although these Measure Applications 

Partnership processes are limited to the particular set of measures that 

CMS selects for consideration in a given year, over time these decisions 

to add and drop measures can lead to an increasingly aligned and more 

meaningful set of quality measures. In addition, each year, the Measure 

Applications Partnership identifies priority areas for new measure 

development, which may help to inform CMS decisions about which new 

measures to promote and in which specific areas (such as cardiology or 

care coordination), so that more meaningful measures in those areas will 

be available for CMS to consider in future years.

Although CMS and private health plans have agreed to align on the seven 

CQMC core measure sets, HHS has not placed a specific priority on 

developing electronic quality measures and associated standardized data 

elements for the measures included in those sets. As we described earlier 

in this report, both CMS and ONC have set objectives for their efforts to 

develop electronic quality measures that will facilitate data collection for 

quality measurement, and which could also have the potential to reduce 

misalignment. These objectives have included the creation of specific 

electronic quality measures that physicians and other providers may use 

to qualify for payments under the Medicare EHR incentive program. The 

objectives also include supporting the efforts of medical specialties and 

other organizations engaged in developing electronic quality measures. 

However, as of May 2016, CMS and ONC had not included in these 

objectives the development of new electronic quality measures for the 

aligned measures included in the seven CQMC core measure sets. While 

some of these measures already had electronic quality measure 

HHS Has Not Prioritized 

Efforts to Develop 

Electronic Quality 

Measures for Aligned Core 

Measures Adopted by 

Both CMS and Private 

Payers
Page 22 GAO-17-5 Alignment of Health Care Quality Measures

specifications developed under the Medicare EHR incentive program, 

many others did not. Because CMS and private payers have agreed to 

align their quality measurement requirements as much as possible on the 

CQMC core measure sets, a growing number of physicians are likely to 

seek to electronically collect data to report on these measures. HHS 

officials stated that there was broad support within the department for 

promoting alignment through the implementation of the CQMC core 

measure sets, and that measures from those core sets will be taken into 

consideration when deciding which electronic measures to develop. 

However, the officials explained that their ability to develop new electronic 

measures is constrained by a number of practical limitations, such as the 

availability of resources and the feasibility of constructing electronic 

versions of certain quality measures using current EHR technology.

Further, HHS officials noted that CMS is one payer among many, 

including a large number of private payers, which can make it more 

challenging to develop electronic measures.

Developing electronic quality measures and standardized data elements 

for all the measures in the CQMC core measure sets, when feasible,

could also help HHS meet its related objective to reduce provider burden. 

According to HHS documents and statements by experts, quality 

reporting becomes easier for physicians and other providers if they have 

the capacity to report their performance on quality measures using their 

EHRs. HHS documents and statements by experts have articulated an 

overarching goal to create an interconnected system of EHRs that 

enables physicians to automatically record the needed information for 

quality measures as part of their normal clinical workflow, and then to 

report their performance to payers directly from their EHRs. According to 

HHS documents and statements by experts, this capability requires that 

each quality measure in use be specified as an electronic quality measure 

and that each EHR be set up to accommodate the standardized data 

elements required for those electronic quality measures.30

The recent adoption of the seven CQMC core quality measure sets by 

CMS and private payers creates an opportunity for CMS and ONC to 

 30Electronic health measures can be specified without standardized data elements, but in 

their absence implementation of the measures is much more difficult and uncertain. HHS 

documents note that electronic health measure specifications currently rely on complex 

implementation guidance that frequently is applied inconsistently across different EHRs. 
Page 23 GAO-17-5 Alignment of Health Care Quality Measures

focus their electronic quality measure efforts on these quality measures, 

which are most likely to reduce misalignment and provider burden. While 

CMS and ONC efforts have previously focused on different measures, 

federal internal controls call for agencies to respond to changed 

conditions by revising programs to maintain their effectiveness. To reduce 

misalignment and provider administrative burden and be consistent with 

federal internal controls, CMS and ONC would have to also take steps to 

prioritize their efforts to promote the development of both electronic 

quality measures and related standardized data elements for aligned 

measures that HHS and private payers have agreed to use.31

Without electronic quality measures and related standardized data 

elements for all the CQMC core measures, physicians are less likely to 

experience the potential benefits of easier quality reporting using EHRs, 

as described by HHS documents and experts. To the extent that 

physicians continue to face significant administrative burdens in collecting 

the needed quality data, they may be less likely to adopt the CQMC 

aligned measures. Furthermore, CMS’s development of electronic quality 

measures for the CQMC core measure sets has the potential to address, 

to some extent, the difficulties posed by variation in design across EHR 

systems built by different IT vendors, which as previously described can 

impede obtaining consistent data to measure quality. Therefore, without 

electronic quality measures, physicians may be less able to use their 

EHRs to report their performance on the CQMC aligned measures to 

CMS and private payers. HHS would thereby miss an opportunity to 

reduce measure misalignment and physician burden through the 

automated data collection of aligned quality measures.

CMS’s efforts to develop new, more meaningful quality measures may not 

lead to greater measure alignment due to a lack of comprehensive 

planning. Increasing alignment across the measures used by federal and 

private payers is one of the objectives explicitly stated in CMS’s Quality 

Measure Development Plan, but a broad range of other objectives are 

listed there as well. They include providing clinically relevant measures 

for all medical specialties; creating more measures focusing on 

outcomes, especially patient reported outcomes; supporting improved 

 31See GAO-14-704G. 

Lack of Comprehensive 

Planning Could 

Jeopardize CMS Efforts to 

Address Misalignment
Page 24 GAO-17-5 Alignment of Health Care Quality Measures

integration of physical and behavioral health; assessing team-based care; 

and fully engaging the perspectives of patients and their caregivers in 

measure development. The plan provides no clear indication of how 

CMS’s ongoing measure development efforts will help CMS to achieve its 

goal of reducing measure misalignment, such as through the 

development of new quality measures that are most likely to help promote 

quality measure alignment among federal and private payers. For 

example, when asked to explain the decision to develop a particular 

measure, CMS officials described the specific measure gap that the 

measure was intended to fill. However, they did not explain how 

developing that measure, as compared to others, would best meet the 

objectives outlined in the Quality Measure Development Plan, including 

reducing measure misalignment. Moreover, in addition to not indicating 

how it would address measure misalignment, CMS has not set timelines 

and methods for tracking its progress in meeting this objective.

CMS is nonetheless making decisions to develop specific new quality 

measures that will affect how well its measure development efforts 

address misalignment. The development, testing, and implementation of 

each individual quality measure require substantial time and resources. 

Consequently, the choices that CMS makes now on new measure 

development will influence its ability to meet its goal to develop measures 

that facilitate greater alignment. The CQMC core measure sets, which 

represent CMS’s main effort to address misalignment across federal and 

private payers, provide a notable example. As noted earlier, the 

negotiations over alignment on the initial CQMC core measure sets were 

constrained by a lack of sufficient meaningful measures, and each of the 

initial seven core measure sets identified a list of targeted topics for future 

measure development for that core measure set, which have the potential 

to increase alignment. However, as of May 2016, CMS’s Quality Measure 

Development Plan had not made development and testing of these 

CQMC-identified measures an explicit objective of CMS’s measure 

development efforts.32 CMS officials stated that they see some topical 

overlap between the quality measures they have currently under 

development and the topics targeted for development in the core 

 32The Quality Measure Development Plan describes CMS’s support for the CQMC and its 

plans to continue actively participating in the CQMC, but the Quality Measure 

Development Plan makes no reference to addressing the measure development needs 

identified in the initial CQMC core measure sets. 
Page 25 GAO-17-5 Alignment of Health Care Quality Measures

measure sets. They also said that they intend to take the core measure 

sets into account in their decisions on additional new measure 

development. However, they have also noted that implementation of the 

Quality Payment Program has led to other objectives that compete for 

their limited measure development resources, including an objective to 

develop a sufficient number of quality measures that are appropriate for 

different medical specialties. Many of these specialties are not covered by 

the CQMC core measure sets.

This lack of comprehensive planning is inconsistent with federal internal 

control standards, which call for agencies to identify the timeframes for 

defined objectives and to assess their progress toward achieving their 

objectives.33 Furthermore, this lack of comprehensive planning is 

inconsistent with leading principles on sound planning we have identified 

in our prior work, which call for developing robust, comprehensive plans 

to achieve their goals. Specifically, we have determined in prior work that 

sound plans include such components as what the plan is trying to 

achieve and how it will achieve those results, as well as priorities, 

milestones, and performance measures to monitor and gauge the 

results.34

Without comprehensive planning for how to target development of new, 

more meaningful measures that address misalignment, CMS cannot 

ensure that it will meet its objective to reduce measure misalignment. In 

particular, if CMS decisions on measure development do not address the 

identified needs for more meaningful measures to be included in the 

CQMC core measure sets, then adoption of those core measure sets is 

likely to be less attractive to physicians and private payers. Since 

adoption of the core measure sets would reduce misalignment for the 

physicians and payers that adopt them, slower adoption is likely to lead to 

slower progress in achieving measure alignment. The CQMC core 

measure sets represent CMS’s main effort to address the misalignment of 

 33See GAO-14-704G. 

34For example, see GAO, Executive Guide: Effectively Implementing the Government 

Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1996), GAO, 

Combating Terrorism: Evaluation of Selected Characteristics in National Strategies 

Related to Terrorism, GAO-04-408T (Washington, D.C.: Feb. 3, 2004) and GAO, 

Veterans’ Health Care: Proper Plan Needed to Modernize System for Paying Community 

Providers, GAO-16-353 (Washington, D.C.: May 11, 2016). 
Page 26 GAO-17-5 Alignment of Health Care Quality Measures

quality measures between federal and private payers—where much of the 

misalignment that affects physicians and other providers originates. 

Unless CMS targets development of new quality measures that can 

promote agreement on further alignment among providers and payers, 

the addition of new quality measures could increase, rather than 

decrease, misalignment by increasing the choices of available quality 

measures. Moreover, without setting explicit timelines and methods for 

tracking progress in achieving its objective to decrease misalignment, 

CMS cannot determine which of its efforts to develop more meaningful 

measures help to reduce misalignment, and which efforts need to be 

modified.

The use of health care quality measures is central to HHS’s and other 

payers’ efforts to improve health care quality. While quality measures can 

encourage improvements in care, they can also be burdensome to 

providers when the measures are misaligned and providers have to report 

different quality measures to different health care payers. Such 

misalignment has the potential to affect the success of HHS’s efforts to 

pay providers based on the quality of care they provide. HHS has 

acknowledged the need to substantially improve quality measurement for 

physicians and other providers, and has a stated goal of improving 

alignment between federal and private payers.

Although HHS has initiated a range of different efforts to reduce 

misalignment, we identified two deficiencies in these efforts. First, HHS 

has not prioritized the development of electronic quality measures with 

standardized data elements for the core sets of aligned measures. CMS 

and private payers have prioritized certain quality measures to be used in 

common so alignment is improved, but HHS has not focused resources 

on developing electronic quality measures for these quality measures. As 

providers increasingly use EHRs, HHS has the opportunity to pursue its 

stated objective to develop electronic quality measures that would allow 

physicians to collect automatically much of the clinical information needed 

for these measures as part of their normal clinical workflow, with a 

consequent decrease in the administrative burden faced by physicians.

Second, CMS has not comprehensively planned its measure 

development efforts to ensure the development of new, more meaningful 

quality measures targeted to reduce misalignment, which could 

jeopardize CMS’s efforts to achieve that goal. Some experts we 

interviewed told us that the paucity of meaningful measures makes it 

difficult for payers and providers to agree on aligned measures. In 

Conclusions
Page 27 GAO-17-5 Alignment of Health Care Quality Measures

contrast, developing new, more meaningful measures to replace older, 

less meaningful ones could help to promote further agreement on aligned 

measures and gain broader support from physicians. However, CMS’s 

plans do not indicate how its efforts will target new measures that will lead 

to greater alignment, rather than simply adding to the array of available 

quality measures that has led to misalignment in the past. Achieving 

greater alignment will make it more likely that the efforts of CMS and 

private payers to hold providers accountable for the quality of their care, 

including CMS’s Quality Payment Program, will reduce administrative 

burden and provide more meaningful information that providers can use 

to identify high-impact quality improvements.

To make it more likely that HHS will achieve its goals to reduce quality 

measure misalignment and associated provider burden, we recommend 

that the Secretary of HHS take two actions:

1. Direct CMS and ONC to prioritize their development of electronic 

quality measures and associated standardized data elements on the 

specific quality measures needed for the core measure sets that CMS 

and private payers have agreed to use.

2. Direct CMS to comprehensively plan, including setting timelines, for 

how to target its development of new, more meaningful quality 

measures on those that will promote greater alignment, especially 

measures to strengthen the core measure sets that CMS and private 

payers have agreed to use.

We provided a draft of this report to HHS for review, and HHS provided 

written comments, which are reprinted in appendix II. HHS also provided 

technical comments, which we incorporated as appropriate. In its written 

comments, HHS concurred with our recommendations and noted a 

number of its activities to help reduce quality measure misalignment. For 

example, as noted in our report, HHS has been working with private 

payers and other stakeholders to identify core sets of quality measures as 

part of the Core Quality Measures Collaborative. HHS stated that it has 

also prioritized the development of electronic measures in general. Our 

first recommendation focuses specifically on the need to prioritize the 

electronic measures included in the agreed upon core measure sets, 

because these measures are likely to address misalignment. HHS 

affirmed its intention to prioritize the development of these electronic 

measures, to the extent feasible, while noting some limitations with 

Recommendations for 

Executive Action

Agency Comments

and Our Evaluation
Page 28 GAO-17-5 Alignment of Health Care Quality Measures

current EHR technology that can put constraints on the development of 

new electronic measures.

In response to our second recommendation, HHS stated that it intends to 

refine planning documents to ensure that they include comprehensive 

planning. HHS noted that it cannot require other entities that develop 

measures, such as private payers, to adhere to particular timelines; 

however, our recommendation is for CMS to target its own development 

efforts on the measures that will promote greater alignment. Furthermore, 

as noted in our report, HHS has a critical role in quality measurement and 

has an opportunity to provide leadership to private payers and other 

stakeholders as they work to independently implement core measure sets 

and increase quality measure alignment.

We are sending copies of this report to the Secretary of Health and 

Human Services and other interested parties. In addition, the report will 

be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact 

me at (202) 512-7114 or at clowersa@gao.gov. Contact points for our 

Office of Congressional Relations and Office of Public Affairs can be 

found on the last page of this report. Other major contributors to this 

report are listed in appendix III.

A. Nicole Clowers

Managing Director, Health Care
Page 29 GAO-17-5 Alignment of Health Care Quality Measures

List of Committees

The Honorable Orrin Hatch

Chairman

The Honorable Ron Wyden

Ranking Member

Committee on Finance

United States Senate

The Honorable Lamar Alexander

Chairman

The Honorable Patty Murray

Ranking Member

Committee on Health, Education, Labor and Pensions

United States Senate

The Honorable Fred Upton

Chairman

The Honorable Frank Pallone

Ranking Member

Committee on Energy and Commerce

House of Representatives

The Honorable Kevin Brady

Chairman

The Honorable Sander Levin

Ranking Member

Committee on Ways and Means

House of Representatives
Appendix I: Literature Review Methodology

Page 30 GAO-17-5 Alignment of Health Care Quality Measures

To examine what is known about the extent and effects, if any, of health 

care quality measure misalignment, we conducted a literature review to 

identify relevant studies published in peer-reviewed journals, trade and 

association publications, conference papers, and government reports 

published from January 2010 to February 2016, as well as a more 

general internet search.1 We searched more than 25 databases for 

research published in relevant peer-reviewed journals, trade and 

association publications, and government sources, including BIOSIS 

Previews®, COS Conference Papers Index, Embase®, ProQuest 

Biological & Health Science Professional, MEDLINE®, New England 

Journal of Medicine, SciSearch®, and SCOPUS. Key search terms 

included various combinations of the terms “healthcare,” “quality or 

performance,” “measure,” “align, misalign, or vary,” “benchmark or 

impact,” and “Medicare or Medicaid,” among others. After excluding 

duplicates, we identified and reviewed 202 abstracts.

For those abstracts we found relevant, we obtained and reviewed the full 

study and selected nine that were relevant to (1) quantifying the extent of 

quality measure misalignment; (2) describing the effects of misalignment, 

including burden to providers; or (3) identifying the factors that contribute 

to misalignment. We also considered whether the studies provided insight 

on efforts to address quality measure misalignment. In addition to our 

literature review, we identified four publications through interviews with an 

expert and Department of Health and Human Services officials that were 

relevant to our report. We examined the methodologies for each of these 

studies and interviewed some of their authors. We determined that the 

studies were sufficiently reliable for our purposes. For a complete list of 

the studies, see the Bibliography.

 1

We selected this timeframe to ensure that we captured literature that best reflects the 

current state of quality measurement efforts—particularly those that may affect measure 

alignment—since the enactment of the Patient Protection and Affordable Care Act in 

2010, which, among other things, included requirements for quality measurement.

Appendix I: Literature Review Methodology
Appendix II: Comments from the Department 

of Health and Human Services

Page 31 GAO-17-5 Alignment of Health Care Quality Measures

Appendix II: Comments from the Department 

of Health and Human Services
Appendix II: Comments from the Department 

of Health and Human Services

Page 32 GAO-17-5 Alignment of Health Care Quality Measures
Appendix II: Comments from the Department 

of Health and Human Services

Page 33 GAO-17-5 Alignment of Health Care Quality Measures
Appendix II: Comments from the Department 

of Health and Human Services

Page 34 GAO-17-5 Alignment of Health Care Quality Measures
Appendix III: GAO Contact and Staff 

Acknowledgments

Page 35 GAO-17-5 Alignment of Health Care Quality Measures

A. Nicole Clowers, (202) 512-7114 or clowersa@gao.gov

In addition to the contact named above, Linda T. Kohn, Director; Will 

Simerl, Assistant Director; Amy Leone, Analyst-in-Charge; Karen Belli, 

Kye Briesath, Krister Friday, and Eric Peterson made key contributions to 

this report. Also contributing were JoAnna Berry, Vikki Porter, Lisa 

Rogers, and Jennifer Whitworth.

Appendix III: GAO Contact and Staff 

Acknowledgments

GAO Contacts

Staff 

Acknowledgments
Bibliography

Page 36 GAO-17-5 Alignment of Health Care Quality Measures

Angier, H., R. Gold, C. Gallia, A. Casciato, C.J. Tillotson, M. Marino, R. 

Mangione-Smith, and J.E. DeVoe. “Variation in Outcomes of Quality 

Measurement by Data Source.” Pediatrics, vol. 133, no. 6, (2014).

Bazinsky, Kate, and Michael Bailit. The Significant Lack of Alignment 

Across State and Regional Health Measure Sets, Health Care 

Performance Measurement Activity: An Analysis of 48 State and Regional 

Measure Sets. Needham, Mass.: Bailit Health Purchasing, LLC, 2013.

Casalino, L.P., D. Gans, R. Weber, et al. “US Physician Practices Spend 

More Than $15.4 Billion Annually to Report Quality Measures.” Health 

Affairs, vol. 35, no. 3, (2016).

Centers for Medicare & Medicaid Services. 2015 National Impact 

Assessment of the Centers for Medicare & Medicaid (CMS) Quality 

Measures Report. Baltimore, Md.: March 2, 2015.

DuGoff, E.H., S. Dy, E.R. Giovannetti, B. Leff, and C.M. Boyd. “Setting 

Standards at the Forefront of Delivery System Reform: Aligning Care 

Coordination Quality Measures for Multiple Chronic Conditions.” Journal 

for Healthcare Quality, vol. 35, no. 5, (2013).

Eapen, Z.J., F. Tang, P.G. Jones, T.M. Maddox, W.J. Oetgen, J.A. 

Spertus, J.S. Rumsfeld, O.A. Heidenreich, E.D. Peterson, and J.P. 

Drozda, Jr. “Variation in performance measure criteria significantly affects 

cardiology practice rankings: Insights from the National Cardiovascular 

Registry’s Practice Innovation and Clinical Excellence Registry.” 

American Heart Journal, vol. 169, no. 6, (2015).

Higgins, A., G. Veselovskiy, and L. McKnown. “Provider Performance 

Measures in Private and Public Programs: Achieving Meaningful 

Alignment with Flexibility to Innovate.” Health Affairs, vol. 32, no. 8, 

(2013).

Institute of Medicine of the National Academies. Vital Signs: Core Metrics 

for Health and Health Care Progress. Washington, D.C.: National 

Academies Press, 2015.

Kanger, C., L. Brown, S. Mukherjee, H. Xin, M. Diana, and A. Khurshid. 

“Evaluating the Reliability of EHR-Generated Clinical Outcomes Reports: 

A Case Study.” eGEMS (Generating Evidence & Methods to improve 

patient outcomes), vol. 2., no. 3, (2014).

Bibliography
Bibliography

Page 37 GAO-17-5 Alignment of Health Care Quality Measures

Meyer, G.S., E.C. Nelson, D.B. Pryor, B. James, S.J. Swenson, G.S. 

Kaplan, J.I. Weissberg, M. Bisognano, G.R. Yates, and G.C. Hunt. “More 

quality measures versus measuring what matters: a call for balance and 

parsimony.” BMJ Quality & Safety, vol. 21. no. 11, (2012).

Navar-Boggan, A.M., B.R. Shah, J.C. Boggan, J.A. Stafford, and E.D. 

Peterson. “Variability in performance measures for assessment of 

hypertension control.” American Heart Journal, vol. 165, no. 5, (2013).

Panzer, R.J., R.S. Gitomer, W.H. Greene, P.R. Webster, K.R. Landry, 

and C.A. Riccobono. “Increasing Demands for Quality Measurement.” 

JAMA, vol. 310, no. 18, (2013).

Zainulbhai, S., L. Goldberg, W. Ng, and A.E. Montgomery. “Assessing 

Care Integration for Dual-Eligible Beneficiaries: A Review of Quality 

Measures Chosen by States in the Financial Alignment Initiative.” The 

Commonwealth Fund, Issue Brief, vol. 2, pub. 1734, (2014).

(100344)
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to http://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.