Department of Health and Human Services 
 
 
OFFICE OF INSPECTOR GENERAL   
     
 
 
 
 
   
E I R :  
ARLY  MPLEMENTATION  EVIEW   
CMS M
S  ANAGEMENT OF THE 
Q P P  
UALITY  AYMENT  ROGRAM
Suzanne Murrin 
Deputy Inspector General for  
Evaluation and Inspections  
OEI-12-16-00400 
December 2016  Report in Brief 
December 2016 
Report N o. OEI-12-16-00400 
Early Implementation Review:  CMSs 
Why OIG Did This Review  
The Medicare Access and CHIP  Management of the Quality Payment Program  
Reauthorization Act of 2015 (MACRA) 
enacted clinician payment reforms 
What OIG Found 
designed to promote quality and value 
From our analysis, we identified CMSs five key  Key Takeaway 
of care.  These reforms, known as the 
management priorities regarding the agencys 
Quality Payment Program (QPP), are a  CMS has made significant 
planning and early implementation of the QPP.  
significant shift in how Medicare  progress towards 
calculates compensation for clinicians  Early on, CMS staff decided that clinicians  implementing the QPP.  
and require the Centers for Medicare  acceptance of the QPP, and readiness to  Although many milestones 
& Medicaid Services (CMS) to develop  participate in it, would be the most critical  remain before the QPP 
a complex system for measuring,  factor to ensuring the programs success.  This  payment adjustments in 2019, 
reporting, and scoring the value and  focus on clinicians informed CMSs  OIG identified two 
quality of care.  CMS issued final  vulnerabilities that are critical 
decisionmaking regarding its other 
regulations on October 14, 2016, and  for CMS to address in 2017, 
management priorities, including:   
the first performance year will begin  because of their potential 
January 1, 2017, with the first    adopting integrated internal business  impact on the programs 
payment adjustments taking effect on  practices to accommodate a flexible,  success:   
January 1, 2019.  Clinicians may  user-centric approach;   (1) providing sufficient 
participate in one of two QPP tracks:  
guidance and technical 
the Merit-Based Incentive Payment    developing information technology (IT) 
assistance to ensure that 
System (MIPS) or Advanced  systems that support and streamline 
clinicians are ready to 
Alternative Payment Models  clinician participation;  
participate in the QPP, and   
(Advanced APMs). 
  developing flexible and transparent 
  (2) developing IT systems to 
Given the importance and complexity  MIPS policies; and   support data reporting, 
of these reforms and the tight  scoring, and payment 
  facilitating participation in Advanced 
timeline, OIG conducted an early  adjustment.  
APMs. 
implementation review of CMSs 
management of the QPP.  We did not  As of December 2016, CMS had finalized key 
assess the extent to which the QPP  policies to implement the QPP, including issuing final regulations and 
will be successful in meeting program 
identifying Medicare models that qualify as Advanced APMs for the 
requirements and goals.   
first performance period.  CMS had also initiated engagement and 
outreach activities to clinicians, launched a public-facing informational 
How OIG Did This Review 
website, and awarded various contracts for technical assistance and 
We interviewed CMS staff and 
training.   
reviewed internal CMS documents as 
well as publicly available information.   CMS must still expand its technical assistance efforts, issue promised 
We conducted qualitative analysis to  subregulatory guidance, award and oversee key contracts, and 
identify key milestones (both those  complete development of backend IT systems necessary to support 
achieved and those yet to come),  critical QPP operations. 
priorities, and challenges related to 
 
QPP implementation. 
  Full report can be found at http://oig.hhs.gov/oei/reports/oei-12-16-00400.asp  Background   
T 
 o improve care for Medicare beneficiaries, the Medicare Access and CHIP (Childrens Health 
Insurance Program) Reauthorization Act of 2015 (MACRA) moves Medicare from a 
volume-based payment system to one that rewards value.1  The Centers for Medicare & 
Medicaid Services (CMS) is implementing core provisions of MACRA as the Quality Payment 
Program (QPP), a set of clinician payment reforms designed to put increased focus on the quality and 
value of care.2  The QPP is a significant shift in how Medicare calculates payment for clinicians and 
requires CMS to develop a complex system for measuring, reporting, and scoring the value and quality 
of care.  MACRA mandated that the first payment adjustments based on QPP performance measures 
go into effect on January 1, 2019.  To meet this statutory deadline, CMS issued final regulations 
implementing the QPP on October 14, 2016, and determined that the first performance year for 
clinicians will begin January 1, 2017.3, 4   
 
Given the importance and complexity of the new payment system and the tight timeline to launch the 
QPP, OIG conducted an early implementation review of CMSs management of the program.  The 
objective of this review was to assess CMSs progress and identify key challenges and potential 
vulnerabilities that CMS faces as it implements the QPP.  
 
CLINICIAN PAYMENT UNDER MEDICARE PART B  
Medicare pays clinicians (such as physicians and nurse practitioners) for their services through the 
Part B benefit.  Clinician services include office visits and surgical, diagnostic, and therapeutic 
procedures.  CMS bases its payment rates for over 7,000 clinician services on the Medicare Physician 
Fee Schedule.5,6  Prior to the passage of MACRA, these payment rates were intended to be updated 
annually using the sustainable growth rate (SGR) formula.7  The SGR was designed by Congress to 
control Medicare spending by either reducing or increasing Part B payment rates when spending fell 
above or below a set target.8, 9  However, many stakeholders criticized the projected annual payment 
rate reductions under the SGR as being too severe.  As a result, Congress overrode the SGR payment 
rate reduction each year from 2003 to 2015 and opted to either maintain or increase payment rates, an 
annual process that came to be known as the doc fix.10  Under this system, Medicare reimbursed 
clinicians based on the volume of services provided.   
 
To begin linking payment to value, Congress created multiple programs between 2006 and 2010 that 
established incentives and penalties related to quality reporting and performance.11, 12, 13, 14, 15  These 
included the Physician Quality Reporting System (PQRS), the Value-Based Payment Modifier (VBM), and 
the Electronic Health Record (EHR) incentive programs.   
  The PQRS initially provided incentive payments to clinicians who reported specific quality 
measures; since 2015, however, clinicians no longer receive incentives and instead receive a 
negative payment adjustment if they fail to report these data.16 
  The VBM adjusts clinician payment up or down based on the quality of care provided as 
compared with the cost of care during a performance period.17   
  The Medicare physician EHR incentive program provides incentive payments to clinicians who 
adopt and meaningfully use health information technology to improve patient care and 
outcomes.  In addition, beginning in 2015, eligible clinicians who did not demonstrate 
meaningful use could receive a negative payment adjustment.18   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  1 
 
  Under MACRA, CMS will phase out all three of these programs in favor of a new set of payment reforms 
built on these prior efforts.   
 
OVERVIEW OF THE QUALITY PAYMENT PROGRAM  
MACRA was enacted in April 2015 with bipartisan support.  It repealed the SGR formula and increased 
Medicare Physician Fee Schedule rates 0.5 percent each year from 2016 through 2019.19  Additionally, 
MACRA phased out Medicare payment adjustments made under the PQRS, VBM, and Medicare 
physician EHR incentive programs.  In place of these programs, MACRA required that CMS make 
quality- and value-based incentive payments to clinicians20 through one of two tracks:  the Merit-Based 
Incentive Payment System (MIPS) or Advanced Alternative Payment Models (Advanced APMs).  CMS 
refers to these two tracks together as the QPP.  
 
MERIT-BASED INCENTIVE PAYMENT SYSTEM 
Using the Physician Fee Schedule as a base rate, MIPS will adjust clinicians Medicare Part B payments 
up or down based on their performance in four categories, which consolidate elements of the existing 
programs that MIPS replaces.  The four MIPS categories are:   
  quality (replaces the PQRS),  
  cost (replaces the VBM),  
  advancing care information (replaces the Medicare physician EHR incentive program), and 
  improvement activities (a new performance category).21   
 
Within each category, CMS has defined a set of possible measures or activities.  For the Cost category, 
CMS calculates certain measures based on claims data.  For all other categories, clinicians select from a 
menu of CMS-approved measures and must report data to CMS.  For example, in the Quality category, 
clinicians must select and report on 6 measures or a set of specialty measures defined by CMS.  
Clinicians may use a variety of options to report these data to CMS; the data can be reported via claims 
data, electronic health records, and qualified clinical data registries.  Additionally, clinicians may report 
individually or as part of a group, and they may use vendors or surrogates (e.g., data registries or a 
group practices administrator) to submit MIPS data on their behalf.    
 
For each clinician, CMS calculates a MIPS Final Score based on the providers performance in each of 
the four categories.  The four areas do not carry equal weight; for example, in 2017, clinicians 
performance on measures in the Quality category will account for 60% of their MIPS Final Score.  (See 
Appendix A for further detail about MIPS scoring.)   
 
CMS uses clinicians MIPS scores for a given year to adjust Medicare Part B payments 2 years laterfor 
example, CMS will use scores for 2017 to adjust payments in 2019.  To determine payment 
adjustments, CMS compares each clinicians MIPS score to a performance threshold.  For 2017, CMS 
set the threshold such that any clinician who reports a minimum amount of data will avoid a negative 
payment adjustment in 2019.  Beginning in the programs third year, CMS will set the threshold at the 
mean or median score of all MIPS-eligible clinicians for a prior period specified by the Secretary of 
Health and Human Services.  Clinicians scoring further above or below the threshold receive larger 
adjustments.  The maximum payment adjustment gradually increases from 4 percent (positive or 
negative) in 2019 to 9 percent (positive or negative) in 2022 and beyond.  MACRA established MIPS as a 
budget-neutral program, so CMS may scale positive payment adjustments under MIPS as needed to 
ensure that they do not exceed penalties.22  In addition, MACRA provided $500 million annually for 
CMS to distribute payments for exceptional performance to clinicians whose MIPS scores exceed a 
certain level.23   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  2   
Clinicians are exempted from MIPS reporting requirements if they (1) meet the low-volume threshold 
of having no more than $30,000 in Medicare Part B charges or no more than 100 Medicare 
beneficiaries during the performance period24, or (2) meet criteria for participation in the Advanced 
APM track (see below for further detail).  Providers who are exempt from MIPS because they fall below 
the low-volume threshold will continue to be paid for Medicare Part B services at the Physician Fee 
Schedule rates without an additional QPP payment adjustment. 
 
ADVANCED ALTERNATIVE PAYMENT MODELS 
The Patient Protection and Affordable Care Act (ACA) established authorities to test several alternative 
payment models (APMs).25  APMs are payment and care delivery models that are designed to 
financially reward high-quality and cost-efficient care.  Some APMs seek to achieve this by providing 
incentive payments to high performers, while others require providers to bear additional risk, sharing 
in gains and/or losses based on the overall cost of care.  Some APMs apply to specific clinical conditions 
(e.g., end-stage renal disease (ESRD)), others to a type of care episode (e.g., joint replacement), and 
others to a population (e.g., primary care patients).26   
 
Building on these initiatives, MACRA enabled providers who participate in Advanced APMs to be 
exempt from MIPS reporting and subject to a different set of incentive payments.  MACRA established 
three criteria for determining which Medicare APMs will qualify as Advanced APMs in the QPP.  Each 
Advanced APM must: 
(1) require participants to use certified EHR technology;  
(2) provide payment for covered professional services based on quality measures comparable to 
those used in the quality performance category of MIPS; and  
(3) require APM Entities27 to bear more than a nominal amount of financial risk, or be a Medical 
Home Model expanded under section 1115A(c) of the Act.28   
 
Providers with a certain proportion of patients or payments that are part of an Advanced APM are 
deemed Qualifying APM Participants for the QPP Advanced APM track.  Qualifying APM Participants 
are excluded from MIPS reporting and payment adjustment, and instead receive a 5-percent annual 
bonus during payment years 2019 through 2024.  Beginning in 2026, Qualifying APM Participants 
receive higher physician fee-schedule updates, compounded annually.  These providers may receive 
additional bonuses specific to the Advanced APMs in which they participate.   
 
TIMELINE 
The performance periods for MIPS and for Advanced APMs are broadly aligned with each other.  For 
both, the first performance period begins January 1, 2017.  MIPS adjustments and Advanced APM 
bonuses based on the 2017 performance period will go into effect in 2019.  The QPP will operate on an 
overlapping 3-year program cycle.  The first year of each cycle is the performance period; the second 
year is for reporting data and calculating scores; and the third year is for adjusting payment.  (See 
Appendix B.)  
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  3   
 
 
STAKEHOLDER CONCERNS 
Providers, professional associations, and Members of Congress have expressed a variety of concerns 
about the QPP.  CMS received over 4,000 comments on the proposed rule for the QPP published in 
May 2016,29 and has continued to receive feedback after issuing the final rule in October 2016.  
Stakeholders major concerns are summarized below: 
 
  Too burdensome for solo, small-practice, and rural providers.  Stakeholders questioned 
whether small and/or rural providers will succeed under the QPP.  Unlike large practices, small 
providers may not have the resources to hire an administrator or third-party vendor to handle 
reporting. 
 
  Too complex.  Stakeholders raised concerns about the QPPs complexityin particular, the 
complicated formula for calculating MIPS Final Scores and determining payment adjustments.  
Stakeholders also noted that if clinicians in Advanced APMs do not know until late in the 
performance period whether they have reached the threshold to be Qualifying APM 
Participants, they may still need to prepare for MIPS reportingreducing one of the incentives 
for participation in the Advanced APM track.   
 
  Applicability and validity of specific MIPS measures.  Stakeholders offered feedback about the 
availability of MIPS measures relevant to different types of clinical practice and whether the 
measures will accurately reflect clinician performance. 
 
  Paucity of Advanced APM opportunities currently available.  Stakeholders stated that more 
Advanced APM opportunities for clinicians, particularly specialists, are needed.  Some 
recommended that CMS simplify and lower the financial-risk standards for Advanced APMs. 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  4   
LESSONS FROM HEALTHCARE.GOV 
Implementing the QPP is a significant undertaking for CMS, 
requiring extensive policy and systems development within 
a relatively short timeframe.  CMS has historically faced 
Lessons from HealthCare.gov  
challenges when implementing complex initiatives of this 
size, such as the launch of the Medicare Part D program and 
 Leadership:  Assign clear project 
the rollout of HealthCare.gov.   
leadership to provide cohesion 
 
across tasks and a comprehensive 
In a prior report, OIG examined CMSs management of  
view of progress.  
HealthCare.gov from the time of its well-publicized initial   Alignment:  Align project and 
failures to its improvements during the second enrollment  organizational strategies with the 
period. 30  OIG synthesized several lessons learned from  resources and expertise available.  
CMSs management of that program, as outlined in the   Culture:  Identify and address 
sidebar.    factors of organizational culture 
  that may affect project success.  
 Simplification:  Seek to simplify 
CMS staff reported that they drew on experiences from 
processes, particularly for projects 
HealthCare.gov to rethink the agencys approach to 
with a high risk of failure.  
launching complex initiatives such as the QPP.  Like 
 Integration:  Integrate policy and 
HealthCare.gov, the QPP requires coordination on policy, 
technological work to promote 
operational, and technological issues, as well as extensive 
operational awareness.  
collaboration across different components within CMS.  In 
 Communication:  Promote 
this report, we have noted points at which CMS staff  acceptance of bad news and 
reported applying the lessons learned from HealthCare.gov  encourage staff to identify and 
to CMSs management of the QPP.  communicate problems.  
   Execution:  Design clear strategies 
HealthCare.gov was a really low moment for  for disciplined execution, and 
continually measure progress.  
the agency, but it was a learning moment, 
 Oversight:  Ensure effectiveness 
which allowed us to learn the lessons of how 
of IT contracts by promoting 
to build new muscles [from the turnaround of] 
innovation, integration, and 
HealthCare.gov and apply them to the MACRA  rigorous oversight.  
program.   CMS official    Planning:  Develop contingency 
plans that are quickly actionable, 
such as redundant and scalable 
METHODOLOGY 
systems.  
To describe progress that CMS has made towards   Learning:  Promote continuous 
implementing the QPP, we interviewed CMS staff and  learning to allow for flexibility and 
reviewed relevant internal CMS documents as well as  changing course quickly when 
publicly available information.    needed.  
 
Scope.  This review describes CMSs activities to implement 
the QPP.  We did not review other HHS agencies involved in 
QPP operations, such as the Office of the National Coordinator for Health Information Technology.  Our 
review primarily describes CMSs activities through August 2016, with additional information that 
relates to regulations issued in October 2016. 
 
Interviews.  We interviewed CMS staff between June 2016 and November 2016.  We asked about roles 
and responsibilities; resources; communication; and challenges.  We also asked about CMSs approach 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  5  to developing the QPP, including its overall strategy, priorities, progress made to date, and the timeline 
for remaining activities.  Additionally, we conducted a focus group of CMS staff working on the QPP to 
better understand how the agencys overarching management priorities are carried out in practice.  
Finally, we received responses from the United States Digital Service (USDS)a component of the 
Office of Management and Budget that provides IT consulting to Federal agenciesregarding its 
partnership with CMS on the QPP.    
 
Documents.  We requested and received from CMS a variety of documents related to its QPP 
implementation efforts.  These included, but are not limited to, materials on operations and 
management activities; communications and outreach activities; IT development activities; and QPP 
organizational structures.  We also reviewed publicly available information, such as congressional 
hearing transcripts, CMS public communications, and public comments that CMS received on the QPP 
proposed rule. 
 
Analysis.  We conducted a qualitative analysis of interview and focus group responses, USDS responses, 
CMS documents, and publicly available information to identify key milestones (both those achieved and 
those yet to come), priorities, and challenges related to QPP implementation.  To identify 
vulnerabilities, we reviewed priorities and milestones yet to be achieved and identified those that 
(a) will require extensive, sustained CMS activity in 2017 to address, and (b) pose a significant risk to 
the QPPs success if insufficiently addressed.   
 
Limitations.   Our review focused on CMSs management of the QPPs early implementation.  At this 
point in the programs development, we did not assess the extent to which CMSs management of the 
QPP, or the QPP itself, will be successful in meeting program requirements and goals.  Our review relied 
on self-reported information from a purposively selected sample of CMS staff involved in QPP 
implementation.  We did not interview all CMS staff involved in the QPP, nor did we interview any 
contractors or external entities (e.g., clinicians participating in user testing).  We reviewed a selective 
set of CMS documents that we requested based on their relevance to our study objectives.   
 
Standards. This study was conducted in accordance with the Quality Standards for Inspection and 
Evaluation issued by the Council of the Inspectors General on Integrity and Efficiency. 
 
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  6  Key Management Priorities 
From interviews with CMS leadership and staff and analysis of key documents, we identified CMSs five 
key management priorities regarding the agencys planning and early implementation of the QPP.  Early 
on, CMS staff decided that clinicians acceptance of the QPP, and readiness to participate in it, would 
be the most critical factor to ensuring the programs success.  This focus on clinicians informed CMSs 
decisionmaking regarding its other management priorities, including: 
  adopting integrated internal business practices to accommodate a flexible, user-centric 
approach;  
  developing IT systems that support and streamline clinician participation;  
  developing flexible and transparent MIPS policies; and  
  facilitating participation in Advanced APMs.   
 
In the following sections, we assess CMSs progress to date for each of its management priorities and 
describe what remains to be accomplished.  Additionally, where appropriate, we identify key 
vulnerabilities that will require intensive activity from CMS during 2017 and could affect the programs 
success.   
 
Figure 2.  CMSs Key Management Priorities for Implementing the QPP 
 
 
Source:  OIG analysis of CMSs implementation of the QPP, 2016. 
 
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  7  CMS PRIORITY:  FOSTER CLINICIAN ACCEPTANCE AND READINESS TO PARTICIPATE 
STATUS:  CMS initiated engagement and outreach activities to foster clinicians acceptance of the 
QPP and readiness to participate in it.  Moving forward, CMS plans to continue these activities 
and expand technical assistance.  However, CMS faces challenges in ensuring that providers have 
the information and tools to participate in the QPP.   
WHATS BEEN DONE  WHATS STILL TO COME 
 Engaged clinicians and stakeholders   Continue clinician engagement and 
 Conducted user testing of the QPP Portal  outreach activities  
 Established Clinician Champions and other   Oversee technical assistance contractors 
partnerships   
 Created a transition year   
 Awarded contracts for direct education, 
support, and technical assistance  
 
VULNERABILITY: 
 CMS must continue to expand its outreach and technical assistance efforts to ensure that 
providersespecially solo, small-practice, and rural providershave the information and tools 
they need to participate in the QPP. 
  
OVERVIEW 
According to CMS staff, the QPPs success relies in large part on clinicians acceptance of the program 
and readiness to participate.  However, stakeholders have expressed concerns about whether 
providersespecially solo, small-practice, and rural providers, who historically have been less likely to 
take part in CMS quality initiativeswill be technically and logistically ready to participate in the QPP.  
To address these concerns, CMS engaged with clinicians of all practice types throughout the early 
implementation process, instituted policies designed 
The biggest goal and the biggest risk is the  to ease the transition to the QPP, and awarded 
acceptance by the physician community.   contracts to provide direct support and technical 
 CMS Acting Administrator  assistance to clinicians participating in the QPP 
   (although some of these activities have yet to begin).  
 
WHATS BEEN DONE 
Engaged clinicians and stakeholders.  Through focus 
groups and listening sessions, CMS found that  (The CMS Acting Administrator) made it 
clinicians lack of trust in CMSs management  clear that we have to put the pens down 
capabilities and concerns about the QPP posed a  and listen to the users. . . . We went across 
challenge to QPP implementation and success.   the country to hear providers perceptions 
Specifically, clinicians reported negative experiences  and their reality. . . . We recorded their 
with previous CMS quality programs; confusion over 
pain points.  CMS official 
QPP requirements; and concerns about the 
programs complexity and potential technological 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  8  obstacles.  CMS staff also reported that some clinicians expressed skepticism about the QPPs stated 
purpose of improving care delivery and instead viewed the program as a vehicle to reduce clinician 
reimbursement.  In response, CMS provided a variety of opportunities for clinicians to obtain 
information and submit feedback as the agency developed the QPP policies and systems.  Specifically, 
CMS held webinars, listening sessions, focus groups, and solicited written comments throughout the 
rulemaking process.  (See Figure 3.)  As implementation moves forward, CMS is continuing to publicize 
the program and educate clinicians, using terminology that it has tested with clinician groups to align 
key messages with providers concerns.   
 
Figure 3. QPP Engagement, Outreach, and Education Activities 
 
Engagement, Outreach, and Education Activities  Number of activities completed 
between 1/1/2016  and 
10/28/2016 
User Research   87 
including focus groups and in-depth interviews 
Travel and Public Events   311 
including speeches, conferences, and listening 
sessions/town halls 
Training   27 
including national and regional trainings 
Channel and Partner Engagement   109 
including monthly partner calls; emails and mailings; 
open door forums; and biweekly Clinician Champions 
meetings 
Source: CMS, CMS Official Response to OIGs Questionnaire, 2016. 
 
Conducted user testing of the QPP Portal.  CMS staff reported that as part of the agencys focus on 
clinicians, it adopted a user-centric approach to developing the QPP Portal that serves as clinicians 
online gateway to the QPP.  When fully operational, the Portal will provide everything from general 
information about the QPP to provider-specific feedback; it will also be the mechanism through which 
clinicians report MIPS data.  In developing the Portal, CMS conducted extensive user testing to identify 
and address potential problems.  For example, CMS showed early versions of the website to volunteer 
clinicians and gathered their feedback on the intuitiveness of the site, the clarity of the language, and 
the resonance of the sites content.  As part of this user testing, CMS also observed as clinicians 
attempted to navigate through the Portal, with 
the goal of identifying aspects of the site that may  We need to prove to users that this system 
confuse or frustrate users.  CMS staff reported  is for them and we are working for them.  
incorporating this user feedback in subsequent  Once we prove this, then we will get the 
iterations of the website.  CMS is continuing to 
buy-in.  CMS official 
conduct user testing as it further develops the 
Portal. 
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  9  Established Clinician Champions and leveraged other partnerships.  CMS staff reported leveraging new 
and existing partners to engage clinicians and provide them with information.  Existing partners 
included frontline clinicians; medical societies and associations; qualified clinical data registries; and 
EHR vendors.  Partners participated in national stakeholder calls, shared social media messages with 
their networks, led Learning and Action Network trainings, and provided feedback to CMS from these 
communications.  Furthermore, CMS developed a new partnership named Clinician Champions to 
assist with QPP outreach.  Clinician Champions are providers who are nominated by CMS and are 
trained to deliver QPP messages at the local level and provide peer-based education.  This effort 
responds directly to feedback that CMS received in focus groups indicating that clinicians preferred to 
receive information from people like me.  CMS staff reported that the Clinician Champions have been 
meeting biweekly since July 2016.   
 
Created a transition year.  To give providers additional time to adjust to the QPP, CMS decided that 
2017 would serve as a transition year for the program.  In interviews, CMS staff described this approach 
as an on-ramp that would enable clinicians to gradually adapt to the QPPparticularly, to the MIPS 
reporting requirementswith little risk of penalty.  Specifically, clinicians who need time to prepare 
their systems and learn about the program need report only a minimum amount of MIPS data in order 
to avoid a negative payment adjustment.  Providers who are farther along in their preparations can opt 
to provide more MIPS data and qualify for a positive payment adjustment.  The 2017 transition year 
will also enable clinicians who are interested in the Advanced APM track to determine which model 
would best suit their practice and prepare for participation.  For the 2018 performance period, CMS 
envisions continuing the ramp-up of the program and its performance thresholds; rulemaking to 
finalize these policies for 2018 will occur in 2017.  
 
Awarded contracts for direct education, support, and technical assistance.  CMS developed a variety of 
technical assistance programs designed to meet the needs of different types of clinicians and practices.  
Technical assistance contractors will provide individualized education, support, and assistance that is 
appropriate to each provider type.  For example, Congress allocated $100 million for technical 
assistance to clinicians working in small practices with less than 15 professionals, with priority given to 
those practices located in areas that are rural, have shortages of health professionals, or are medically 
underserved.  These funds will be disbursed to approved entities31 through the QPP-SURS (Small, 
Underserved and Rural Support) contracts.  CMS plans to award the QPP-SURS contracts in early 2017, 
and awardees are expected to begin working with clinicians upon award.  CMS also established similar 
initiatives to provide assistance for all types of practices, such as the Quality Innovation Network-
Quality Improvement Organization contract program and the Transforming Clinical Practice Initiative 
contract program.  These contracts have been awarded and outreach and assistance activities are 
underway.   
 
WHATS STILL TO COME 
Continue clinician engagement and outreach activities.  As the first performance year begins, CMS will 
face both an increased demand for information as well as new feedback from clinicians who may not 
have engaged with the QPP earlier in the implementation process.  Additionally, the first year of the 
QPP is a transition year that does not require full reporting.  Although this is likely to initially 
encourage clinician participation, CMS may face resistance in future years as the program requirements 
become more stringent.  To achieve its goals, CMS will need to sustain its clinician-engagement efforts. 
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  10  Oversee technical assistance contractors.  To ensure that clinicians receive timely assistance, CMS must 
monitor and oversee the performance and effectiveness of technical assistance contractors as direct 
support activities begin.   
 
VULNERABILITY: CMS must ensure that providersespecially solo, small-practice, and 
rural providershave the information and tools they need to participate in the QPP 
If providers lack the knowledge, tools, or skills to participate, they will struggle to 
meet the QPP reporting requirements.  Frustrated providers may even opt not to participate in the QPP 
despite the payment penalty, limiting the programs ability to meet its goals.  To mitigate this risk, CMS 
must continue to monitor clinician readinessespecially as the first reporting deadline approachesto 
identify and address any problems early on.  CMS has begun its technical assistance and training 
efforts, but these activities must quickly be ramped up to full scale and continued throughout 2017 to 
support Medicare clinicians participation in the QPP.     
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  11  CMS PRIORITY:  ADOPT INTEGRATED, FLEXIBLE BUSINESS PRACTICES 
STATUS:  CMS established an integrated, flexible management approach to launch the QPP.   
WHATS BEEN DONE  WHATS STILL TO COME 
 Developed an overall QPP strategy   Award QPP sprint contracts in a timely 
 Assigned executive leadership to each  manner   
program component   Expand oversight of contractors 
 Established integrated project teams and   Hire qualified staff to sustain CMSs shift to 
shared office space  agile practices 
 Partnered with United States Digital Service to 
adopt agile IT development methods 
 Adopted a new contracting approach 
 Awarded a systems integrator contract 
 
 
OVERVIEW 
Implementing the QPP requires CMS to coordinate policy, technology, communications, and operations 
activities.  Additionally, because the legacy programs on which the QPP is based are dispersed among 
various CMS components, staff with necessary expertise and experience are similarly dispersed.  Staff 
working with many of the APMs, for example, report to the CMS Center for Medicare & Medicaid 
Innovation, while those involved in the Value-Based Payment Modifier program are located in the CMS 
Center for Medicare.  To address this logistical and organizational complexity, CMS adopted an 
integrated, flexible approach to both program management and IT development.  CMS staff reported 
that this new approach was informed by lessons it learned from HealthCare.gov, which was initially 
hampered by fragmentation, poor communication, and an inability to change course when problems 
arose.  Based on that experience, CMS strove to implement the QPP with clear project leadership; to 
develop structures and processes that promote communication and integration of policy and 
technological work; and to promote a user-focused, change-oriented organizational culture. 
 
WHATS BEEN DONE 
Developed an overall QPP strategy.  One of CMSs early 
From the beginning, we had a strategic 
steps in planning the QPP was to create an overarching 
plan with really clear goals that were 
strategy, which includes the following objectives:  
informed by the user, clinician, and 
(1) improve beneficiary outcomes and engage 
patient community.  CMS official  
patients through patient-centered Advanced 
APM and MIPS policies;  
(2) enhance clinician experience through flexible and 
transparent program design and interactions with easy-to-use program tools;  
(3) increase the availability and adoption of robust Advanced APMs;  
(4) promote program understanding and maximize participation through customized 
communication, education, outreach, and support that meet the needs of the diversity of 
physician practices and patients, especially the unique needs of small practices;  
(5) improve data and information sharing to provide accurate, timely, and actionable feedback to 
clinicians and other stakeholders; and  
(6) ensure operational excellence in program implementation and ongoing development.32 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  12   
CMS staff described this strategy as fundamental in guiding its daily management of the QPP.  They 
reported that because the objectives clearly define the long-term goal, they have been able to be 
creative and flexible as they develop the QPP. 
 
Assigned executive leadership to each program component.  
[I]t . . . is much clearer to me where I 
In February 2016, CMS established a QPP executive 
need to go to fix a problem.  I can get a 
leadership team to guide staff and to coordinate 
much more rapid audience with higher 
decisionmaking.  The Director of the Center for Clinical 
leadership if problems arise and I can 
Standards and Quality was designated as the QPP Chief 
cross-talk with my peers . . . across 
Executive Officer, with responsibility for leading the 
workstreams.  CMS official  
executive team and overseeing all QPP programmatic 
functions across CMS.  QPP executives were drawn from 
different CMS components to lead key aspects of implementation (e.g., Technology, Policy, 
Operations).  Within each of these areas, CMS created workstream teams to create policy, build IT 
systems, plan operations, manage external communications, and address other major tasks.  (See 
Appendix C for the organizational chart of the QPP Executive Leadership Team.)  In this way, CMS 
sought to ensure that responsibility for each major element of QPP implementation would be clearly 
assigned and that the project would be overseen by a single executive with primary responsibility for 
coordinating the program as a whole.  (See Appendix D for further detail about the roles and 
responsibilities assigned to each member of the QPP Executive Leadership Team.)   
 
CMS staff reported that the QPP executive team meets multiple times each week and also meets 
regularly with CMS senior leaders.  Meeting materials that we reviewed described progress on QPP 
activities; plans for upcoming activities; program risks and corresponding mitigation strategies; and 
issues requiring a decision from senior leaders.  According to CMS staff, the QPP leadership structure 
and regular meetings have provided clear communication channels across CMS divisions to raise issues 
to the executive leadership team for input and resolution.   
 
Established integrated project teams and shared office space.  CMS created a shared office space for 
staff and contractors working on the QPP and staffed each workstream with personnel from various 
CMS divisions with relevant expertise.  Contractors are also embedded in the CMS teams.  Several staff 
described this as a new way of working within CMS and noted that the integrated project teams have 
increased transparency and communication.  Additionally, CMS staff reported that being physically 
co-located with contractors has facilitated closer oversight of their work.   
 
According to CMS staff, teams coordinate closely to ensure that interrelated decisions are informed by 
all relevant workstreams.  For example, one CMS official reported that when CMS developed the final 
rule, it drafted some policies more generally to provide the IT team with greater flexibility for its 
operations.  As another example, the communications team shared with the IT team feedback from 
office managers of group practices regarding how they preferred to use the Portal.  In response to this 
feedback, the IT team developed a feature that enables users to download a list of quality measures for 
later discussion with colleagues. 
 
Partnered with the United States Digital Service to adopt agile IT development methods.  To develop the 
complex IT infrastructure necessary to support the QPP, CMS partnered with USDS.33  USDS advised 
CMS to adopt an agile IT development methodologyan iterative approach in which the final 
product is created through a series of 1-week or 2-week sprints of work to complete smaller 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  13  components.34  This allows the IT team to regularly adjust priorities; conduct user testing and security 
testing; and respond to user feedback.   
 
CMS staff reported that using agile development has 
[The agile development approach is 
enabled closer monitoring of the work of both CMS staff 
to] pilot small, move fast, fail fast, and 
and contractors.  Additionally, because each component of 
have a backup plan.  CMS official  
the system is initially tested during the processrather 
than waiting to test until the system is fully builtthe team 
can detect and mitigate issues earlier.   
 
Adopted a new contracting approach.  USDS staff also advised CMS staff to develop a Blanket 
Purchasing Agreement (BPA) for agile IT development because this method is better aligned with the 
iterative nature of agile IT development.35  According to CMS procurement staff, using a BPA strategy 
for IT development is a new contracting strategy for the agency.  CMS staff reported that the BPA will 
support the agile IT-development approach for the QPP by permitting the agency to procure in smaller 
increments based on each sprint, allowing more rapid procurement based on the speed of planning and 
executing each sprint, and establishing a fixed pricing structure for each sprint.  Staff also noted that 
the agency can walk away from a contactor that does not meet our needs after a shorter period of 
time (e.g., at the end of a 6-month sprint contract).  Finally, budget staff noted that this procurement 
approach has led to more efficient and precise budget estimates.  In September 2016, CMS awarded 
the BPA to six IT firms.    
 
Awarded a systems integrator contract.  Following the initial, failed launch of HealthCare.gov, OIG 
reviewed CMSs IT procurement and recommended that CMS assess whether to assign a lead systems 
integrator for other complex IT projects.36  In our interviews, CMS staff reported that the agency had 
decided to award a systems integration contract for the QPP.  This contractor is responsible for 
analysis of system development schedules and business and systems requirements; management and 
support of system changes; support for architecture design and development, capacity planning, 
integrated end-to-end and performance testing; and support for business operations and 
maintenance.37  CMS awarded this contract on September 22, 2016, but work was delayed until a bid 
protest was withdrawn on November 9, 2016.  CMS staff reported that the systems integration 
contractor began work shortly after the resolution of the bid protest.  Ongoing systems integration will 
be critical as the QPP expands into a larger operation with new staff and contractors.   
 
WHATS STILL TO COME 
 
Award QPP sprint contracts in a timely manner.  Having recently finalized the agile IT development BPA, 
CMS must now begin procuring specific contracts for the QPP sprints.  Timely procurement is critical to 
ensuring that the QPP sprint schedule remains on course.    
 
Expand oversight of contractors.  With the finalization of the agile IT development BPA and the system 
integration contract, CMS will expand the pool of active contractors working on the QPP development 
and operation.  With a larger pool of contractors working on the QPP, CMS will need to expand its 
oversight and monitoring of their activities.   
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  14  Hire qualified staff to sustain CMSs shift to agile practices.  
Implementing a major technical and 
CMS staff described USDS as a major driver of its ability to 
operational program such as QPP 
adopt agile practices.  However, USDSs role in the QPP is 
requires interdisciplinary management, 
time-limited.  USDS staff stated that they are training CMS 
technical, and product management 
staff and contractors in preparation for USDSs eventual 
expertise.  Hiring for those skills in 
departure, and CMS staff noted that CMSs direct hiring 
authority for the QPP has been helpful in bringing qualified  government is difficult . . . .  
staff onboard.  Nonetheless, both CMS and USDS staff   USDS correspondence 
reported that hiring staff with expertise in agile 
development poses a challenge.  As new staff join the QPP, CMS will need to continue training in agile 
development and QPP internal processes to sustain its new business practices. 
 
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  15  CMS PRIORITY:  DEVELOP IT SYSTEMS 
STATUS:  In 2016, CMS prioritized developing the public-facing QPP website.  Upcoming 
milestones include enabling individualized QPP accounts for clinicians and expanding a 
service desk.  However, CMS faces challenges in building the complex backend IT systems 
required to receive clinicians data, calculate their MIPS scores, and carry out other 
functions vital to the programs success.   
WHATS BEEN DONE  WHATS STILL TO COME 
 Developed informational website (launched on   Enable individualized accounts for clinicians 
October 14, 2016)   Expand the service desk 
 Build backend IT systems  
 
VULNERABILITY: 
 CMS must build and test the backend IT systems necessary for data submission and validation, 
calculation of MIPS Final Scores, payment adjustment, and other key functions. 
 
OVERVIEW 
The QPP Portal consists of three major components (see Figure 4 on the next page):  
  a public-facing informational website,  
  individualized accounts for clinicians, and  
  backend systems necessary to receive and validate clinicians data, provide individualized 
performance feedback, calculate clinicians MIPS scores, and adjust Part B payments 
accordingly.   
The informational website component was launched in October 2016, while the remaining components 
are still under development.  CMS also plans to expand a service desk that will assist clinicians in using 
the Portal. 
 
WHATS BEEN DONE  
We understood early on that the Portal 
Developed and launched the informational website.  The 
was going to make or break the 
QPP informational website launched on October 14, 
2016.38  The website provides general information about  physician experience with the MIPS 
the QPP, the MIPS track, and the Advanced APM track.  It  program.  The way we communicate to 
also clarifies clinicians reporting options and how to  them and how much of a hassle it is for 
qualify for either the MIPS or Advanced APM tracks.  For  them to communicate with us is 
the MIPS track, the website enables users to browse the  important.  CMS official 
various MIPS measures available for selection.   
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  16   
Figure 4.  Overview of the QPP Portal 
 
 
 
Source:  OIG analysis of CMSs implementation of the QPP, 2016. 
 
WHATS STILL TO COME 
Enable individualized accounts.  For many clinicians, registering for and using their individualized QPP 
account will be their first significant interaction with the QPP.  CMS staff reported that individualized 
accounts will be available in January 2017.  These accounts will ultimately enable CMS to:  
  verify the users identity;  
  inform clinicians of their eligibility for the Advanced APM track versus the MIPS track, so that 
clinicians know whether they must select and report MIPS measures; and 
  provide individualized performance feedback.   
CMS staff reported that, later in 2017, the QPP accounts will also be used to gather information on a 
clinicians network of surrogates and vendors (e.g., entities that may report data on the clinicians 
behalf).  
 
CMS staff noted that CMS has, for the first time, contracted with a vendor to provide identity as 
a servicean approach in which a third party provides scalable identity-management services, rather 
than having those functions housed and managed onsite.  CMS staff reported selecting this approach 
because it will provide flexibility in capacity, enabling CMS to pay for more capacity only at times of 
higher demand.  CMS staff anticipated that such flexibility will prevent user access problems during 
peak use timesan issue that initially caused significant problems during the launch of 
HealthCare.govand will also prove cost-efficient, because CMS will pay only for the level of capacity 
actually needed each month.      
 
CMS staff reported that CMS plans to pilot-test the security and reliability of this new service prior to 
fullscale deployment.  Such testing is critical; a poorly functioning identity management service could 
impede clinicians ability to successfully submit data, receive performance feedback reports, and 
conduct other QPP activities using the Portal.  To mitigate this risk, CMS plans to encourage clinicians to 
voluntarily register for a QPP account several months prior to the first deadline for data submission.  
This soft rollout of individual accounts may allow CMS to identify and address problems that arise 
with the new identity management service prior to the peak registration period.   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  17   
Expand the service desk.  CMS has initiated a QPP service desk to answer questions and resolve 
problems.  CMS staff reported that in the future, the service desk will not only provide IT support for 
the Portal, but will also offer broader program assistance.  This will include sharing information about 
training opportunities and assisting providers who have filed requests to have their MIPS Final Scores 
reviewed.   CMS staff reported that its goal is to eventually create a single integrated system across the 
currently separate service desks of the legacy systems (i.e., PQRS, VBM, and the Medicare physician 
EHR incentive program) and the new QPP portal.  Achieving this milestone will require CMS to 
(1) modify service-desk contracts as they come up for renewal, and (2) develop new IT tools to share 
information across separate helpdesk functions (e.g., an integrated ticketing system for all incoming 
requests).   
 
Build backend IT systems.  With limited time between the passage of MACRA and the beginning of the 
first performance year, CMS focused its IT development on ensuring that the public-facing QPP website 
would be up and running when the final rule was issued.  However, the agency must now build the 
backend systems that will enable CMS to:  
  receive and validate clinician performance and attestation data; 
  provide useful performance feedback to clinicians;  
  calculate MIPS Final Scores for MIPS-eligible clinicians; and 
  adjust future Part B payments. 
 
VULNERABILITY: CMS must build and test the backend systems necessary to fully 
support the QPP   
Building and testing the extensive IT systems necessary to support critical QPP 
operations will require significant and sustained effort over the forthcoming year.  In the past, CMS has 
sometimes experienced delays and complications related to major IT initiatives, such as those required 
for the continued operation of Medicare Part D and HealthCare.gov.  If the complex systems underlying 
the QPP are not operational on schedule, the program will struggle to meet its goal of improving value 
and quality.   
 
CMS has sought to mitigate these risks by (1) planning to use legacy program systems (i.e., PQRS, VBM, 
and EHR) as a backup option for MIPS data submission, and (2) using 2017 as a transition year in 
which MIPS scores are calculated butas long as the clinician submits a minimum of datawill not 
result in a negative payment adjustment.  Nonetheless, the QPP is more likely to succeed if all IT 
infrastructure is ready when needed. 
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  18  CMS PRIORITY:  DEVELOP MIPS POLICIES 
STATUS:  CMS issued MIPS-related regulations that finalized key policies for the first 
performance year.  Promised subregulatory guidance is still needed to ensure that providers 
have sufficient information as the first performance year begins. 
WHATS BEEN DONE  WHATS STILL TO COME 
 Published QPP final rule, including MIPS   Issue promised subregulatory guidance 
policies for 2017    Finalize policies for virtual groups 
   Rulemaking in 2018 and beyond 
 
OVERVIEW  
CMS met challenging timeframes to issue final regulations implementing MIPS on October 14, 2016.  
These regulations finalize MIPS policies for the first performance year.  Further rulemaking will be 
necessary in 2018 and beyond.  Additionally, CMS has not yet issued promised subregulatory guidance 
for a variety of topics on which stakeholders have requested clarification. 
 
WHATS BEEN DONE 
Published final rule.  On October 14, 2016, CMS issued a final rule with comment period.  This final rule 
finalized MIPS policies for 2017, the first performance period.  In developing the final rule, CMS staff 
reviewed over 4,000 comments received regarding the proposed rule that was published on May 9, 
2016 and worked to develop MIPS policies that would address the numerous concerns raised by 
stakeholders while remaining in compliance with the 
statute.  Despite the complexity of the rulemaking  Getting the policy right is critical.  This is 
process and a challenging timeframe, CMS was able to  new territory, and we need to get the 
issue the final rule on schedule.  (See Appendix E.)   program on the right glide path.  
 
 CMS Acting Administrator 
Consistent with its strategic objectives of prioritizing 
  
clinician experience and flexible and transparent 
program design, CMS made numerous revisions from the proposed rule based on stakeholders 
comments.  Of these changes, two are particularly significant: 
  Implementing the 2017 performance period as a transition year.  The final rule reenvisioned the 
first performance year as a transition year with minimal or no negative impact on payments.  
Under the final regulations, providers can pick their pace and participate in the QPP to a 
smaller or greater extent in 2017 with little risk of penalty.  This significant change from the 
proposed rule responded to numerous stakeholder comments requesting delayed MIPS 
implementation.   
  Increasing the low-volume threshold.  Many stakeholder comments on the proposed rule 
reflected concerns about the ability of solo and small-practice providersparticularly in rural 
areasto successfully participate in MIPS.  In response, CMS raised the low-volume threshold 
so that fewer clinicians will be required to participate in MIPS.  Under the final regulations, 
clinicians are exempt from MIPS if they have no more than $30,000 in Medicare Part B charges 
or serve no more than 100 beneficiaries during a 12-month period.  CMS estimates that as 
compared to the proposed rule, the final rule exempts between approximately 150,000 and 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  19  200,000 additional clinicians from MIPS.  CMS now estimates that in total, between 43 and 
47 percent of all clinicians billing to Medicare Part Bbetween 592,000 and 642,000 
clinicianswill be required to submit MIPS data in the first year.  Of these, approximately 
148,000 are solo or small-practice providers.39  The clinicians required to participate in the first 
year represent between approximately 73 and 78 percent of total Medicare payments for 
Part B professional services. 
 
WHATS STILL TO COME 
Issue promised subregulatory guidance.  MIPS reporting and scoring is complex and requires clinicians 
to adapt to significant changes in the Medicare Part B payment system.  Clinicians have requested 
additional guidance and tools for compliance, and in the QPP final rule, CMS stated its intention to 
provide subregulatory guidance on a variety of topics, including: 
  group reporting options; 
  the form and manner of data submissions by clinicians, including standards that third-party 
vendors will need to follow; 
  information and instructions regarding CMS data validation and auditing; and 
  documentation expectations for the transition year. 
 
However, with only weeks to go until the beginning of the 2017 performance year, CMS has yet to issue 
most of its planned subregulatory guidance.  The earlier clinicians have this information, the better 
prepared they will be to fully and successfully participate in MIPS in 2017. 
 
Finalize policies for virtual groups.  Section 1848(q)(5)(I) of MACRA established voluntary virtual 
groups that would allow solo and small practitioners to report as a group with at least one other solo 
or small practitioner for a performance period.  Clinicians who wish to report as a virtual group in this 
manner must elect to do so before the start of the relevant performance period.  Stakeholders have 
expressed support for this option, but CMS was not able to implement it for the 2017 performance 
period because of significant barriers related to both technology and operations.  CMS has stated 
that it intends to implement virtual groups for the 2018 performance period.  To achieve this 
milestone, CMS must finalize the relevant policies promptly to ensure sufficient time for necessary IT 
infrastructure development, as well as for clinicians to plan for their participation.   
 
Rulemaking in 2018 and beyond.  The final rule issued in October 2016 sets MIPS policies for the first 
performance year, but additional rulemaking will be necessary for the 2018 performance yearfor 
example, to finalize policies around virtual groups, to identify new quality measures, and to set the 
2018 performance threshold.  In addition, CMS will use future rulemaking to address comments 
received and concerns expressed in response to on the October 2016 final rule.  As in 2017, CMS will 
face a challenging timeframe to finalize and issue the necessary regulations in sufficient time for 
clinicians, vendors, and other entities to be ready for the 2018 performance year. 
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  20  CMS PRIORITY:  FACILITATE PARTICIPATION IN ADVANCED APMS 
STATUS:  CMS identified Medicare models that qualify as Advanced APMs for the first 
performance period.  Participation in Advanced APMs is expected to be limited in 2017; CMS 
has begun identifying ways to increase opportunities for clinicians to participate in this 
program track in the future. 
WHATS BEEN DONE  WHATS STILL TO COME 
 Identified which existing Medicare models   Determine which clinicians are Qualifying 
meet criteria for Advanced APMs  APM Participants 
 Established policy for determining Qualifying   Increase Advanced APM opportunities 
APM Participants   Increase clinician participation in Advanced 
 Published QPP final rule, including Advanced  APMs over time 
APM policies for 2017    
 Awarded contracts for technical assistance to 
prepare clinicians to participate in Advanced 
APMs  
 
OVERVIEW 
CMS issued regulations finalizing Advanced APM policies for the first performance year, including the 
initial list of Medicare models that qualify as Advanced APMs and the criteria CMS will use to identify 
additional models.  (See Appendix E for key milestones in the rulemaking process.)  In 2017, CMS will 
need to inform clinicians of whether they are Qualifying APM Participants or whether they will need to 
report MIPS measures.  In the future, CMS will need to take steps to increase clinician participation in 
Advanced APMs in order to meet HHS goals.  
 
WHATS BEEN DONE 
Identified which existing Medicare models meet criteria for Advanced APMs.  CMS reviewed its existing 
Medicare models with regard to the three criteria defined by MACRA40 and identified models, 
demonstrations, or programs that currently qualify as Advanced APMs for the first performance 
period.41  See Figure 5 on the next page.   
 
Established policy for determining Qualifying APM Participants.  The final rule established that for the 
first performance period, clinicians are designated Qualifying APM Participants if they serve 20 percent 
of their Medicare patients or receive 25 percent of their Part B payments through an Advanced APM. 42  
In 2016, CMS procured a contractor for APM program analysis whose main responsibility will be to 
identify clinicians participating in the Advanced APMs and determine which are meeting the threshold 
to be Qualifying APM Participants during the first performance period. 
 
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  21   
Figure 5:  Overview of Advanced APMs Available in the first QPP Performance Period 
  
Source:  OIG analysis of CMSs implementation of the QPP, 2016. 
*In Track 1 of the Shared Savings Program, the ACOs share in program savings.  Without requiring ACOs to also assume risk for losses, Track 1 
does not meet the criteria to be an Advanced APM in the QPP.  Because ACOs in Tracks 2 and 3 of the Shared Savings Program are required to 
assume risks for both savings and losses, Tracks 2 and 3 qualify as Advanced APMs under the QPP. 
 
Awarded contracts for technical assistance to facilitate clinician participation in Advanced APMs.  In its 
final rule, CMS estimated that between 5 and 8 percent of all clinicians billing under Medicare Part B
between 70,000 and 120,000 clinicianswill be Qualifying APM Participants in the Advanced APM track 
during the first performance period.  CMSs goal is to increase participation in subsequent program 
years by expanding the Advanced APM opportunities available to meet the needs of different types of 
clinicians.  However, this goal will be met only if clinicians are interested in and prepared to participate 
in Advanced APMs.   
 
To this end, CMS is investing in technical assistance and training opportunities to help clinicians prepare 
for Advanced APMs.  In June 2016, CMS announced a new funding opportunity for Support and 
Alignment Networks 2.0, which is part of its larger Transforming Clinical Practice Initiative.43  The goal 
of these networks is to increase the adoption of APMs on a large scale by providing tailored technical 
assistance, training, and other learning resources to primary care and specialty providers on how to 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  22  enhance the quality, efficiency, and coordination of the care they provide.  CMS announced that it 
intended to make up to $10 million available over the next 3 years to fund these networks.  In 
September 2016, CMS announced that it had awarded two cooperative agreementsone to the 
American Psychological Association and the other to the Virginia Cardiac Services Quality Initiative. 
 
WHATS STILL TO COME 
Determining which clinicians are Qualifying APM Participants.  In response to the proposed rule, some 
commenters expressed concern that if clinicians do not know until the end of the performance period 
whether they meet the threshold to be Qualifying APM Participants, they may also have to prepare for 
MIPS reporting.  Preparing to participate in both QPP tracksAdvanced APMs and MIPSwould be 
burdensome and could be a disincentive to participate in the QPP. 
 
CMS is planning to provide clinicians with information throughout the year about their likelihood of 
meeting the Qualifying APM Participant threshold.  Specifically, during the first performance period, 
the contractor for APM program analysis will track which clinicians are providing Part B services 
through Advanced APMs and periodically estimate whether they are likely to meet the threshold to be 
Qualifying APM Participants.  CMS anticipated that it will make three Qualifying APM Participant 
determinationsQuarter 1, Quarter 2, and Quarter 3using data available through March 31, through 
June 30, and through August 31, respectively.  CMS plans to share its early estimates of Qualifying APM 
Participant determinations with clinicians through their individualized QPP Portal accounts. 
  
Increase Advanced APM opportunities.  As noted above, there 
For each model, we look at the 
are a limited number of Advanced APM opportunities 
savings opportunities and the 
available during the first QPP performance year.  Increasing 
magnitude of how many 
the availability of Advanced APMs for clinicians is a significant 
undertaking, and CMS is planning a multiyear expansion  beneficiaries could be covered as 
effort.  CMS staff described a range of options, including  we assess the likelihood of 
(1) modifying some current Medicare models to meet the  improvements to both cost and 
criteria for Advanced APMs, (2) opening additional rounds of  quality.  
the existing Advanced APMs to new participants, and   CMS official 
(3) developing new models.  In considering which options to 
  
pursue, CMS staff reported that they plan to prioritize models 
with the highest yieldthose that provide the greatest number of opportunities for clinicians to 
participate.  As CMS develops future Advanced APMs, it is also considering models that can meet the 
needs of specialists.  As it seeks to expand Advanced APM options, CMS must also ensure that it meets 
statutory requirements to test only models that have the potential either to generate savings while 
maintaining quality, or to improve quality while not increasing spending.44 
 
In addition to the Advanced APM models developed by CMS, MACRA established an independent 
Physician-Focused Payment Model Technical Advisory Committee (PTAC) so that external stakeholders 
may propose new models for consideration.45  In 2016, the PTAC developed criteria for reviewing 
proposed models and making recommendations to the Secretary of Health and Human Services as to 
which models should be adopted.  It anticipated that the submission of proposals would begin in 
December 2016.  CMS staff reported ongoing communications with the PTAC, including providing it 
with requested resources on CMSs approach to planning Advanced APMs and developing savings 
estimates.   
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  23  Increase clinician participation in Advanced APMs over time.  In addition to increasing the number and 
variety of Advanced APM opportunities, CMS seeks to increase clinician adoption of these new care 
delivery models.  Along with the steps that CMS is already planningsuch as prioritizing high-yield 
models and funding technical assistance designed to facilitate clinician participationCMS must 
monitor the interplay of MIPS and Advanced APM financial rewards to identify any unintended 
consequences that would affect clinician participation.  Specifically, some policy analysts have 
speculated that there could be disincentives to participate in the Advanced APM track because in the 
early years of the program, high performers might gain more financially under the MIPS track.46  As 
CMS gains more experience with clinician participation in the two QPP tracks and how payment 
adjustments are distributed, it will need to identify and address any competing incentives that would 
impede the programs ability to meet its goals.       
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  24  Conclusion 
Since MACRAs enactment in April 2015, CMS has made significant progress towards implementing the 
QPP.  During 2016, CMS focused on laying the groundwork for the QPP by fostering clinician acceptance 
and readiness to participate; adopting integrated business practices; building IT systems; developing 
MIPS policies; and facilitating participation in Advanced APMs.  As the beginning of the first 
performance year approaches, CMS has finalized key policies, launched an informational website, and 
awarded a variety of contracts for technical assistance and training.  Overall, CMS has worked to 
address many of the key lessons learned from the initial launch of HealthCare.gov, such as the need for 
clear leadership; integration of policy and technological work; clear and frequent communication; and 
flexibility to adapt to new information.   
 
Although many milestones remain to be completed, two aspects of implementation merit particular 
attention in 2017, because they will be crucial in determining the QPPs success:  (1) providing sufficient 
guidance and technical assistance to ensure that clinicians are ready to participate in the QPP, and 
(2) developing backend IT systems.   
 
First, continued engagement with clinicians is essential to CMSs efforts to ensure that providers have 
the information and tools they need to meet the challenges of the QPP.  CMSs technical assistance and 
training efforts must quickly be ramped up to full-scale, and the extensive subregulatory guidance 
promised in the final rule should be issued, to help Medicare clinicians: 
  understand the QPP;  
  take advantage of the transition year to familiarize themselves with the program;  
  select the QPP participation option that best suits their practice; and  
  use QPP performance feedback to improve their care delivery to Medicare beneficiaries. 
 
Second, it is crucial that the backend IT systems are ready to securely and reliably support the QPP.  
Some key functions must be operational in 2017, with others required in 2018.  These systems are 
necessary so that:  
  clinicians can report QPP data to CMS accurately and with minimized burden;  
  CMS can validate the data it receives and calculate MIPS Final Scores;  
  CMS can provide useful performance feedback information to clinicians; and  
  CMS can accurately adjust Part B payment based on clinicians QPP participation.   
 
CMS has built a promising foundation for managing the initial transition to the QPP.  However, its 
ability to ensure clinicians readiness to participate in the program and develop the necessary IT 
infrastructure will help determine whether the QPP succeeds in its goal to promote quality and value of 
care.  OIG will continue to monitor CMSs progress in developing and operating the QPP and will 
conduct additional reviews as appropriate. 
 
AGENCY COMMENTS AND OIG RESPONSE 
In its comments on OIGs draft report, CMS reiterated its goal of providing patient-centered, 
high-quality care for Medicare beneficiaries.  CMS described its prior and ongoing activities to engage 
clinicians through listening tours and other outreach and noted that a variety of technical assistance 
efforts are underway.  CMS also said that it recognized the particular challenges faced by small and 
rural practices and described its efforts to support such practices.  Regarding the two vulnerabilities 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  25  that OIG identified, CMS stated that as it implements the QPP, it is committed to continuing to engage 
with clinicians and provide them with assistance, and to optimize backend IT systems support.  (See 
Appendix F for the full text of CMSs comments.)   
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  26  Appendix A  
OVERVIEW OF THE MIPS FINAL SCORE FOR PERFORMANCE YEAR 1, 2017 
In the final rule, CMS finalized the following categories of MIPS measures and the proportion that each 
will contribute to the final score.47 
 
 
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  27  Appendix B  
QPP TIMELINE: PERFORMANCE PERIODS, FEEDBACK, SCORING, AND PAYMENT ADJUSTMENT 
The QPP will operate on an overlapping 3-year program cycle.  The figure below shows the first three 
program cycles, which will begin in 2017, 2018, and 2019, respectively.  Subject to future rulemaking, 
program cycle four will begin in 2020, program cycle five in 2021, and so on. 
 
 
        
2017 
 
January  December:   
MIPS performance period 
January  August:  
Advanced APM performance period 
July:   
Clinicians receive feedback reports 
2018  2018 
March 31:    January  December:   
Clinicians send MIPS data to CMS  MIPS performance period 
January  August:  
April  June:   
Advanced APM performance period 
CMS analysis and scoring 
  July:   
Clinicians receive feedback reports 
2019  2019  2019 
January  December:   March 31:    January  December:   
Payment adjustment period (based  Clinicians send MIPS data to CMS  MIPS performance period 
on 2017 data)  April  June:    January  August:  
  CMS analysis and scoring  Advanced APM performance period 
  July:   
Clinicians receive feedback reports 
2020  2020 
January  December:   March 31:   
Payment adjustment period (based  Clinicians send MIPS data to CMS 
on 2018 data) 
April  June:   
CMS analysis and scoring 
 
2021 
January  December:  
Payment adjustment period (based 
on 2019 data) 
 
 
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  28   
Appendix C   
QPP EXECUTIVE LEADERSHIP TEAM  STRUCTURE 
 
Source: CMS, Quality Payment Program Resource Allocation, June 2016. 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  29  Appendix D   
QPP EXECUTIVE LEADERSHIP TEAM  ROLES AND RESPONSIBILITIES 
 
Roles  Responsibilities for the QPP 
Chief Executive  Oversees and directs all programmatic functions, including overarching 
Officer  program strategy and implementation; serves as internal and external authority 
for MACRA. 
Chief Strategy  Develops program strategy and measurement plan.  Facilitates annual planning. 
Officer 
Chief Operating  Manages program planning and operations, including program infrastructure, 
Officer  processes, human capital, budgets, and contracting.  Implements and oversees 
program governance and reporting. 
APM Executive  Leads development and implementation of APMs considering departmental, 
agency, and industry goals and constraints.  Coordinates APM development and 
implementation closely within/across the organization working with 
counterparts to align/integrate implementation efforts and resolve challenges 
and barriers. 
Chief Product  Leads product strategy, design, and modification to achieve desired customer 
Officer  experience.  Coordinates all product-related functions to produce an integrated 
customer experience.  Leads customer analysis, user research, and product 
prioritization. 
Chief Policy  Lead policy development, coordinating with internal and external parties to 
Officers  achieve legislative requirements and the intent of MACRA.  Lead impact 
analysis and facilitates implementation analysis.  Coordinate policy 
development and implementation closely within/across the organization. 
Chief Technology  Develop technology alternatives analysis and target architecture for 
Officers  implementing the policy and achieving the desired customer experience goals 
across infrastructure, systems, and applications.  Determine development 
approach and standards and oversee development and enhancements. 
Chief Marketing  Develops program narrative, branding, messaging, content, and education 
Officer  materials to inform and respond to the needs of the customers and critical 
audiences.  Develops and maintains partnership and channel strategies to 
engage customers and audiences and provide critical program-feedback loops. 
Leads content development, review, and publication. 
Source: CMS, Quality Payment Program Resource Allocation, June 2016. 
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  30  Appendix E   
KEY MILESTONES IN THE QPP RULEMAKING PROCESS 
 
 
   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  31  Appendix F   
COMMENTS FROM THE CENTERS FOR MEDICARE & MEDICAID SERVICES 
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  32   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  33   
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  34  Acknowledgments 
This report was prepared under the direction of David Tawes, Regional Inspector General for Evaluation 
and Inspections in the Baltimore regional office, and Louise Schoggen, Assistant Regional Inspector 
General.  
 
Heather Barton served as the team leader for this study, and Jennifer Hutnich was the lead analyst for 
this study.  Other Office of Evaluation and Inspections staff who conducted this study include 
Evan Godfrey and Lukas Glos.  Office of Evaluation and Inspections staff who provided support include 
Clarence Arnold and Christine Moritz.  Jessica Swanstrom from the Office of Management and Policy 
also provided assistance with report graphics. 
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  35   
Endnotes 
1 Medicare Access and CHIP (Childrens Health Insurance Program) Reauthorization Act of 2015 (MACRA), P.L. No. 
114-10 (April 16, 2015).  
2 CMS posted the final rule on its public website on October 14, 2016, and it was published in the Federal Register 
on November 4, 2016.  See 81 Fed. Reg. 77008 (Nov. 4, 2016). 
3 MACRA, P.L. No. 114-10 (April 16, 2015). 
4 81 Fed. Reg. 77008 (Nov. 4, 2016).  
5 Omnibus Budget Reconciliation Act of 1989, P.L. No. 101-239,  6102 codified at Social Security Act (SSA)  1848. 
The Medicare physician fee schedule is derived using a resource-based relative value scale, which includes three 
resource components:  (1) total physician work, (2) practice expenses, and (3) malpractice expenses.  Each 
component is measured in terms of relative value units (RVUs).  The Medicare physician fee schedule payment 
rates are based on RVUs, adjusted for geography, and multiplied by a national conversion factor to derive dollar 
amounts.   
6 Section 1848(a)(1) of the Social Security Act established the Medicare physician fee schedule as the basis for 
Medicare reimbursement for all physician services beginning in January 1992.   
7 SSA  1848(f).   
8 Annual spending targets were updated each year by applying a growth rate known as the sustainable growth 
rate (SGR). The SGR formula incorporated four factors:  (1) inflation, (2) changes in enrollment in Medicares 
fee-for-service program, (3) the estimated 10-year average annual growth rate of real gross domestic product per 
capita, and (4) the impact of changes in law or regulation.  These factors were multiplied to yield an overall rate of 
growth.  To determine the next years spending target, the previous years target was increased by the overall 
rate of growth estimated for the next year.  
9 Congressional Budget Office (CBO), The Sustainable Growth Rate Formula for Setting Medicares Physician 
Payment Rates, September 2006, p. 5.  Accessed at https://www.cbo.gov/sites/default/files/109th-congress-
2005-2006/reports/09-07-sgr-brief.pdf on October 13, 2016.   
10 CBO, Medicares Payments to Physicians:  The Budgetary Impact of Alternative Policies Relative to CBOs 
March 2012 Baseline, July 2012.  Accessed at https://www.cbo.gov/publication/41512?index=12240 on 
October 14, 2016.   
11 Tax Relief and Health Care Act of 2006, P.L. No. 109-432,  101.   
12 Medicare Improvements for Patients and Providers Act of 2008, P.L. No. 110-275  131. 
13 Patient Protection and Affordable Care Act, P.L. No. 111-148,  3002 (March 23, 2010), as amended by the 
Health Care and Education Reconciliation Act of 2010, P.L. No. 111-152 (March 30, 2010), collectively referred to 
as the Affordable Care Act (ACA).  
14 ACA,  3007. 
15 American Recovery and Reinvestment Act of 2009, P.L. No. 111-5,  4101-4103.   
16 42 CFR  414.90(c) and (e).   
17 42 CFR  414.1275.    
18 80 Fed. Reg. 62762 (Oct. 16, 2015).  
19 MACRA,  101. 
20 Clinicians included in the QPP include physicians, physician assistants, nurse practitioners, clinical nurse 
specialists, and certified registered nurse anesthetists. MACRA,  101(c)(1)(C)).  
21 MACRA,  101(c), SSA  1848(q)(2)(A). 
22 MACRA,  101(c), SSA 1848(q)(6)(F)(i)(I). 
23 MACRA,  101(c), SSA  1848(q)(6)(F)(iv)(I).  
24 CMS will use a low-volume threshold determination period to identify providers that meet the low-volume 
threshold and are thus excluded from MIPS reporting and payment adjustment.  The low-volume determination 
period includes two 12-month segments; a provider who meets the threshold during either of the segments is 
excluded from MIPS.  The initial segment spans from the last 4 months of a calendar year 2 years prior to the 
performance period through the first 8 months of the next calendar year.  The second segment spans from the 
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  36                                                                                                                                                                                           
last 4 months of a calendar year 1 year prior to the performance period through the first 8 months of the 
performance period in the next calendar year.  81 Fed. Reg. 77008, 77065 (Nov. 4, 2016). 
25 ACA,  3021 and 3022.  
26 The Comprehensive End-Stage Renal Dialysis (ESRD) Care Model is designed to identify, test, and evaluate new 
ways to improve care for beneficiaries with ESRD.  The Comprehensive Care for Joint Replacement Model aims to 
improve care and reduce payment for hip and knee replacements through episode-based payments.  The 
Comprehensive Primary Care Plus program is being implemented in 14 regions where CMS, commercial insurance 
plans, State Medicaid agencies, and other selected payer partners will align on payment, data sharing, and quality 
metrics.  More information about each APM is available at 
https://innovation.cms.gov/initiatives/index.html#views=models.  
27 An APM entity is an entity that participates in an APM or payment arrangement with a non-Medicare payer 
through a direct agreement or through Federal or State law or regulation. See 81 Fed. Reg. 77008  414.1305 
(Nov. 4, 2016). 
28 MACRA,  101(e)(2)). 
29 81 Fed. Reg. 28161 (May 9, 2016).  Comments on the proposed rule are available via 
https://www.regulations.gov/docket?D=CMS-2016-0060.  
30 OIG, HealthCare.gov: CMS Management of the Federal Marketplace (OEI-06-14-00350), February 2016.  
Available at https://oig.hhs.gov/oei/reports/oei-06-14-00350.pdf. 
31 Approved entities include quality improvement organizations, regional extension centers, and regional health 
collaboratives.   
32 81 Fed. Reg. 77008 (Nov. 4, 2016). 
33 USDS,  Our Work.  Accessed at https://www.usds.gov/work on November 29, 2016.  
34 CMS. Agile Development for Merit-based Incentive System (MIPS), Solicitation HHSM-500-2016-RFP-0037, 
Statement of Objectives.  Accessed at 
https://www.fbo.gov/index?s=opportunity&mode=form&id=a1784e8e8067c0b6b648074ccb8d8d31&tab=core&_
cview=1 on October 12, 2016. 
35 A BPA is an ongoing agreement for recurring procurements that streamlines ordering procedures, reduces 
procurement lead time, and simplifies invoicing and other contractual processes.  General Services 
Administration, Blanket Purchase Agreements.  Accessed at http://www.gsa.gov/portal/content/199353 on 
October 3, 2016. 
36 OIG, Federal Marketplace: Inadequacies in Contract Planning and Procurement (OEI-03-14-00230), 
January 2015.  Available at https://oig.hhs.gov/oei/reports/oei-03-14-00230.pdf. 
37 CMS,  MACRA 101 and Quality Payment Program:  Office of Management and Budget Briefing.  May 2016.  
38 While it worked with USDS staff to create the Agile Development BPA for future procurement on other aspects 
of the QPP Portal, CMS used its existing IT contracts to develop the public-facing website.  The QPP website is at 
https://qpp.cms.gov/.   
39 We define solo or small practice as a practice with less than ten clinicians. 
40 MACRA established the following criteria for QPP Advanced APMs: (1) require participants to use certified EHR 
technology; (2) provide payment for covered professional services based on quality measures comparable to 
those used in the quality performance category of MIPS; and (3) require APM Entities to bear more than a 
nominal amount of financial risk, or be a Medical Home Model expanded under section 1115A(c) of the Social 
Security Act.  (See MACRA, 101(e)(2)). 
41 CMS will publish a final list of the Advanced APM models for 2017 performance period by January 1, 2017.  The 
list of currently available Advanced APMs included in this report was current as of October 28, 2016.  In the final 
rule, CMS indicated that additional models may become available in 2017.  CMS is exploring how to modify some 
APMs, including the Maryland All-Payer Model and the Comprehensive Care for Joint Replacement Model, to 
meet the criteria for Advanced APMs under the QPP.  More information is available at 
https://qpp.cms.gov/learn/apms. 
42 42 CFR  414.1430(a)(1), (3), (b)(1) and (3); 81 Fed. Reg. 77008 (Nov. 4, 2016).   
43 CMS, CMS-1L1-16-001 Grant Notice: Transforming Clinical Practice Initiative Support and Alignment Networks 
2.0 (June 10, 2016).  Accessed at http://www.grants.gov/view-opportunity.html?oppId=284529 on September 14, 
2016. 
44 ACA,  3021(c). 
 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  37                                                                                                                                                                                           
45 The PTAC is composed of 11 appointees, and the HHS Assistant Secretary for Planning and Evaluation provides 
it with operational support.  MACRA,  101(e)(1), SSA  1868(c).   
46 Patel, K.; Adler, L.; Darling, M.; Ginsburg, P.; Lieberman, S.M.; How the Money Flows Under MACRA.  July 12, 
2016.  Accessed at https://www.brookings.edu/research/how-the-money-flows-under-macra/ on July 20, 2016. 
47 81 Fed. Reg. 77008 (Nov. 4, 2016). 
CMSs Early Implementation of the Quality Payment Program (OEI-12-16-00400)  38