Department of Health and Human Services 
 
 
OFFICE OF INSPECTOR GENERAL   
 
   
 
 
 
 
   
F R :  
OLLOWUP  EVIEW   
CMS M
S  ANAGEMENT OF THE 
Q P P  
UALITY  AYMENT  ROGRAM
Suzanne Murrin 
Deputy Inspector General for  
Evaluation and Inspections  
OEI-12-17-00350 
December 2017  Report in Brief 
December 2017 
Report N o. OEI-12-17-00350 
Followup Review:  CMSs Management of the 
Why OIG Did This Review  
The Medicare Access and CHIP  Quality Payment Program  
Reauthorization Act of 2015 (MACRA) 
enacted clinician payment reforms 
What OIG Found 
Key Takeaway 
designed to promote quality and value 
During 2017, CMS made significant efforts to 
of care.  These reforms, known as the  CMS has made progress 
address the two vulnerabilities that OIG 
Quality Payment Program (QPP), are  towards implementing the 
identified in its 2016 management review
a significant shift in how Medicare  QPP, but challenges remain.  
calculates compensation for clinicians  developing IT systems and preparing clinicians  CMS appears on track to 
and require the Centers for Medicare  to participate in the QPP.    deploy the IT systems needed 
  for data submission by 
& Medicaid Services (CMS) to develop 
a complex system for measuring,  With regard to IT, CMS appears on track to  January 1, 2018.  OIG has 
reporting, and scoring the value and  deploy the systems needed for data submission  identified two vulnerabilities 
that are critical for CMS to 
quality of care.  The first performance  by January 1, 2018.  It has added new content 
address in 2018 because of 
year began on January 1, 2017.    and functionality to the public website; 
their potential impact on the 
  deployed an IT product to make eligibility 
programs success:   
In 2016, OIG conducted an early  determinations; and tested, but not yet fully 
implementation review of CMSs  deployed, applications to support data  (1)  If clinicians do not receive 
management of the QPP.  While we  submission and real-time scoring.  sufficient technical 
found that CMS had made significant    assistance, they may 
struggle to succeed under 
progress in developing the QPP, we 
With regard to clinician readiness, CMS has 
the QPP or choose not to 
identified two vulnerabilities that 
conducted outreach, communicated eligibility 
participate.  
were critical for CMS to address in 
information, issued subregulatory guidance, 
2017:  (1) providing sufficient  (2)  If CMS does not develop 
and established a Service Center to respond to 
guidance and technical assistance to  and implement 
questions.  CMS also awarded a variety of 
ensure that clinicians were ready to  a comprehensive program 
technical assistance contracts; however, 
participate in the QPP, and  integrity plan for the QPP, 
contractors efforts to date have focused 
(2) developing information technology  the program will be at 
primarily on general education for a broad 
(IT) systems to support data reporting,  greater risk of fraud and 
audience rather than specialized technical  improper payments. 
scoring, and payment adjustment.  
assistance to address practice-specific needs.    
OIG conducted this followup review to 
Clinician feedback collected by CMS 
assess CMSs progress in mitigating 
these potential vulnerabilities and to  demonstrates widespread awareness of the QPP, but also uncertainty 
identify emerging risks.    about eligibility, data submission, and other key elements of the 
program. 
How OIG Did This Review   
We interviewed CMS staff and  With regard to emerging challenges, we found that CMS has not yet 
reviewed internal CMS documents as  developed a comprehensive program integrity plan for the QPP.  
well as publicly available information.   Appropriate oversightparticularly to ensure the accuracy of 
We conducted qualitative analysis to  clinician-submitted datais critical to prevent improper QPP payment 
adjustments.  Although CMS included oversight provisions in the QPP 
identify key milestones (both those 
final rule for 2017 and has initiated oversight planning, it still needs to 
achieved and those yet to come), 
clearly designate leadership responsibility for QPP program integrity 
priorities, and challenges related to 
and develop a plan to prevent and address fraud and improper 
QPP implementation. 
payments. 
  Full report can be found at http://oig.hhs.gov/oei/reports/oei-12-17-00350.asp  Background   
T 
 he Medicare Access and CHIP (Childrens Health Insurance Program) Reauthorization Act of 
2015 (MACRA) was enacted in April 2015 with bipartisan support.1  The Centers for Medicare & 
Medicaid Services (CMS) is implementing core provisions of MACRA as the Quality Payment 
Program (QPP), a set of clinician payment reforms designed to put increased focus on the 
quality and value of care.2, 3  The QPP is a significant shift in how Medicare calculates payment for 
clinicians and requires CMS to develop a complex system for measuring, reporting, and scoring the 
value and quality of care.   
 
In 2016, OIG conducted an early implementation review of CMSs management of the QPP.4  We found 
that CMS had made significant progress towards implementing the QPP, including fostering clinician 
acceptance, adopting integrated business practices, building IT systems, and developing key program 
policies.  However, our review also identified two potential vulnerabilities that were critical for CMS to 
address in 2017 because of their potential impact on the programs success:   
(1) Completing information technology systems to support critical QPP functions.  In the past, 
CMS has experienced delays and complications related to major information technology (IT) 
initiatives.  If CMS does not complete the complex IT systems underlying the QPP on 
schedule, implementation of quality-based payment adjustments may be delayed.   
(2) Ensuring clinician readiness to participate in the QPP.  If clinicians lack sufficient information 
and assistance, they may struggle to meet QPP reporting requirements or choose not to 
participate at all.  
 
The objectives of this followup review were to assess CMSs progress in mitigating these potential 
vulnerabilities and to identify emerging challenges.  If CMS fails to sufficiently address these issues, the 
QPP may be unable to achieve its goal of promoting high-value care and patient outcomes while 
minimizing burden on clinicians. 
  
CLINICIAN PAYMENT UNDER MEDICARE PART B  
Medicare pays clinicians (such as physicians and nurse practitioners) for their services through the 
Part B benefit.  Clinician services include office visits and surgical, diagnostic, and therapeutic 
procedures.  CMS bases its payment rates for over 7,000 clinician services on the Medicare Physician 
Fee Schedule.5, 6  Prior to the passage of MACRA, these payment rates were intended to be updated 
annually using the sustainable growth rate (SGR) formula.7  The SGR was designed by Congress to 
control Medicare spending by either reducing or increasing Part B payment rates when spending fell 
above or below a set target.8, 9  However, in practice, the SGR led to projected reductions in annual 
payment rates that many stakeholders criticized as being too severe.  As a result, Congress overrode 
the SGR payment-rate reduction each year from 2003 to 2015 and opted to either maintain or increase 
payment rates, an annual process that came to be known as the doc fix.10   
 
OVERVIEW OF THE QUALITY PAYMENT PROGRAM  
MACRA requires that on January 1, 2019, CMS must begin making Medicare Part B payment 
adjustments to clinicians based on the quality and value of care they provide rather than based on the 
volume of services provided.  These adjustments are determined by clinicians performance as assessed 
through one of two tracks:  the Merit-Based Incentive Payment System (MIPS) or Advanced Alternative 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               2  Payment Models (Advanced APMs).  CMS refers to these two tracks together as the QPP.  (See Exhibit 1 
for a summary of the QPP program tracks.)   
 
  Exhibit 1.  Overview of Tracks for the Quality Payment Program 
Merit-Based Incentive Payment  Advanced Alternative Payment Models 
System (MIPS) Track  (Advanced APMs) Track 
 Components of MIPS final score:   CMS determines which models meet 
o Quality   criteria for Advanced APMs. 
o Advancing Care Information 
 
o Improvement Activities 
 Qualifying APM Participants have a certain 
o Cost (not included in 2017 MIPS 
OR  proportion of their Medicare payments or 
score) 
patients through the Advanced APM. 
 
 
 Payment adjustment: MIPS must be 
 Payment adjustment: Qualifying APM 
budget neutral, so CMS may scale 
Participants will receive an annual 
positive payment adjustments as needed 
5% lump-sum bonus from 2019 through 
to achieve neutrality. High performers 
2024.  Beginning in 2026, they will receive 
will be eligible for additional payments 
higher physician fee-schedule updates, 
for exceptional performance. 
compounded annually. 
 
Source: OIG analysis of CMSs implementation of QPP, 2017. 
 
Merit-Based Incentive Payment System.  Using the Physician Fee Schedule as a base rate, MIPS will 
adjust clinicians Medicare Part B payments up or down based on their performance in four categories:   
  quality,  
  advancing care information, 
  improvement activities, and 
  cost.11, 12   
 
Within each category, CMS has defined a set of possible measures or activities.  For the first three 
categories, clinicians select from a menu of CMS-approved measures and must report data to CMS 
annually.  For the Cost category, CMS calculates certain measures based on claims data.  As described 
below in Exhibit 2, clinicians may use a variety of options to report these data (e.g., claims data, 
electronic health records, and qualified clinical data registries).  Additionally, clinicians may report 
individually or as part of a group, and they may use vendors or authorized staff (e.g., data registries or 
the administrator of a group practice) to submit MIPS data on their behalf.    
 
For each clinician, CMS calculates a MIPS final score based on the clinicians performance across the 
categories during a designated time period, or performance period, during the calendar year.13  CMS 
uses clinicians MIPS scores for a given year to adjust Medicare Part B payments 2 years later.  
For example, CMS will use scores for 2017 (the first year for which CMS will assess performance under 
the QPP) to adjust payments in 2019.  To determine payment adjustments, CMS compares each 
clinicians MIPS score to a performance threshold and increases or decreases that clinicians Part B 
payments accordingly.  (See Appendix A for further detail about MIPS scoring.)   
 
   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               3  Exhibit 2. Submission of MIPS Data 
 
Source: OIG analysis of CMSs implementation of QPP, 2017. 
Note: Qualified clinical data registries are distinct from regular data registries because they are not limited to submitting MIPS 
measures.  Qualified clinical data registries are permitted to submit alternative measures that, subject to CMS review and approval, 
will count towards the MIPS score. 
 
For the 2017 performance period, clinicians are generally exempted from MIPS reporting requirements 
if they:  
(1) meet the low-volume threshold of having no more than $30,000 in Medicare Part B charges 
or no more than 100 Part-B enrolled Medicare beneficiaries during the performance period14;  
(2) enroll in Medicare for the first time; or 
(3) meet criteria for participation in the Advanced APM track (see below).15 
CMS estimates that between 592,119 and 642,119 clinicians are eligible to participate in MIPS for the 
2017 performance year. 
 
CMS has designated 2017 as a transition year in which clinicians may pick their pace for reporting 
MIPS data.16  Clinicians who are subject to MIPS may elect one of the reporting options outlined in 
Exhibit 3.  During the first performance period, CMS will not negatively adjust payments if clinicians 
submit a minimum amount of data to CMS.  Clinicians who submit more measures or data for the full 
year will be eligible for positive payment adjustments. 
 
Exhibit 3. Pick Your Pace Options To Report MIPS Data for the First Performance Year  
 
Pace  Reporting Period  Payment Adjustment 
Submit Nothing  Clinician does not submit any data  Negative 4-percent payment 
to CMS  adjustment 
Submit  Clinician reports a minimum  No payment adjustment 
Something  amount of 2017 data to CMS  
Submit a Partial  Clinician reports data for a 90-day  Either no payment adjustment or a 
Year  period in 2017 to CMS  positive adjustment, depending on 
the data submitted 
Submit a Full  Clinician reports data for the full  Eligible for a positive payment 
Year  year (2017) to CMS  adjustment 
Source: CMS, Quality Payment Program: Pick Your Pace in MIPS, from https://qpp.cms.gov/.  
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               4   
Advanced APMs.  MACRA established criteria for designating certain payment and care delivery models 
as Advanced APMs.17  See Appendix B for the list of the programs designated as Advanced APMs for the 
first performance period.  These models are designed to financially reward high-quality and 
cost-efficient care.  Clinicians with a certain proportion of their Medicare Part B patients or payments 
received through an Advanced APM are deemed Qualifying APM Participants.  These clinicians are 
exempt from MIPS reporting and are instead subject to a different set of incentive payments.  
Specifically, Qualifying APM Participants will receive a 5-percent annual bonus during payment years 
2019 through 2024; beginning in 2026, Qualifying APM Participants will receive higher physician 
fee-schedule updates.   
 
IT SYSTEMS TO SUPPORT THE QPP  
Building the IT systems to support the QPP is a significant undertaking for CMS, requiring both 
public-facing products (e.g., an interface for data submission) and back-end systems (e.g., a module to 
calculate MIPS final scores).  Further, these complex systems must be completed on schedule so that 
key elements of the program, such as data submission, can occur according to the timeframe specified 
in statute and regulation.  The IT systems for the QPP encompass the following six products: 
  The platform is the infrastructure that underlies and supports all of the other QPP products.  It 
ensures that various development efforts are coordinated and employ common methods. 
  The website is the central site where all clinicians, their partners, and developers interested in 
interacting with the QPP come to perform tasks.  Ultimately, it will include both public 
webpages with general information and other pages where individual users can access secure, 
authenticated accounts providing QPP performance information. 
  The eligibility product uses CMS data sources to determine clinicians eligibility for the QPP 
(i.e., whether they are required to participate to avoid a negative payment adjustment), 
including whether they are qualified to participate under the MIPS track or the Advanced APM 
track.  
  The data submission product enables clinicians, as well as other staff or vendors authorized to 
provide data on their behalf (e.g., office administrators, registries), to submit MIPS data to 
CMS.  CMS will support a variety of submission mechanisms. 
  The scoring product will enable CMS to calculate each clinicians final MIPS score based on the 
data submitted.  These scores will also be used to determine the payment adjustment that 
each MIPS clinician will receive in 2019. 
  The feedback product will produce individualized reports providing clinicians with information 
about their performance, including their respective final MIPS scores and payment 
adjustments. 
 
As OIG reported in our previous review, the QPP website was launched in October 2016.  Additionally, 
eligibility determinations were made available to clinicians beginning in April 2017.  The remaining 
products were still under development as of fall 2017.   
 
CLINICIAN READINESS  
According to CMS staff, clinicians acceptance of the QPP and readiness to participate are critical to the 
programs success.  However, stakeholders have expressed concerns about whether providers
especially solo, small-practice, and rural providerswill be technically and logistically ready to 
participate in the QPP.  Clinicians will require information and support from CMS for three key activities 
during 2017, as described below:    
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               5   
Determining the appropriate QPP track.  Clinicians need to determine whether they are eligible for the 
MIPS or the Advanced APMs trackor, alternatively, whether they are exempted from participation 
altogether (for example, if they are under the low-volume threshold).  
 
Selecting measures and collecting data.  For the 2017 performance year, MIPS-participating clinicians 
must choose from an array of 271 Quality measures,18 15 Advancing Care Information measures,19 and 
92 Improvement activities.20  Clinicians need to select the combination of measures and activities best 
suited to their respective practices while also meeting MIPS reporting requirements.  They must also 
decide whether they will participate in MIPS as individuals or as part of a group and which reporting 
mechanism (e.g., data registries, electronic health record (EHR) vendors) will best fit their needs.  
Qualifying APM Participants do not need to select MIPS measures. 
 
Submitting data to CMS.  MIPS-participating clinicians must report their performance data to CMS by 
March 31, 2018.  Qualifying APM Participants do not need to report any additional QPP data to CMS.21  
  
To assist clinicians with these tasks, CMS has sponsored outreach events and awarded contracts under 
a variety of technical assistance programs.22  Technical assistance contractors are responsible for 
providing customized education, support, and assistance that is appropriate to each clinician and 
practice type.  Effective and timely technical assistance may also help reduce the burden on clinicians 
as they navigate the new program.   
 
TIMELINE 
The performance periods for MIPS and for Advanced APMs are broadly aligned with each other.  For 
both, the first performance period began on January 1, 2017.  Cliniciansor their authorized staff
have from January 1, 2018, through March 31, 2018, to report their 2017 performance data to CMS.  
Using the data submitted, CMS will calculate final MIPS scores and provide performance feedback to 
clinicians in July 2018.  MIPS payment adjustments and Advanced APM bonuses based on 2017 
performance will go into effect in 2019.  The QPP will operate on an overlapping 3-year program cycle.  
The first year of each cycle is the performance period; the second year is for reporting data and 
calculating scores; and the third year is for adjusting payment.  (See Appendix C.)  
 
METHODOLOGY 
To describe CMSs progress in mitigating the vulnerabilities that OIG identified in our 2016 report and 
to identify emerging challenges, we interviewed CMS staff and reviewed relevant internal CMS 
documents as well as publicly available information.   
 
Scope.  This review describes CMSs activities to implement the QPP.  We did not review other HHS 
operating divisions or staff divisions involved in QPP operations, such as the Assistant Secretary for 
Planning and Evaluation.  Our review primarily describes CMSs activities through fall 2017. 
 
Interviews.  We interviewed CMS staff between July 2017 and September 2017.  For each of the 
vulnerabilities that OIG identified in last years report (IT system completion and clinician readiness), 
we asked about CMSs current priorities; progress to date; the timeline for remaining activities; roles 
and responsibilities; communication; and challenges.  Additionally, we asked CMS officials to identify 
any emerging risks to QPP implementation.  Finally, we interviewed CMS staff regarding the agencys 
activities to ensure adequate oversight and program integrity for the QPP, and we requested followup 
information in writing. 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               6   
Documents.  We requested and received from CMS a variety of documents related to its QPP 
implementation efforts.  These included, but are not limited to, materials on operations and 
management activities; communications and outreach activities; technical assistance and training 
activities; IT development activities; and QPP organizational structures.  We also reviewed publicly 
available information, such as CMS training and technical assistance materials; resource tools for 
developers; and program guidance documents. 
 
Analysis.  We conducted a qualitative analysis of interview responses, CMS documents, and publicly 
available information to identify key milestones (both those achieved and those yet to come), 
priorities, and challenges related to developing IT systems and fostering clinician readiness.  
 
To identify emerging challenges, we conducted a qualitative analysis of interview transcripts and 
documentation to determine whether additional challenges were (a) directly described or (b) suggested 
by gaps in the initial information CMS provided.  Following this analysis, we requested an additional 
interview and written information specific to CMSs program integrity efforts related to the QPP, which 
we then reviewed to determine the extent to which the agency has planned and/or undertaken QPP 
oversight activities. 
 
To identify vulnerabilities, we reviewed priorities and milestones yet to be achieved and identified 
those that both (a) will require extensive, sustained CMS activity in 2018 to address and (b) pose 
a significant risk to the QPPs success if insufficiently addressed.     
 
Limitations.   Our review focused on CMSs management of the QPPs implementation during the first 
performance year.  At this point in the programs development, we did not assess the extent to which 
CMSs management of the QPP, or the QPP itself, will be successful in meeting program requirements 
and goals.  Our review relied on self-reported information from a purposively selected sample of CMS 
staff involved in QPP implementation.  We did not interview all CMS staff involved in the QPP, nor did 
we interview any contractors or external entities (e.g., clinicians participating in user testing).  We 
reviewed a selective set of CMS documents that we requested based on their relevance to our study 
objectives.   
 
With regard to IT security and functionality testing, OIG did not conduct an independent evaluation or 
audit of the QPP IT systems.  The information reported here is based on interviews with CMS officials 
involved in IT development and testing as well as on documentation about testing results that CMS 
provided to OIG.   
 
Standards. This study was conducted in accordance with the Quality Standards for Inspection and 
Evaluation issued by the Council of the Inspectors General on Integrity and Efficiency. 
   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               7  COMPLETING IT SYSTEMS TO SUPPORT CRITICAL QPP FUNCTIONS 
STATUS:  CMS has made significant progress developing the IT systems necessary to support the 
QPP.  CMS appears on track to deploy the systems needed for submission of MIPS data by January 
1, 2018.  During 2018, CMS will need to finalize clinicians MIPS scores, provide them with feedback 
on their performance, and calculate their 2019 payment adjustments. 
WHATS BEEN DONE  WHATS STILL TO COME 
 Platform  Developed a consistent approach   Platform  Complete security testing of new 
and a flexible, scalable infrastructure   QPP functions and maintain ongoing security 
 Website  Added new content and functionality  monitoring 
to public QPP website   Website  Add authenticated accounts by 
 Eligibility  Developed and deployed IT  January 1, 2018 
products to make QPP eligibility determinations   Eligibility  Make final determinations of 
 Data submission  Tested, but has not fully  eligibility 
deployed, applications to support submission of   Data submission  Finalize and release data 
MIPS data   submission applications by January 1, 2018 
 Scoring  Tested, but has not yet deployed,   Scoring  Fully score data submitted by 
real-time scoring   clinicians and calculate payment 
adjustments 
 Feedback  Provide clinicians with feedback 
on their MIPS performance in July 2018 
 
OVERVIEW 
The IT systems for the QPP encompass six products:  platform, website, eligibility, data submission, 
scoring, and feedback.  As depicted in Exhibit 4 on the following page, the platform is the infrastructure 
that provides the fundamental support for all QPP products and ensures a consistent approach to IT 
development and deployment across the program.  The QPP website is the main conduit for all public-
facing QPP IT functions and will include both a public site with general program resources and an 
authenticated account site where clinicians can access individualized QPP participation information.  
The public site includes overviews of the MIPS and Advanced APM tracks, a resource library, and a tool 
where clinicians can look up their QPP eligibility status.  The authenticated account site will ultimately 
include capabilities for clinicians, and their authorized staff, to learn more about their eligibility status, 
submit MIPS data, receive a preliminary score after submitting data, and receive performance feedback 
about their final MIPS score and associated payment adjustment.   
 
Overall, IT development appears on track to deploy all products necessary for data submission to begin 
on January 1, 2018.  Below, we describe the status of the six QPP products as of August 2017.  We also 
describe CMSs plans for developing additional QPP IT capabilities. 
   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               8   
 
Exhibit 4. Relationship Among the QPP IT Products 
 
Source: OIG analysis of CMSs implementation of QPP, 2017. 
 
WHATS BEEN DONE 
Platform  Developed a consistent approach and a flexible, scalable infrastructure.  CMS has 
continued to use the agile development methods it adopted last year to build, test, and deploy IT 
products for the QPP.  Agile development is an iterative approach23 that provides the IT team with 
frequent opportunities to adjust priorities, conduct testing, respond to user feedback, and add new 
functionalities.  Additionally, CMS requires all teams to use open-source products to share and test 
code as it is developed.  CMS staff said that requiring the same tools and methods across the QPP IT 
teams will ensure that the separately developed products are consistent and can be integrated.  CMS 
staff also reported that the iterative process has enabled them to respond quickly to policy changes 
that necessitate coding changes, because the infrastructure is in place to quickly test and then deploy 
revised code.      
 
CMS staff reported that CMS has also adopted an iterative approach to functionality and security 
testing for the QPP so that testing occurs throughout the IT development process.  OIG has not 
independently evaluated the IT security of QPP products.  However, CMS provided documentation 
indicating that it had conducted functionality and security testing throughout the IT development 
process, including a Security Impact Analysis whenever 
significant new functions were introduced.  Additionally, 
In historical terms, there was 
CMS staff reported that the testing team uses multiple 
always concern with what you will 
analysis tools to test all new code for potential security 
find [during the] late stages of 
flaws, and that new code is subject both to automated 
development. Doing [testing in an 
functionality testing, as well as user story tests in which 
the team assesses how the website functions for a typical  iterative] fashion relieves a lot of 
user in certain scenarios.    that anxiety. You will have already 
  found the problems along the way 
CMS also provided documentation indicating that, as of  and have already resolved them.   
August, its functionality tests had primarily revealed defects  CMS official  
rated as minor.  Few defects were determined by CMS to 
be critical or major, and CMS staff reported that these 
problems had been addressed.  Similarly, CMS provided documents indicating that potential security 
problems identified during testing had been resolved. 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               9   
Finally, CMS said that to provide flexibility, it designed the QPP IT platform to rely on cloud computing 
through Amazon Web Services rather than the traditional server room used for prior IT initiatives.  
Under this approach, capacity is scalable and can be purchased as needed, enabling CMS to boost 
capacity during high-demand periods, such as upcoming submissions of MIPS data, and then reduce 
capacity when it is no longer needed.  
 
Website  Added new content and functionality to public QPP website.  CMS launched the QPP 
website on October 14, 2016.  During 2017, CMS added new content and functionalities such as 
technical assistance resources, guides to MIPS measures applicable to particular specialties, and lists of 
CMS-approved vendors for data submission.  CMS staff reported that as the IT team develops new 
website features, it continues to conduct user research to gather feedback and identify areas for 
improvement.   
 
Eligibility  Developed tools to determine clinicians QPP eligibility.  To determine which clinicians are 
required to report MIPS data, CMS developed an eligibility-system component that compares data from 
Part B claims and the Provider Enrollment, Chain and Ownership System (PECOS)24 to the criteria 
established in the MACRA statute and regulation.25  In December 2016, CMS conducted an initial 
determination based on clinicians claims from September 2015 through August 2016.  CMS will repeat 
this analysis in December 2017 using claims from September 2016 through August 2017. 
 
To identify clinicians eligible to participate in the Advanced APM track, CMS extracted Medicare claims 
data and information about participating clinicians from Advanced APMs program records and 
manually compared those data with the Qualifying APM Participant criteria established in the QPP final 
rule for 2017.26  CMS used this process to take three snapshots (on March 31, June 30, and 
August 31) to assess eligibility for the Advanced APM track.  In October, CMS publicly released a tool 
that clinicians can use to look up whether they attained Qualifying APM Participant status based on the 
snapshot data.   
 
Data submission  Tested, but has not fully deployed, applications to support submissions of MIPS 
data.  In developing the data submission product, CMS staff reported that the agency is seeking to 
improve upon clinicians negative experiences with its previous quality reporting programs.  
Specifically, clinicians expressed concerns that under prior programs, they had no way of knowing after 
submitting data whether they had done so correctly.  Additionally, clinicians noted that the format they 
were required to use was labor-intensive.   
 
The validation process looks at the 
To address these concerns, CMS is including automated data 
general file format and . . . things 
validation checks in the MIPS data submission system that is 
that dont make sense 
under development.  This system is intended to provide 
mathematically.  And before the user 
clinicians and vendors with immediate feedback if the files 
gets out of the session, they can 
they upload do not pass the data validation checks, and will 
allow them to resubmit data files during the data submission  know whether they need to fix 
period (January 1 to March 31, 2018).  Additionally, CMS  something. . . . This is a huge 
intends to provide clinicians with a range of formats and  improvement over how things were 
options for submitting data.  (See Appendix D.)   validated in the legacy applications.  
   CMS official 
In addition to supporting a variety of data submission 
formats, CMS is encouraging the use of new Application 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               10  Program Interfaces (APIs) developed this year.  APIs are a method for data submission that allow 
machine-to-machine communication.  Through APIs, software developerssuch as registries and 
electronic health record vendorscan retrieve data directly for use in the developers own 
applications.  This provides additional options and convenience to clinicians, and also facilitates 
efficient handling during high-volume submission periods.  To assist vendors, CMS made the APIs for 
MIPS data submission available for developers to test beginning in July 2017.  As of late August 2017, 
over 100 vendor organizations were participating in the API testing. 
 
Scoring  Tested, but has not yet deployed, real-time preliminary scoring.  CMS has developed 
infrastructure to support real-time scoring so that clinicians can view their respective MIPS scores as 
soon as their data is submitted.  The preliminary score is intended to provide immediate feedback on 
how a clinician performed in each MIPS category (Quality, Improvement Activities, and Advancing Care 
Information).27 
 
WHATS STILL TO COME 
Platform  Complete integration and security testing of new QPP functions and maintain ongoing 
security monitoring.  As the first submission period approaches, CMS plans to continue and complete 
integration testing of all products being deployed.  CMS must also complete security testing of the 
eligibility, submission, and scoring products, which was planned for mid-November 2017.  Once these 
QPP IT products are deployed, CMS must conduct ongoing security monitoring to ensure that these 
functions continue operating securely and reliably.      
 
Website  Deploy authenticated accounts by January 1, 2018.  CMS must deploy secure, authenticated 
accounts no later than the beginning of the data submission period on January 1, 2018.  MIPS-eligible 
clinicians require authenticated QPP accounts to submit MIPS 
data, to receive a preliminary score after data submission, and to 
receive performance feedback reports about their final score and  We have to support base function in 
associated payment adjustment.  Clinicians will also use  year one, but the pipe dream is 
authenticated accounts to indicate which staff and vendors (e.g.,  having a user coming from [other 
EHRs and registries) are authorized to submit MIPS data on their  CMS systems] and then reusing their 
behalf.  Once designated by a clinician, authorized staff and  existing authorization to access QPP 
vendors must also obtain authenticated accounts before accessing 
in a way that minimizes their 
the systems for QPP data submission and feedback.  CMS is 
burden.   CMS official  
currently attempting to integrate the login information that 
clinicians use to access their existing accounts under CMSs legacy 
systems into each clinicians new QPP account.  It plans to deploy authenticated QPP accounts in 
December 2017. 
 
Eligibility  Make final eligibility determinations.  In December 2017, CMS plans to re-run its earlier 
eligibility analyses to make final determinations as to which clinicians are required to report MIPS data, 
which are Qualifying APM Participants, and which are exempt for 2017.  Additionally, for the Advanced 
APM track, CMS is developing an IT application (the APM Management System) to automate Qualifying 
APM Participant determinations.  CMS staff reported that as of October 2017, the APM Management 
System had passed all necessary security testing and was expected to be finalized in fall 2017. 
 
Data submission  Finalize and release data submission applications by January 1, 2018.  CMS needs 
to complete its testing of the applications for submission of MIPS data and fully deploy the final 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               11  versions no later than January 1, 2018, when the first data submission period begins.  CMS staff 
reported that as of November 2017, the data submission product was on track to meet this deadline. 
 
Scoring  Fully score data submitted by clinicians and calculate payment adjustments.  CMS needs to 
complete and deploy the real-time, preliminary scoring function when the data submission period 
begins.  By the end of the data submission period on March 31, 2018, CMS will need to have finalized 
the IT product to calculate the final MIPS scores.  In spring 2018, CMS will also need to finalize and 
implement the process for calculating the payment adjustments associated with final MIPS scores.  
CMS staff reported that as of July 2017, the scoring product was on track to meet these deadlines.  
 
Feedback  Provide clinicians with feedback on their MIPS performance during summer 2018.  During 
summer 2018, CMS must provide MIPS-eligible clinicians with feedback on their individual performance 
and the associated payment adjustment they will receive in 2019.  During 2017, CMS prepared for this 
by conducting user testing to better understand the types of performance feedback that would be most 
useful to clinicians and clinicians preferences for displaying that information in feedback reports.  
During spring 2018, CMS must finalize the format and functionality for the performance feedback 
reports, so that these reports are ready to compile and distribute in July 2018.   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               12   ENSURING CLINICIAN READINESS TO PARTICIPATE IN THE QPP 
STATUS:  CMS has directed an extensive campaign to raise clinicians awareness of the QPP, 
including providing education and assistance; communicating eligibility determinations; and 
establishing a Service Center.  CMS has stated that as implementation continues, it will place 
increased focus on specialized technical assistance to meet practice-specific needs. 
WHATS BEEN DONE  WHATS STILL TO COME 
 Conducted an outreach campaign to raise   Continue outreach activities to maximize  
clinician awareness of the QPP  clinician awareness and understanding 
 Provided technical assistance to clinicians   Increase focus on specialized technical 
and practices to assist with QPP  assistance to support successful 
preparedness  participation 
 Communicated clinicians initial QPP   
eligibility determinations   
 Issued subregulatory guidance 
 Established a Service Center to respond to 
questions and refer clinicians to technical 
assistance contractors  
 
VULNERABILITY: 
 If clinicians do not receive sufficient information and assistance, they may struggle to succeed 
under the QPP or choose not to participate.  This is of particular concern for small practices and 
clinicians in rural or medically underserved areas, who may lack the resources to fully engage in 
the QPP without customized technical assistance to meet practice-specific needs.   
  
OVERVIEW 
Since the QPPs inception, CMS officials have consistently stated that clinicians acceptance of and 
readiness to participate is crucial to the programs success.  For 2017the programs first performance 
periodCMS set a goal of 90 percent participation in QPP.  
To reach this goal, CMS has used multiple channels to  We started with a baseline of zero
educate clinicians, provide technical assistance, and collect  no one had knowledge about the 
feedback.  For example, CMS held numerous webinars and 
program, and we had a short time to 
other events; issued subregulatory guidance; and 
bring physicians up to speed.  Now, 
established a Service Center to respond to questions and 
we have everyone talking about 
resolve problems.  CMS also awarded a variety of contracts 
QPP.   CMS official  
to provide technical assistance specific to clinicians practice 
types and needs. 
 
Through these efforts, CMS has raised awareness of the QPP, and a majority of eligible clinicians have 
reported to CMS that they intend to participate.  CMS staff said that because the QPP was an entirely 
new initiative, it was necessary for early outreach efforts to focus on general education and awareness.  
However, CMS staff report that as QPP implementation continues, a greater focus on specialized, 
practice-specific technical assistance will be needed to help clinicians fully participate in the new 
program.   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               13  WHATS BEEN DONE 
Conducted an outreach campaign to educate clinicians about the QPP.  As of September 2017, CMS 
and its cosponsors had reached an estimated 176,000 people (including clinicians, practice managers, 
vendors, and other audiences) through a total of 631 outreach events.  These included nationally 
available webinars, speeches at medical association meetings, in-person regionally based trainings, 
town hall listening sessions, and other educational events.  Of the 405 events for which detailed 
information was available at the time of our analysis, approximately one-quarter were targeted to 
clinicians in rural areas.   
 
As part of its communications strategy, CMS has coordinated with stakeholders such as professional 
associations, vendors, the Rural Health Association, and clinician groups.  CMS also entered into 
a variety of cobranding arrangements for education and 
training delivery, partnering with organizations such as 
We know providers can speak better 
medical associations,28 so that clinicians can turn to a trusted 
to providers, so we capitalize on 
source and receive consistent messages.  Finally, CMS has 
peer-to-peer communications.               
continued to work with 12 Clinician Champions who 
 CMS official  
provide CMS with clinician perspectives while also sharing 
information about the QPP among their peers. 
 
A CMS tracking survey conducted in spring 2017 demonstrates both progress and the need for 
continued outreach.  The survey found high levels of awareness (71 percent) among practice managers, 
but lower levels of awareness among clinicians and midlevel practitioners (60 percent and 44 percent, 
respectively).  Respondents indicated substantial confusion regarding the Advanced APM track versus 
the MIPS track.  Overall, 63 percent reported intending to participate in 2017.  While these numbers 
have likely increased, it remains to be seen whether CMSs goal of 90 percent participation can be 
achieved.  CMS officials indicated that the agency will repeat the tracking surveys semiannually to 
continue to assess clinician awareness. 
 
Provided specialized technical assistance to clinicians and practices.  To succeed under the QPP, 
clinicians mustat minimumunderstand the program requirements and how to report performance 
data.  Because the QPP is designed to reward high-quality, high-value care, the program may also lead 
providers to implement changes to their clinical practices to increase focus on quality and care 
coordination.  Specialized technical assistance can help clinicians understand how to select measures; 
develop and implement MIPS score improvement plans; select EHR vendors; and other tasks. 
 
To address the needs of different practices, CMS established three technical assistance groups that are 
charged with assisting specific types of clinicians and practices.  Eleven QPP Small, Underserved, and 
Rural Support (QPP-SURS) contractors serve small practices and clinicians located in rural and medically 
underserved areas.  Additionally, 14 Quality Innovation Networks-Quality Improvement Organizations 
(QIN-QIOs) provide QPP assistance to large practices, i.e., those with 15 or more clinicians.  Finally, 
41 Transforming Clinical Practice Initiative (TCPI) cooperative agreement grantees help clinicians to 
modify their practices in ways that align with MIPS incentives and to prepare for eventual adoption of 
alternative payment models.  In addition to providing education and assistance, these three groups of 
technical assistance providers also serve as a key source of feedback to CMS, relaying the questions and 
concerns they hear from clinicians. 
 
CMS officials stated that as of July 2017, QPP-SURS and QIN-QIO contractors had devoted the majority 
of their efforts to broadly targeted outreach geared towards general awareness and education, while 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               14  25 to 30 percent of these contractors efforts were devoted to more specialized technical assistance 
focusing on practice-specific needs.  Differences in definitions, timeframes, and reporting formats over 
time and among contractor types make it difficult to quantify the proportion of clinicians who have 
received assistance.  Generally, contractors report the number of clinicians they have engaged through 
a range of activities, including online tools, webinars, and direct assistance (e.g., conducting technology 
assessments, supporting clinicians EHR use, and responding to questions).  Some of these activities are 
targeted to individual practices, while others are directed to small groups or a broader audience.  The 
overall estimate of how many clinicians have been reached by each contractor group is summarized 
below. 
  QPP-SURS contractors reported having provided technical assistance to approximately 
67,000 clinicians as of August 31, 2017.  This constitutes 37 percent of the total number of 
eligible clinicians in these contractors jurisdictions.  The small and rural practices served by 
QPP-SURS contractors are the population most likely to require support.  
  QIN-QIO contractors report monthly numbers rather than an unduplicated cumulative total of 
clinicians served.  During August 2017, the most recent month with data available, contractors 
reported providing technical assistance to approximately 120,000 clinicians.  This was slightly 
higher than the numbers reported for July (approximately 96,000 clinicians) and June 
(approximately 84,000 clinicians).  At the time of our review, the total number of eligible 
clinicians in all QIN-QIO jurisdictions was not available. 
  TCPI grantees do not report their numbers of technical assistance contacts; however, nearly all 
reported having a defined technical assistance program to prepare clinicians for QPP 
participation.  TCPI grantees described a variety of individualized technical assistance activities 
serving that goal, including troubleshooting EHR issues; working with clinicians to ensure that 
they correctly enter and validate data; and helping to develop and connect advanced practices 
to appropriate APMs. 
Communicated clinicians initial QPP eligibility determinations.  In the preamble to the QPP final rule 
for 2017, CMS indicated its intention to communicate MIPS eligibility information in December 2016.  
However, CMS did not begin to notify clinicians of MIPS eligibility until April 2017, a delay that raised 
concern among some stakeholders.  In May, CMS added 
an eligibility lookup tool to the QPP website; clinicians can 
[W]hat is powerful about [the 
use the tool at any time to receive individualized 
eligibility lookup tool] is that it allows 
information about their MIPS eligibility status.  Clinicians 
people to understand what is required 
who had questions about their MIPS eligibility or disagreed 
of them [individually].  We took 
with the information provided by CMS were directed to 
hundreds of pages of regulation and  contact the QPP Service Center.  CMS documents indicate 
put it into one interface.   CMS  that from June through August 2017, questions about MIPS 
official   eligibility were the largest source of inquiries to the Service 
Center.   
 
In June 2017, CMS also notified clinicians as to whether they were likely to meet the threshold for 
Qualifying APM Participant for 2017.  Additionally, a Qualifying APM Participant lookup tool is available 
on the QPP website.  CMS staff reported receiving few questions from clinicians about their predicted 
Qualifying APM Participant status.   
 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               15  Issued subregulatory guidance.  CMS issued subregulatory guidance on a variety of key issues, 
including group reporting options; data submission and reporting; data validation and auditing; MIPS 
measures; MIPS scoring; and APMs.  All guidance is publicly available through the QPP Resource 
Library. 
 
Established a Service Center to respond to questions and 
Every person that calls [the Service 
refer clinicians to technical assistance contractors.  CMS 
Center] is offered the opportunity to 
established a Service Center, accessible by phone or email, 
get additional technical assistance.   
to answer questions about the QPP and refer clinicians to 
 CMS official 
the appropriate technical assistance contractors.  CMS 
reported that as of late September 2017, the Service Center 
had received more than 47,500 inquiries from more than 27,000 unique users in 2017.   
 
CMS reported that 82 percent of all inquiries were resolved during the first contact with the Service 
Center.  In many cases (56 percent of all inquiries), the resolution included referral to a technical 
assistance contractor.  Technical assistance contractors reported to CMS that they typically made 
contact with their referrals within 24 hours.  Overall, callers reported a high degree of satisfaction with 
their Service Center experience; when surveyed by CMS, more than 90 percent indicated that their 
concerns were satisfactorily addressed. 
 
The Service Center also serves as a key source of clinician feedback, alerting CMS to potential problems 
and identifying common areas of confusion or misunderstanding.  For example, CMS reported that the 
initial notification letters sent to clinicians incorrectly identified some physical therapists as being 
eligible for MIPS, when in fact physical therapists are excluded.  A provider called the Service Center 
about the issue, and the Service Center immediately informed the IT team.  The IT team identified and 
fixed the source of the error, ensuring that clinicians using the eligibility lookup tool would now receive 
correct information.  Finally, CMSs Communications team worked with partners in the physical therapy 
community to inform them of the issue as well as the corrected information.  According to CMS, the 
entire incident took fewer than 72 hours to resolve.   
 
WHATS STILL TO COME 
Continue outreach activities to maximize clinician awareness and understanding of the QPP.  CMS 
reported that to ensure that all clinicians are aware of the QPP and understand what it requires, CMS 
will continue to conduct extensive outreach.  Although CMS officials described its technical assistance 
contractors as being on track to meet their goal of engaging all eligible clinicians in their regions by 
the end of 2017, data from contractors monthly reports suggest that there is significant variation 
among the contractors responsible for different geographic regions.  For example, as of August 31, 
2017, some contractors reported having engaged nearly all providers in their jurisdictions while others 
reported having engaged less than one-quarter.  Further, Service Center data indicate that clinicians 
continue to have questions about both eligibility and scoring criteria, and that small practices, in 
particular, need information and assistance.  
 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               16  Increase focus on specialized technical assistance to 
The tone and tenor of the types of 
support successful QPP participation.  To date, CMS has 
questions the contractors are 
emphasized general education and outreach to raise 
receiving will change from What is 
awareness of the QPP.  However, officials stated that the 
this? to How do I do it?  So, [the 
balance of effort should shift towards more specialized 
amount] of hands-on, customized 
technical assistance as the program enters its second year.  
technical assistance [provided by 
Such practice-specific technical assistance is likely to be 
especially important to small and rural clinicians, who CMS  contractors] is likely going to 
has anticipated may require help with clinical quality  increase as clinicians move beyond 
measurement, change management, practice workflow  their initial awareness of the 
redesign, EHR vendor selection, and other issues.    program.     CMS official 
 
 
VULNERABILITY:  IF CLINICIANS DO NOT RECEIVE SUFFICIENT TECHNICAL ASSISTANCE, THEY MAY 
STRUGGLE TO SUCCEED UNDER THE QPP OR CHOOSE NOT TO PARTICIPATE  
Clinician feedback collected by CMS demonstrates widespread basic awareness of the QPP, 
but also indicates uncertainty regarding details of participation such as who must report 
and how to submit data.  Further, to date, CMS contractors have focused largely on general education 
initiatives, with fewer resources devoted to more customized, practice-specific technical assistance.  
CMS needs to continue to assess progress and increase the proportion of contractors efforts devoted 
to specialized technical assistance to support high levels of clinician participation.  Small practices and 
clinicians in rural or medically underserved areas, who may have fewer administrative resources and 
less experience with prior CMS quality programs, should be prioritized for assistance.   
   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               17  EMERGING ISSUE:  ENSURING ADEQUATE OVERSIGHT AND PROGRAM INTEGRITY 
STATUS:  CMS has not yet developed a comprehensive program-integrity plan for the QPP.  
Appropriate oversight, particularly to ensure the accuracy of MIPS data, is critical to prevent fraud 
and improper payment adjustments. 
WHATS BEEN DONE  WHATS STILL TO COME 
 Included oversight provisions in the QPP final   Clearly designate leadership responsibility 
rule for 2017  for QPP program integrity 
 Established oversight process for   Develop and implement a comprehensive 
CMS-approved vendors  QPP program integrity plan that addresses: 
 potential vulnerabilities in the MIPS 
data submission system and 
 the accuracy of MIPS data submitted 
by clinicians, particularly the 
self-attestation measures 
VULNERABILITY: 
 If CMS does not develop and implement a comprehensive program integrity plan for the QPP, the 
program will be at greater risk of fraud and improper payments. 
  
OVERVIEW 
In addition to assessing CMSs progress in addressing the two vulnerabilities that we identified in our 
2016 reviewcompleting IT systems and ensuring provider readiness), we also sought to identify any 
emerging challenges or risks to the program.  From our interviews with CMS staff and our reviews of 
program documents, we identified a critical management issue:  delays in adequately planning for QPP 
program integrity.  We found that CMS has not yet developed a comprehensive plan for QPP program 
integrity, nor has the agency clearly designated roles and responsibilities for oversight of the program.   
 
Appropriate oversight, particularly to ensure the accuracy of MIPS data, is crucial to prevent fraud and 
improper payments.  Specifically, if the data that clinicians submit do not accurately represent their 
performance, they may inappropriately receive positive or negative payment adjustments.  Below, we 
describe CMSs efforts to date and identify key issues that remain to be addressed.    
 
WHATS BEEN DONE 
Included oversight provisions in the QPP final rule.  In the final rule for 2017, CMS included program 
integrity provisions for both the MIPS track and the Advanced APM track.  In terms of the MIPS track, 
the rule addressed auditing and requirements for record retention.  Specifically, according to the rule, 
CMS will selectively audit MIPS-eligible clinicians on a yearly basis and, if clinicians are selected for an 
audit, they must comply with requests for data-sharing and documents.29  To support such audits, CMS 
required MIPS-eligible clinicians and groups to retainfor up to 10 years after the performance 
periodcopies of medical records, charts, reports and any electronic data utilized to determine which 
measures and activities were applicable and appropriate for reporting under MIPS.  If a MIPS-eligible 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               18  clinician or group is found to have submitted inaccurate data for MIPS, CMS indicated that it would 
reopen, revise, and recoup any resulting overpayments. 
 
The QPP final rule for 2017 also included program integrity provisions applying to Advanced APMs.  
CMS will vet and monitor individuals and entities (e.g., clinician groups) participating in Advanced APMs 
as well as new applicants applying to participate.30  The final rule indicated that if a clinician is out of 
compliance with program requirements, or is terminated by an Advanced APM for program integrity 
reasons, the agency may reduce or deny the APM payment incentive to that clinician.  CMS also 
indicated that it would reopen and recoup any payments made in error.  
 
Established oversight process for CMS-approved vendors.  Because of the role of registries in gathering 
and reporting MIPS data on behalf of clinicians, CMS established an oversight process for registries.  
The process includes the following requirements: attestation that all data and results are accurate and 
complete; submission of a data validation report to CMS; and compliance with any CMS requests to 
review submitted data.  If data inaccuracies affect more than 3 percent of a vendors total MIPS-eligible 
clinicians, CMS may give that vendor a low rating for data quality on its listing of qualified registries and 
place it on probation.  Data inaccuracies affecting more than 5 percent of a vendors total MIPS-eligible 
clinicians could lead to the vendors disqualification for the following year.   
 
WHATS STILL TO COME 
Clearly designate leadership responsibility for QPP program integrity.  In discussing CMSs 
management approach to QPP implementation, CMS officials stressed the importance of assigning 
executive leaders for key aspects of the program, such as policy, communications, and IT development.  
However, CMS has not designated an individual as having leadership responsibility for QPP program 
integrity, although CMS noted that program integrity officials at various levels were present at biweekly 
QPP leadership meetings.  In interviews and written responses, CMS communicated that program 
integrity planning and activities would be shared across multiple divisions, but was not able to 
delineate how responsibilities are divided.  Additionally, CMS indicated that integrated project teams 
do not include staff dedicated to program integrity, although CMS noted that program integrity staff 
serve as a resource for the QPP teams and have collaborated on tasks such as provider authentication 
for the QPP websites authenticated accounts.  
 
Develop and implement a comprehensive QPP program integrity 
CMS is currently in the 
plan.  CMS needs to develop and implement a comprehensive program 
early stages of developing 
integrity plan.  CMS staff reported that senior staff who manage the 
an oversight plan for QPP 
QPP and the agencys overall program integrity efforts are planning to 
data.   CMS 
meet in December 2017 to initiate development of an oversight plan 
correspondence to OIG   
for the QPP.  CMS staff reported that they plan to use the Government 
Accountability Offices (GAO) Fraud Risk Management Framework as 
the basis for developing QPP oversight.31  They anticipated addressing three main program integrity 
issues: (1) oversight of the eligibility and enrollment of clinicians; (2) ensuring proper payments, and 
(3) ensuring the validity of MIPS data. 
 
In addition to addressing the risks that CMS identifies as it applies the GAO Fraud Risk Management 
Framework to the QPP, the plan should address two particular issues that OIG identified during our 
review: 
  potential vulnerabilities in the MIPS data submission system and 
  the accuracy of MIPS data submitted by clinicians, particularly the self-attestation measures. 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               19   
Potential vulnerabilities in the MIPS data submission system.  The QPP IT system for data submission is 
intended to provide clinicians with real-time feedback about how the data they submit affects their 
MIPS score.  This preliminary score will not indicate what level of payment adjustment the clinician will 
receive, but it is intended to provide immediate feedback on whether the clinician has earned the 
maximum points available.  More points are likely to translate to a higher positive payment adjustment.  
In addition to providing real-time scoring, the submission system is also intended to permit users to 
resubmit data files that will supersede their prior submissions.  This resubmission function is scheduled 
to be available during the open period for QPP data submission, i.e., January through March 2018. 
 
The submission systems capabilities to provide real-time scoring and allow resubmissions were 
designed to provide clinicians with efficient feedback and the opportunity to correct data errors.  
However, these same functions make it possible for a user to game the submission system to obtain 
a higher MIPS score than the one to which the user is entitled, therefore potentially resulting in an 
improper positive payment adjustment.  For example, to submit data on Advancing Care Information 
measures, clinicians will select a series of checkboxes to certify to their EHR use.  Once the user 
completes the checklist, the system will provide a real-time score and indicate whether the clinician 
received the maximum points available.  If users are not satisfied with their first score, the system will 
permit them to go back, complete the checklist again with different answers, and see how that affects 
their score.  Although users may resubmit their data to obtain the highest score to which they are 
lawfully entitled, this same resubmission process could be abused if users submit inaccurate data solely 
for the purpose of obtaining the maximum possible points. 
 
CMS staff confirmed that the data submission system will maintain multiple versions of submissions, 
making it possible for a reviewer to determine how and when clinicians change their data submissions.  
However, staff with expertise in program integrity were not consulted in the design of the system to 
ensure that data are collected and maintained in such a way as to facilitate oversight.  Additionally, 
CMS does not yet have a program integrity plan in place to ensure that data submissions are reviewed 
for evidence of fraud. 
 
Accuracy of MIPS data submitted by clinicians, particularly the self-attestation measures.  To ensure 
proper MIPS payment adjustments, CMS must also ensure the accuracy and validity of data submitted 
by clinicians and their authorized staff.  For the 2017 transition year, CMS indicated that its data 
validation efforts will focus on those clinicians who submitted fewer than the required number of 
quality measures and will determine whether the clinicians submitted all applicable measures for their 
specialty.32  Additionally, CMS incorporated data validation functions into its data submission system, 
but these automated checks were designed only to identify incorrect file formats and quality-measure 
data that do not make sense (e.g., a reported quality measure where the numerator is larger than 
denominator).  CMSs automated data validation processes cannot detect whether incorrect or falsified 
data are being submitted.  Rather, CMS solely relies on the certification of either the clinicians or their 
authorized staff that they are submitting accurate data.  To address this vulnerability, CMSs QPP 
program integrity plan needs to include a robust audit strategy to review the accuracy and validity of 
MIPS data, particularly the self-attestation measures.   
 
In a prior report, OIG identified problems with the accuracy of the attestation data submitted by 
providers during the legacy EHR Meaningful Use program, which has been replaced by the Advancing 
Care Information category in MIPS.33  Like the latter, the former required clinicians to self-attest to 
performing certain activities.  OIG found that from May 2011 through June 2014, CMS paid an 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               20  estimated $729 million in Medicare EHR incentive payments to eligible professionals who did not 
comply with Federal requirements.  OIG found that these errors occurred because providers did not 
maintain sufficient support for their attestations and because CMS conducted minimal documentation 
reviews of the self-attestations.  OIG recommended that as CMS implements MACRA, it should include 
stronger program integrity safeguards that allow for more consistent verification of the reporting of 
required EHR measures.  Without robust additional oversight, CMSs current process for validating 
Advancing Care Information attestation data seems likely to perpetuate the same problems found with 
the predecessor EHR program.  In addition, the new MIPS reporting category on Improvement Activities 
is also based on self-attestation measures and may therefore also be vulnerable to fraud, abuse, or 
improper payments. 
 
VULNERABILITY:  IF CMS DOES NOT DEVELOP AND IMPLEMENT A COMPREHENSIVE QPP PROGRAM 
INTEGRITY PLAN, THE PROGRAM WILL BE AT GREATER RISK OF FRAUD AND IMPROPER PAYMENTS 
To ensure that the QPP succeeds, CMS must effectively prevent, detect, and address fraud 
and improper payments.  QPP payment adjustments are intended to reward high-value, 
high-quality care.  Safeguarding the validity of MIPS data and the accuracy of QPP payment 
adjustments is critical to ensure that these payments are based on clinicians actual performance.      
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               21  Conclusion 
OIGs previous review found that CMS had made significant progress towards implementing the QPP, 
but we also identified two potential vulnerabilities:  completing IT systems and ensuring clinicians 
readiness to participate in the QPP.  In this followup review, we assessed CMSs progress in addressing 
these vulnerabilities and also sought to identify emerging challenges.  
 
With respect to the first vulnerability, we found that CMS appears on track to deploy the IT systems 
needed for submission of MIPS data by January 1, 2018.  CMS has continued to use the agile 
development methods it adopted last year, incorporating functionality and security testing throughout 
the process.   
 
With respect to the second vulnerability, we found that CMS has carried out an extensive outreach 
campaign.  However, thus far, the majority of efforts have been directed toward general awareness 
rather than specialized technical assistance designed to meet practice-specific needs.  CMS staff 
indicated that as implementation continues, cliniciansparticularly those in small or rural practices
will have a greater need for customized assistance to support successful QPP participation.   
 
Additionally, we identified program integrity as an emerging challenge.  The QPP relies on clinicians to 
submit their own performance data, including self-attestations.  Providers may mistakenly submit 
inaccurate data, or could even potentially submit falsified data (for example, attesting to activities they 
did not perform) to receive a positive payment adjustment.  Despite these risks, CMS has conducted 
minimal planning with regard to program integrity.  Further, although CMS has designated executive 
leadership for other aspects of the QPP, it has not designated an executive lead on QPP program 
integrity.  Without a comprehensive plan for program integrity, the QPP will be at heightened risk of 
fraud and improper payments, particularly related to MIPS payment adjustments.  
 
If clinicians do not fully understand how to participate in the QPP, or if they lack the tools and support 
to make the practice changes necessary to respond to QPP incentives, the program may have limited 
success.  Additionally, without adequate program integrity measures in place, the performance data 
submitted may not reflect the true cost or quality of care provided, similarly compromising the QPPs 
ability to achieve its goals. 
 
OIG will continue to monitor CMSs progress in developing and operating the QPP and will conduct 
additional reviews as appropriate.   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               22  Appendix A  
OVERVIEW OF THE MIPS FINAL SCORE FOR PERFORMANCE YEAR 1, 2017 
In the final rule for 2017, CMS finalized the proportion that each MIPS category will contribute to the 
final score.34 
 
Measures:  Cliniciansmust report on six measures or a defined set of 
specialty measures, which they may select from a variety of approved 
Quality: measures based on what is most applicable to their practice.
60% in Year 1 Reporting:  Clinicians may use a variety of options, including claims 
data, EHRs, and qualified clinical data registries. 
Predecessor program:  Physician Quality Reporting System (PQRS)
Measures: Two overall cost measures:  (1) per-capita costs for all 
attributed beneficiaries and (2) Medicare spending per beneficiary,
Cost: as well as 10 episode-based cost measures, as applicable to the clinician
0% in Year 1 Reporting:  Via claims data, so there are no additional reporting 
requirements for clinicians
Predecessor program:  Value-Based Modifier (VBM) Program
Measures: Five required measures with additional optional measures, 
focusing on interoperability and the use of EHR technology to support 
Advancing Care  health care delivery
Information:   Reporting:  Clinicians may use a variety of options, including EHRs and 
25% in Year 1 qualified clinical data registries
Predecessor program: Medicare EHR incentive program for eligible 
professionals
Improvement activities are designed to improve health care delivery, 
including care coordination, beneficiary engagement, population 
management, and health equity. 
CMS will annually publish an inventory list that will designate certain 
Improvement 
improvement activities as "high" or "medium" activities.
Activities:
Measures:  Most clinicians attest that they completed up to four 
15% in Year 1 improvement activities for a minimum of 90 days.
Reporting:  Clinicians may use a variety of options, including attestation 
data, EHRs, and qualified clinical data registries.
Predecessor program:  None
 
   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               23  Appendix B  
OVERVIEW OF ADVANCED APMS AVAILABLE IN THE FIRST QPP PERFORMANCE PERIOD 
Shared Savings Program Accountable Care Organizations (ACOs) are 
composed of doctors, hospitals, and other health care providers and 
Shared Savings 
suppliers who are supposed to provide coordinated, high-quality care.
Program 
In Tracks 2 and 3, the ACO assumes risks for shared losses as well as for 
(Tracks 2 and 3)
savings.*
Participants: 42 ACOs in Tracks 2 and 3 (as of October 2, 2017)
The Next Generation ACO Model allows provider groups to assume higher 
Next Generation 
levels of financial risk and reward than are available under the current 
Accountable Care 
Shared Savings Program. 
Organization Model
Participants: 44 ACOs (as of October 2, 2017)
Comprehensive End-Stage Renal Disease (ESRD) Care Model coordinates 
care for ESRD beneficiaries. 
Comprehensive       
Two tracks have been determined to be Advanced APMs: (1) the large 
End-Stage Renal 
dialysis organization (LDO) arrangement and (2) the non-LDO two-sided 
Disease Care Model 
risk arrangement.
(LDO and two-sided 
In these two-sided risk arrangements, physician practices assume risks for 
risk tracks) losses as well as savings.
Participants: 37 ESRD model participants (as of October 2, 2017)
The Comprehensive Primary Care Plus Model focuses on regionally based 
multipayer payment reform and primary care delivery transformation.
Comprehensive  CMS, commerical insurance plans, State Medicaid agencies, and other 
Primary Care Plus  selected payer partners will align on payment, data sharing, and quality 
Model metrics.
Participants: 2850 primary care practices in 14 regions (as of October 2, 
2017)
In the Oncology Care Model, physician practices enter into payment 
arrangements that include financial and performance accountability for 
Oncology Care Model 
episodes of care surrounding chemotherapy administration to cancer 
(Two-sided risk 
patients. 
arrangement)
CMS is also partnering with commercial payers in the model. 
Participants: 192 practices with 14 payers (as of October 2, 2017)
The Comprehensive Care for Joint Replacement model tests bundled 
payment and quality measurement for an episode of care associated with 
Comprehensive Care 
hip and knee replacements to encourage hospitals, physicians, and 
for Joint Replacement 
providers of post-acute care to work together to improve the quality and 
Model (Track 1)
coordination of care from the initial hospitalization through recovery. 
Participants: 67 geographic areas (as of October 2, 2017)
  
Source:  OIG analysis of CMSs implementation of the QPP, 2017. 
*In Track 1 of the Shared Savings Program, the ACOs share in program savings.  Without requiring ACOs to also assume risk for losses, Track 1 
does not meet the criteria to be an Advanced APM in the QPP.  Because ACOs in Tracks 2 and 3 of the Shared Savings Program are required to 
   
assume risks for both savings and losses, Tracks 2 and 3 qualify as Advanced APMs under the QPP.
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               24  Appendix C  
QPP TIMELINE: PERFORMANCE PERIODS, FEEDBACK, SCORING, AND PAYMENT ADJUSTMENT 
The QPP will operate on an overlapping 3-year program cycle.  The figure below shows the first three 
program cycles, which will begin in 2017, 2018, and 2019, respectively.  Subject to future rulemaking, 
program cycle four will begin in 2020, program cycle five in 2021, and so on. 
 
 
        
2017 
 
January  December:   
MIPS performance period 
January  August:  
Advanced APM performance period 
2018  2018 
March 31:    January  December:   
Clinicians send MIPS data to CMS  MIPS performance period 
January  August:  
April  June:   
Advanced APM performance period 
CMS analysis and scoring 
July:   
Clinicians receive feedback reports 
 
2019  2019  2019 
January  December:   March 31:    January  December:   
Payment adjustment period (based  Clinicians send MIPS data to CMS  MIPS performance period 
on 2017 data)  April  June:    January  August:  
  CMS analysis and scoring  Advanced APM performance period 
July:   
Clinicians receive feedback reports 
 
2020  2020 
January  December:   March 31:   
Payment adjustment period (based  Clinicians send MIPS data to CMS 
on 2018 data) 
April  June:   
CMS analysis and scoring 
July:   
Clinicians receive feedback reports 
 
2021 
January  December:  
Payment adjustment period (based 
on 2019 data) 
 
 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               25  Appendix D  
AVAILABLE FORMATS FOR SUBMISSION OF MIPS DATA 
 
Providers  MIPS Categories 
ans  ACI) 
nici A)  on (
Submission Format  Individual Clinicians  Group practice (any size)   Group practices of 25+ clionly  Quality  Improvement Activities (I Advancing Care Informati Cost 
Application Program Interface (API)  Registries, qualified clinical               
data registries, or electronic health records may integrate directly 
with CMS via API and pass data in QPP data format 
Extensible Markup Language (XML)  Registries or qualified               
clinical data registries may generate an XML file in QPP data 
format and submit to CMS via manual file upload on QPP website 
Quality Report Document Architecture (QRDA)-III  Export data               
from EHR in QRDA-III format, then manually upload on QPP 
website 
Attestation  Participants may manually attest Advancing Care               
Information (ACI) and Improvement Activities (IA) measures using 
the QPP website 
Quality Data Code  Supply Quality Data Code on a Part B claim               
Claims  CMS will score performance in the cost category based               
on analysis of claims data 
CMS Web Interface  Group practice staff may use CMS Web               
Interface on the QPP Website to download beneficiary sample 
data, add quality measures data offline, and then upload to CMS 
Web Interface.  For ACI and IA categories, group practice staff 
may either upload data exported from an EHR or manually attest 
using the CMS Web Interface. 
Source:  OIG analysis of CMSs implementation of the QPP, 2017. 
Note: Qualified clinical data registries are distinct from regular data registries because they are not limited to submitting MIPS 
measures.  Qualified clinical data registries are permitted to submit alternative measures that, subject to CMS review and approval, 
will count towards the MIPS score. 
   
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               26  Acknowledgments 
 
Heather Barton served as the team  leader for this study, and  Jennifer Hutnich served  as the lead  
analyst. Others in the Office of Evaluation and Inspections who conducted  the  study include  
Bahar Adili, Matthew Panicker, and Brian Tinsley.  Office of Evaluation and Inspections staff who 
provided support include Clarence Arnold, Evan Godfrey, and Christine Moritz.   Jessica Swanstrom from 
the Office of Management and Policy also provided assistance with report graphics. 
 
This report was prepared under  the  direction  of David Tawes, Regional Inspector General for Evaluation  
and Inspections in the Baltimore regional office, and Louise Schoggen, Assistant Regional Inspector 
General.   
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
To obtain additional information concerning this report or to obtain copies, contact the Office of Public  
Affairs at Public.Affairs@oig.hhs.gov. 
Followup: CMSs Management of the Quality Payment Program (OEI121700350)                               27    
Endnotes 
 
1 Medicare Access and CHIP (Childrens Health Insurance Program) Reauthorization Act of 2015 (MACRA), 
P.L. No. 114-10 (April 16, 2015).  
2 81 Fed. Reg. 77008 (Nov. 4, 2016). 
3 Clinicians included in the QPP include physicians, physician assistants, nurse practitioners, clinical nurse 
specialists, and certified registered nurse anesthetists.  MACRA,  101(c)(1)(C)).   
4 OIG, Early Implementation Review: CMSs Management of the Quality Payment Program, OEI-12-16-00400.  
Available from: https://oig.hhs.gov/oei/reports/oei-12-16-00400.pdf.   
5 Omnibus Budget Reconciliation Act of 1989, P.L. No. 101-239,  6102 codified at Social Security Act (SSA)  1848. 
The Medicare physician fee schedule is derived using a resource-based relative value scale, which includes three 
resource components: (1) total physician work, (2) practice expenses, and (3) malpractice expenses.  Each 
component is measured in terms of relative value units (RVUs).  The Medicare physician fee schedule payment 
rates are based on RVUs, adjusted for geography, and multiplied by a national conversion factor to derive dollar 
amounts.   
6 Section 1848(a)(1) of SSA established the Medicare physician fee schedule as the basis for Medicare 
reimbursement for all physician services beginning in January 1992.   
7 SSA  1848(f).   
8 Annual spending targets were updated each year by applying a growth rate known as the sustainable growth 
rate (SGR). The SGR formula incorporated four factors: (1) inflation, (2) changes in enrollment in Medicares 
fee-for-service program, (3) the estimated 10-year average annual growth rate of real gross domestic product per 
capita, and (4) the impact of changes in law or regulation.   
9 Congressional Budget Office (CBO), The Sustainable Growth Rate Formula for Setting Medicares Physician 
Payment Rates, September 2006, p. 5.  Accessed at https://www.cbo.gov/sites/default/files/109th-congress-
2005-2006/reports/09-07-sgr-brief.pdf on October 13, 2016.   
10 CBO, Medicares Payments to Physicians:  The Budgetary Impact of Alternative Policies Relative to CBOs 
March 2012 Baseline, July 2012.  Accessed at https://www.cbo.gov/publication/41512?index=12240 on 
October 14, 2016.   
11 MACRA,  101(c), SSA  1848(q)(2)(A). 
12 During the first performance period, CMS will calculate cost measures for each clinician, but these will not be 
included in the MIPS Final Score that determines payment adjustment. 
13 For 2017, the performance period is defined in regulation as a minimum of a continuous 90-day period within 
CY 2017, up to and including the full CY 2017.  45 CFR  414.1320. 
14 CMS will use a low-volume threshold determination period to identify providers that meet the low-volume 
threshold and are thus excluded from MIPS reporting and payment adjustment.  The low-volume determination 
period includes two 12-month segments; a provider who meets the threshold during either of the segments is 
excluded from MIPS.  The initial segment spans from the last 4 months of a calendar year 2 years prior to the 
performance period through the first 8 months of the next calendar year.  The second segment spans from the 
last 4 months of a calendar year 1 year prior to the performance period through the first 8 months of the 
performance period in the next calendar year.  81 Fed. Reg. 77008, 77065 (Nov. 4, 2016). 
15 Providers who are exempt from MIPS and are not Qualifying APM Participants will continue to be paid for 
Medicare Part B services at the Physician Fee Schedule rates without an additional QPP payment adjustment. 
16 CMS, Quality Payment Program: Pick Your Pace in MIPS.  Accessed at https://qpp.cms.gov/ on September 14, 
2017. 
17 MACRA,  101(e)(2)). 
18 CMS, 2017 MIPS Quality Performance Category Fact Sheet.  Accessed at 
https://www.cms.gov/Medicare/Quality-Payment-Program/Resource-Library/2017-MIPS-Quality-Performance-
Category-Fact-Sheet.pdf on November 14, 2017. 
 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               28   
19 CMS, Advancing Care Information Performance Category Fact Sheet. Accessed at 
https://www.cms.gov/Medicare/Quality-Payment-Program/Resource-Library/Advancing-Care-Information-
Performance-Category-Fact-Sheet.pdf on November 14, 2017. 
20 81 Fed. Reg. 77008, 7781777831 (Nov. 4, 2016). 
21 Each Advanced APM program has its own established processes for collecting and reporting quality data.  
Qualifying APM Participants need only to continue following the regular quality-reporting process established by 
their particular APM program. 
22 Examples include the QPP-Small, Underserved, and Rural Support; Quality Innovation Network-Quality 
Improvement Organization; and Transforming Clinical Practice Initiative contract programs. 
23 CMS. Agile Development for Merit-Based Incentive System (MIPS), Solicitation HHSM-500-2016-RFP-0037, 
Statement of Objectives.  Accessed at 
https://www.fbo.gov/index?s=opportunity&mode=form&id=a1784e8e8067c0b6b648074ccb8d8d31&tab=core&_
cview=1 on October 12, 2016. 
24 Previous GAO and OIG reports have found problems with inaccurate and outdated PECOS data; CMS is 
continuing to make improvements to the system.  See GAO, Initial Results of Revised Process to Screen Providers 
and Suppliers, and Need for Objectives and Performance Measures, GAO-17-42, November 2016; GAO, Additional 
Actions Needed to Improve Eligibility Verification of Providers and Suppliers, GAO-15-448, June 2015; and OIG, 
Enhanced Enrollment Screening of Medicare Providers:  Early Implementation Results, OEI-03-13-00050, 
April 2016.  
25 MACRA,  101(c)(1)(C)(ii)), SSA  1848(q)(2)(A), 81 Fed. Reg. 77008 (Nov. 4, 2016).  
26 MACRA  101(c)(1), SSA  1848(q)(1)(C)(ii), 81 Fed. Reg. 77008, 7706277070 (Nov. 4, 2016). 
27 CMS has clarified that the preliminary MIPS score provided at the time of data submission does not represent a 
guarantee of each clinicians final MIPS score. See CMS, QPP Submissions API Documentation - Scoring Developer 
Tools, available at https://qpp.cms.gov/developers.   
28 For example, CMS has partnered with CAPG (formerly known as the California Association of Physician Groups), 
a nationwide association that represents medical group practices focused on capitated, coordinated care.  
29 81 Fed. Reg. 77008, 7735877362 (Nov. 4, 2016). 
30 81 Fed. Reg. 77008, 7748977491 (Nov. 4, 2016). 
31 GAOs framework outlines four phases for managing fraud risk: (1) Commit to combating fraud by creating an 
organizational culture and structure conducive to fraud risk management; (2) Plan regular fraud risk assessments 
and assess risks to determine a fraud risk profile; (3) Design and implement strategy with specific control activities 
to mitigate assessed fraud risk and collaborate to help ensure effective implementation; and (4) Evaluate 
outcomes using a risk-based approach and adapt activities to improve fraud risk management. GAO, 
A Framework for Managing Fraud Risks in Federal Programs, July 2015.  Accessed at 
https://www.gao.gov/products/GAO-15-593SP on September 14, 2017.   
32 CMS, Quality Payment Program 2017 MIPS Data Validation, August 2017.  Accessed at 
https://qpp.cms.gov/about/resource-library on September 18, 2017. 
33 OIG, Medicare Paid Hundreds of Millions in Electronic Health Record Incentive Payments that Did Not Comply 
With Federal Requirements, June 2017.  Accessed at https://oig.hhs.gov/oas/reports/region5/51400047.asp on 
September 14, 2017. 
34 81 Fed. Reg. 77008 (Nov. 4, 2016). 
Followup: CMSs Management of the Quality Payment Program (OEI-12-17-00350)                               29