O I G   
FFICE OF  NSPECTOR  ENERAL
http://oig.hhs.gov 
 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, as amended, is to protect 
the integrity of the Department of Health and Human Services (HHS) programs, as well as the health and welfare of 
beneficiaries served by those programs.  This statutory mission is carried out through a nationwide network of 
audits, investigations, and inspections conducted by the following operating components: 
OFFICE OF AUDIT SERVICES 
The Office of Audit Services (OAS) provides auditing services for HHS, either by conducting audits with its own audit 
resources or by overseeing audit work done by others.  Audits examine the performance of HHS programs and/or its 
grantees and contractors in carrying out their respective responsibilities and are intended to provide independent 
assessments of HHS programs and operations.  These assessments help reduce waste, abuse, and mismanagement 
and promote economy and efficiency throughout HHS. 
OFFICE OF EVALUATION AND INSPECTIONS 
The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide HHS, Congress, and the 
public with timely, useful, and reliable information on significant issues.  These evaluations focus on preventing 
fraud, waste, or abuse and promoting economy, efficiency, and effectiveness of departmental programs.  To 
promote impact, OEI reports also present practical recommendations for improving program operations.  
OFFICE OF INVESTIGATIONS 
The Office of Investigations (OI) conducts criminal, civil, and administrative investigations of fraud and misconduct 
related to HHS programs, operations, and beneficiaries.  With investigators working in all 50 States and the District of 
Columbia, OI utilizes its resources by actively coordinating with the Department of Justice and other Federal, State, 
and local law enforcement authorities.  The investigative efforts of OI often lead to criminal convictions, 
administrative sanctions, and/or civil monetary penalties. 
OFFICE OF COUNSEL TO THE INSPECTOR GENERAL 
The Office of Counsel to the Inspector General (OCIG) provides general legal services to OIG, rendering advice and 
opinions on HHS programs and operations and providing all legal support for OIGs internal operations.  OCIG 
represents OIG in all civil and administrative fraud and abuse cases involving HHS programs, including False Claims 
Act, program exclusion, and civil monetary penalty cases.  In connection with these cases, OCIG also negotiates and 
monitors corporate integrity agreements.  OCIG renders advisory opinions, issues compliance program guidance, 
publishes fraud alerts, and provides other guidance to the health care industry concerning the anti-kickback statute 
and other OIG enforcement authorities. 
    EXECUTIVE SUMMARY
 
HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace     OEI-06-14-00350 
FOREWORD 
This case study examines implementation of HealthCare.gov and the Federal Marketplace by the Centers for  
Medicare & Medicaid Services (CMS), from passage of the Affordable Care Act (ACA) in 2010 through the second open 
enrollment period in 2015.  As required by the ACA, HealthCare.gov is the Federal website that facilitates purchase of 
private health insurance for consumers who reside in States that did not establish health insurance marketplaces.  At 
its launch on October 1, 2013, and for some time after, HealthCare.gov users were met with website outages and 
technical malfunctions.  After corrective action by CMS and contractors following the launch, CMS ended the first open 
enrollment period with 5.4 million individuals having selected a plan through the Federal Marketplace. 
In our oversight role for the Department of Health and Human Services (HHS), the Office of Inspector General (OIG) 
has a significant body of ongoing and planned audits and evaluations regarding the Federal Marketplace and other 
ACA provisions of high interest and concern to HHS, Congress, and other stakeholders.  These include reviews and 
suggestions for improvements related to the accuracy of Federal financial assistance payments; verifications of 
eligibility determinations for insurance, premium tax credits, and cost-sharing reductions; and CMSs management of 
marketplace contracts and the security of personal information.  OIG reports about the marketplaces are available 
online at www.oig.hhs.gov/reports-and-publications/aca/, and additional information about our planned and ongoing 
work is available online in OIG Work Plan documents at http://www.oig.hhs.gov/reports-and-
publications/workplan/index.asp. 
ABOUT THIS CASE STUDY 
The objective of this case study was to gain insight into CMS implementation and management of the Federal 
Marketplace, focusing primarily on HealthCare.gov.  Our review spans 5 years, providing a chronology of events and 
identifying factors that contributed to the websites breakdown at launch, its recovery following corrective action, and 
implementation of the Federal Marketplace through the second open 
enrollment period.  OIG calls on CMS to address identified problems  WHATS INSIDE THIS REPORT 
and employ lessons learned from management of this project to avoid 
future problems with program implementation and to further    Detailed chronology of CMSs 
strengthen CMS.  In conducting this review, we interviewed 86 current  development and implementation of 
and former HHS and CMS officials, staff, and contractors involved with  the Federal Marketplace and 
the development and management of the website.  We also reviewed  HealthCare.gov in four chapters: 
thousands of HHS and CMS documents, including management reports,    Preparation & Development 
internal correspondence, and website development contracts. 
  Final Countdown to Launch 
WHAT WE FOUND     Launch, Correction, & First Open 
The development of HealthCare.gov faced a high risk of failure, given  Enrollment 
the technical complexity required, the fixed deadline, and a high degree    Turnaround & Second Open 
of uncertainty about mission, scope, and funding.  Still, we found that  Enrollment 
HHS and CMS made many missteps throughout development and 
  Key Contributing Factors to the 
implementation that led to the poor launch.  Most critical was the 
website breakdown and recovery 
absence of clear leadership, which caused delays in decisionmaking, 
lack of clarity in project tasks, and the inability of CMS to recognize the    Call for continued progress with  
magnitude of problems as the project deteriorated.  Additional HHS and  Lessons Learned for ongoing CMS 
CMS missteps included devoting too much time to developing policy,  management of HealthCare.gov 
which left too little time for developing the website; making poor 
  Glossary of key terms 
technical decisions; and failing to properly manage its key website 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace          i  EXECUTIVE SUMMARY 
development contract.  CMSs organizational structure and culture also hampered progress, including poor 
coordination between policy and technical work, resistance to communicating and heeding warnings of bad news, 
and reluctance to alter plans in the face of problems.  CMS continued on a failing path to developing HealthCare.gov 
despite signs of trouble, making rushed corrections shortly before the launch that proved insufficient.  These 
structural, cultural, and tactical deficiencies were particularly problematic for HealthCare.gov given the significant 
challenges of implementing a new program involving multiple stakeholders and a large technology build. 
Following the launch on October 1, 2013, CMS and contractors pivoted quickly to corrective action, reorganizing the 
work to focus on key priorities and to improve execution.  This required significant and focused effort to measure 
website performance, correct problems with website capacity and functions, and establish a new project structure.  
Key factors that contributed to recovery of the website included CMS adopting a badgeless culture for the project, 
wherein all CMS staff and contractors worked together as a team, and a practice of ruthless prioritization that 
aligned work efforts with the most important and achievable goals.  CMS recovered the HealthCare.gov website for 
high consumer use within 2 months, and adopted more effective organizational practices, such as closer integration of 
policy and technical functions, developing redundancies in anticipation of problems, and flexibility in learning from and 
modifying processes. 
CALL FOR CONTINUED PROGRESS 
CMS continues to face challenges in implementing the Federal Marketplace, and in improving operations and services 
provided through HealthCare.gov.  As of February 1, 2016, CMS reported that over 9.6 million consumers had selected 
a health plan through the Federal Marketplace or had their coverage automatically renewed.  As CMS moves forward, 
challenges include completing the automated financial management system and continuing to address areas OIG has 
identified in past reports as problematic or needing improvement.  The agency has focused on this project for years 
and now must keep attuned to these challenges as it shifts focus to other work. 
OIG calls on CMS to continue progress in applying lessons learned from HealthCare.gov to avoid future problems and 
to maintain improvement across the agency.  These lessons comprise core management principles that address both 
specific project challenges and organizational structure, and could apply to other organizations.  CMS concurred with 
OIGs call for continued progress, stating that it will continue to employ the lessons below and that, since OIGs review, 
it has implemented several initiatives to further improve its management. 
LESSONS LEARNED   
   
Leadership  Communication 
Assign clear project leadership to provide cohesion across  Promote acceptance of bad news and encourage staff to 
tasks and a comprehensive view of progress.  identify and communicate problems. 
   
Alignment  Execution 
Align project and organizational strategies with the  Design clear strategies for disciplined execution, and 
resources and expertise available.  continually measure progress. 
   
Culture  Oversight 
Identify and address factors of organizational culture that  Ensure effectiveness of IT contracts by promoting 
may affect project success.  innovation, integration, and rigorous oversight. 
   
Simplification  Planning 
Seek to simplify processes, particularly for projects with a  Develop contingency plans that are quickly actionable, 
high risk of failure.  such as redundant and scalable systems. 
   
Integration  Learning 
Integrate policy and technological work to promote  Promote continuous learning to allow for flexibility and 
operational awareness.  changing course quickly when needed. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace     ii  KEY CONTRIBUTING FACTORS TO BREAKDOWN
 
 
PREPARATION & DEVELOPMENT 
March 2010December 2012 
Policy Development Delays 
Initial work to create the Federal Marketplace required extensive policy development 
that delayed HHS and CMS in planning for the technical and operational needs of the 
HealthCare.gov website. 
Poor Transition to CMS 
A poor transition of the Federal Marketplace from HHS to CMS early on caused 
inefficiencies that resulted in communication breakdowns and needlessly complex 
implementation.  
Lack of Clear Leadership 
HealthCare.gov lacked clear project leadership to give direction and unity of purpose, 
responsiveness in execution, and a comprehensive view of progress. 
Mismanagement of Key Contract 
CMS mismanaged the key website development contract, with frequent changes, 
problematic technological decisions, and limited oversight of contractor performance. 
 
FINAL COUNTDOWN TO LAUNCH 
January 2013September 2013 
Compressed Timeline for Technical Build 
CMS continued to change policy and business requirements, which compressed the 
timeframe for completing the websites technical development. 
Resistance to Bad News  
CMS leaders and staff failed to recognize the magnitude of problems, became 
resistant to bad news about the websites development, and failed to act on warnings 
and address problems. 
Path Dependency 
As problems worsened, CMS staff and contractors became path dependent, 
continuing to follow the same plan and schedule rather than change course as 
circumstances warranted. 
Corrections Weak and Late 
CMS attempted last-minute corrections that were weak and too late to effect change, 
retaining a fixed deadline for launch, despite poor progress. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace          iii  KEY CONTRIBUTING FACTORS TO RECOVERY
 
 
LAUNCH, CORRECTION, & FIRST OPEN ENROLLMENT 
October 2013March 2014 
Quick Pivot to New Strategy 
CMS and its contractors began correction of website problems immediately following 
launch, making a quick pivot to change their strategy. 
Adoption of Badgeless Culture 
CMS and its contractors adopted a badgeless culture that encouraged full 
collaboration by CMS staff and contractors regardless of employer status and job title, 
fostering innovation, problem solving, and communication among teams. 
Integration of All Functions 
CMS integrated all functions into its organizational structure to align with project 
needs, enhancing CMS and contractor accountability and collaboration. 
Planning for Problems 
CMS planned for problems, establishing redundant (backup) systems in the event of 
further breakdowns, and restructuring its key development contract to ensure better 
performance. 
 
TURNAROUND & SECOND OPEN ENROLLMENT 
April 2014February 2015 
Ruthless Prioritization 
CMS adopted a policy of ruthless prioritization to reduce planned website 
functionality, focusing resources on the highest priorities. 
Quality Over On-Time Delivery 
CMS prioritized quality over on-time delivery, employing extensive testing to identify 
and fix problems and delaying new website functionality if unready for perfect 
execution. 
Simplifying Processes 
CMS simplified systems and processes to enable closer monitoring of progress, 
increased transparency and accountability, and clearer prioritization. 
Continuous Learning 
CMS adopted continuous learning for policy and technological tasks, balancing project 
plans with system and team capacity, and changing course as needed to improve 
 
operations.
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace          iv  TABLE OF CONTENTS
 
EXECUTIVE SUMMARY ........................................................................................................................................................ i 
KEY CONTRIBUTING FACTORS TO BREAKDOWN ............................................................................................................... iii 
KEY CONTRIBUTING FACTORS TO RECOVERY ................................................................................................................... iv 
BACKGROUND .................................................................................................................................................................... 1 
CHRONOLOGY CHAPTERS 
Preparation & Development:  March 2010December 2012  
HHS initially housed the marketplace project in a new office that made early gains, but was hindered  
by limited resources and competing expectations ................................................................................................ 4 
Integrating into a large organizational structure at CMS brought new challenges to the Federal  
Marketplace project, primarily caused by unclear project leadership ................................................................... 7 
CMS struggled to provide timely guidance to States, secure future funding for the development of  
HealthCare.gov, and retain key staff for the project ........................................................................................... 11 
IT contracting for the FFM encountered significant problems, including limited bids, uncertainty in  
funding, and disjointed CMS contract management ........................................................................................... 13 
Continued insufficient coordination and direction by CMS led to delays in guidance to contractors,  
and set the stage for problems with HealthCare.gov operations ........................................................................ 15 
Key Factors Contributing to Breakdown:  Preparation and Development ........................................................... 20 
Final Countdown to Launch:  January 2013September 2013  
Several entities voiced concerns about the status of HealthCare.gov, but warnings were either not  
fully communicated or not acted upon ............................................................................................................... 21 
The HealthCare.gov website build was alarmingly behind schedule, with CMS scrambling for minimal 
functionality ....................................................................................................................................................... 24 
Last minute attempts to correct and avoid further problems with HealthCare.gov were ad hoc and  
insufficient ........................................................................................................................................................... 26 
CMS prepared to launch HealthCare.gov on October 1, 2013, as planned, optimistic in spite of  
problems and never seriously considering delay ................................................................................................. 28 
Key Factors Contributing to Breakdown:  Final Countdown to Launch ................................................................ 31 
Launch, Correction, & First Open Enrollment:  October 2013March 2014  
The HealthCare.gov launch quickly revealed multiple problems with the website, and initial efforts to  
fix the problems were hampered by lack of coordination ................................................................................... 32 
After initial difficulties, CMS and contractors worked with outside experts to repair HealthCare.gov,  
instilling changes in the project culture and work processes .............................................................................. 34 
CMS and the expanded technical team improved HealthCare.gov by December 1, 2013, and continued 
improvements through early 2014 ...................................................................................................................... 39 
  TABLE OF CONTENTS 
The Federal Marketplace project and development of HealthCare.gov came at a substantial cost  
financially and organizationally to CMS ............................................................................................................... 42 
Key Factors Contributing to Recovery:  Launch, Correction, & First Open Enrollment ........................................ 45 
Turnaround & Second Open Enrollment: April 2014February 2015  
In April 2014, CMS turned to preparing for the next open enrollment and added functionality,  
sharply prioritizing to limit scope and focus resources ........................................................................................ 46 
Changes made by CMS during preparation for second open enrollment also focused on longer-term 
improvements to the Federal Marketplace and throughout CMS ....................................................................... 52 
Challenges remain for CMS in operating HealthCare.gov, with public scrutiny high and website and  
other automated functions not yet complete ..................................................................................................... 57 
Key Factors Contributing to Recovery:  Turnaround & Second Open Enrollment ............................................... 60 
CALL FOR CONTINUED PROGRESS 
OIG calls for CMS to continue applying lessons from the HealthCare.gov recovery in its management  
of the Federal Marketplace and broader organization  ....................................................................................... 61 
AGENCY COMMENTS AND OIG RESPONSE ..................................................................................................................... 66 
APPENDIXES 
A.  List of Key Entities in Implementation and Operation of the Federal Marketplace, 20102015 .................... 67 
B.  Timeline of Key Events in CMS Implementation of the Federal Marketplace, 20102015 ............................. 68 
C.  Glossary of Selected Federal Marketplace Terms ........................................................................................... 70 
D.  Comments from the Centers for Medicare & Medicaid Services, October 7, 2015 ....................................... 75 
ACKNOWLEDGEMENTS ..................................................................................................................................................... 77 
ENDNOTES ........................................................................................................................................................................ 78
  BACKGROUND
 
OBJECTIVE 
To evaluate the Department of Health and Human Services (HHS or the Department) and Centers for Medicare & 
Medicaid Services (CMS or the agency) implementation and management of the Federal Marketplace, focused 
primarily on the development and operation of its website, HealthCare.gov. 
PATIENT PROTECTION AND AFFORDABLE CARE ACT 
The Patient Protection and Affordable Care Act (ACA) was signed into law on March 23, 2010.1  The ACA expanded 
access to health insurance coverage by enacting insurance reforms, requiring many businesses to offer health 
insurance coverage, and requiring most individuals to obtain coverage.  Generally, those who do not comply must pay 
a penalty.2  The ACA also required, and provided Federal funding for, the establishment of a health insurance exchange 
(marketplace) in each State that would be operational on or before January 1, 2014.3  For States that elected not to 
establish their own marketplaces, the Federal Government was required to operate a marketplace on behalf of the 
State.4  The marketplaces provide those seeking health insurance a single point of access to view qualified health plan 
(health plan)5 options, determine eligibility for coverage, and purchase insurance coverage.  Individuals also use the 
marketplaces to determine eligibility for insurance affordability programs (e.g., Medicaid, premium tax credits, and 
cost-sharing reductions) that lower insurance premiums and costs of care.6  At the beginning of the third open 
enrollment period, November 1, 2015, the Federal Government operates a marketplace (Federal Marketplace) for 
38 States, including 7 State-partnership marketplaces for which HHS and the State share responsibilities for core 
functions and 4 Federally supported State marketplaces in which States perform most marketplace functions.7  
Thirteen States (including the District of Columbia) operate their own marketplaces.8 
The ACA required the Secretary of HHS to specify an initial open enrollment and annual open enrollment periods each 
subsequent year during which individuals may enroll in a health plan.9  The first open enrollment period was 6 months 
in duration, lasting October 1, 2013March 31, 2014.10  The second open enrollment period was 3 months in duration, 
lasting November 15, 2014February 15, 2015.11  Special enrollment periods (SEP) allow consumers who experience 
certain life changes or other circumstances to purchase insurance outside of open enrollment,12 and CMS has several 
times offered SEPs to provide other consumers additional time to purchase plans when situations beyond their control 
limited their ability to select a plan during open enrollment.13 
After several challenges to the ACA, the Supreme Court heard two cases about the constitutionality of certain 
provisions of the Act.  In June 2012, the Court upheld the mandate that most individuals must have health insurance, 
but ruled unconstitutional the requirement that States expand their Medicaid programs.14  The Court ruled in 
June 2015 that the ACA provides premium tax credits for insurance purchased through all marketplaces, Federal and 
State.15  Several Federal court challenges to the ACA are pending. 
CENTERS FOR MEDICARE & MEDICAID SERVICES 
CMS, an agency within HHS, has had responsibility for managing the marketplace programs since January 2011.16  CMS 
manages more than 85 percent of HHSs $1.2 trillion budget, primarily for operation of the Medicare and Medicaid 
programs.17  To implement the ACA provisions related to the marketplaces, CMS worked in collaboration with public 
and private entities, including other Federal agencies as required by the ACA,18 State Medicaid agencies, private 
contractors, health insurance issuers (issuers), and not-for-profit organizations. 
CMS has core responsibility for operation of the Federal Marketplace.  In this role, CMS must ensure accurate eligibility 
determinations, process enrollments, facilitate Medicaid enrollment for those who qualify, and communicate timely 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           1  BACKGROUND 
and accurate information to issuers and consumers.  CMS also provides support functions for the State marketplaces 
and administers Federal financial assistance and premium stabilization programs related to the marketplaces.  See 
Appendix A for a list of the referenced HHS and CMS divisions involved in the Federal Marketplace. 
HEALTHCARE.GOV 
HealthCare.gov is the public website for the Federal Marketplace through which individuals can browse health 
insurance plans, enroll in coverage plans, and apply for Federal financial assistance to help cover the premium and 
other costs.  The Federal Marketplace links consumers from HealthCare.gov to multiple supporting systems that 
facilitate the enrollment process and payment to issuers.  For purposes of this report, key components of 
HealthCare.gov and the Federal Marketplace include: 
  Enterprise Identity Management (EIDM) system, which was used during the first and second open 
enrollment periods to enable consumers to create accounts and verify their identities.  The EIDM was 
developed to support multiple CMS systems. 
  Federally-facilitated Marketplace (FFM), the core of the overall Federal Marketplace system, which includes 
three main subcomponents to facilitate various aspects of acquiring health insurance: 
o  Eligibility and Enrollment determines consumer eligibility for health plans and Federal financial 
assistance and manages enrollment transactions with issuers, 
o  Plan Management coordinates with issuers to determine coverage specifics, and 
o  Financial Management tracks effectuated enrollments (wherein the consumer has selected a plan and 
also paid the premium), and manages payments to issuers for Federal financial assistance (premium 
tax credits and cost-sharing reductions) and premium stabilization. 
  Data Services Hub (Hub) routes information requests from the Federal and State marketplaces and Medicaid/ 
Childrens Health Insurance Program (CHIP) agencies to other Federal agencies and back, such as to and from 
the Internal Revenue Service (IRS) and the Social Security Administration (SSA). 
SCOPE 
This case study evaluates HHS and CMS implementation and management of the Federal Marketplace, primarily the 
website HealthCare.gov.  Our review is limited to the actions of HHS and CMS personnel and divisions and their 
contractors, spanning from passage of the ACA in March 2010 through the end of the Federal Marketplace second 
open enrollment period in February 2015.  See Appendix B for a timeline of key implementation dates. 
METHODOLOGY 
To evaluate HHS and CMS management of HealthCare.gov, we based our review on analysis of data from three 
sources: 
  Interviews with officials and staff from HHS, CMS, contractors, and other stakeholders:  We conducted 
interviews with 86 respondents, individually or in small groups, regarding their roles and involvement during 
the implementation of the Federal Marketplace, the strategy for development, factors that contributed to the 
website problems, and actions taken to address those problems.  We present interview data in the report in 
both aggregate analysis and individual quotations. 
o  HHS senior leadership at the time of the HealthCare.gov launchrespondents included the Secretary, 
Deputy Secretary, Assistant Secretary for Administration, Assistant Secretary for Financial Resources, 
Senior Counselor, Chief Information Officer (CIO), and Chief Technology Officer (CTO). 
o  CMS senior leadership at the time of the HealthCare.gov launch and afterrespondents included the 
Administrator, Principal Deputy Administrator and Acting Administrator, Chief of Staff, Chief Operating 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           2  BACKGROUND 
Officer (COO), Chief Financial Officer (CFO), and Deputy Administrators for the Center for Consumer 
Information and Insurance Oversight (CCIIO) and Center for Medicaid and CHIP Services (CMCS). 
o  CMS leadership and CMS staffrespondents included Directors and Deputy Directors for the Office of 
Information Services (OIS) and Office of Acquisition and Grants Management (OAGM), Director for the 
Office of Communications (OC), and Deputy Directors for CCIIO.  We also interviewed key staff such as 
a Regional Administrator managing the Consortium for Medicare Health Plans Operations, and CMS 
Government Task Leaders who were the technical representatives responsible for monitoring 
HealthCare.gov contractors technical progress. 
o  Contractor representativesrespondents included representatives from Accenture Federal Services, 
LLC (Accenture); CGI Federal Services, Inc. (CGI Federal); Quality Software Services, Inc. (QSSI); and 
Terremark Federal Group, Inc. (Terremark). 
o  Other stakeholdersrespondents included a small number of others involved in the HealthCare.gov 
project or in a position to observe the project, including navigators hired to assist consumers in 
selecting plans and research organizations studying the ACA. 
  Documents from HHS and CMS:  We used records management software to search through approximately 
2.5 million project management documents and correspondence.  The documents included presentations, 
memorandums, emails, meeting agendas, status reports, technical requirement documents, and 
documentation exchanged between CMS and other entities, such as contracts and Technical Direction Letters. 
  External documents and witness testimony:  We reviewed independent research and analysis about the 
Federal Marketplace and the implementation of large information technology (IT) projects from other 
Government agencies and independent research organizations.  We also reviewed written and oral testimony 
to Congress by HHS and CMS staff and other stakeholders regarding HealthCare.gov. 
LIMITATIONS 
Although we believe the nature and extent of our review provided a sufficient basis for our findings, we note two 
potential limitations:  (1) we purposively selected respondents at HHS, CMS, contractors, and stakeholders for 
interviews based on our review of HHS and CMS documentation and discussions with experts, but we did not interview 
all persons involved; and (2) CMS provided access to a large number of documents on the basis of search terms and 
parameters provided by OIG.  We reviewed documents selectively on the basis of relevance to our objective as 
determined by OIG. 
RELATED OFFICE OF INSPECTOR GENERAL REPORTS 
This report is one in a series of OIG reports that evaluate the Federal and State marketplaces.  OIG found a number of 
problems with CMSs implementation of the Federal Marketplace, including that CMS did not adequately plan for and 
monitor contracts and that CMS could not verify the accuracy of payments to issuers.  OIG also identified areas for 
improvement in CMS eligibility verification and information security controls.  OIG posts all ACA-related reports on its 
website (www.oig.hhs.gov/reports-and-publications/aca/) and continues its oversight of the marketplaces as 
articulated in the OIG Work Plan (http://oig.hhs.gov/reports-and-publications/archives/workplan/2015/FY15-Work-
Plan.pdf) and the Health Reform Oversight Plan (http://oig.hhs.gov/reports-and-
publications/archives/workplan/2015/health-reform-plan-2015.pdf). 
STANDARDS 
This study was conducted in accordance with the Quality Standards for Inspection and Evaluation issued by the Council 
of the Inspectors General on Integrity and Efficiency. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           3  CHRONOLOGY
 
 
The clock was already ticking. 
Former CCIIO Official 
C 1 
HAPTER 
 
PREPARATION DEVELOPMENT 
 & 
March 2010December 2012 
 
Marketplace as a  The ACAs system for individuals to purchase private health insurance and enroll in Medicaid 
label for the insurance  was a topic of heated debate in Congress, States, and the media before and after the laws 
program evolved over time.  
passage.  Twenty-six States filed suit against the Federal Government regarding various 
Early implementation used 
aspects of the ACA within the first year after passage, and in the 6 years since, the U.S. House 
the term exchange or 
HIX for Health Insurance  of Representatives has taken more than 60 votes to change or repeal the ACA.19  Those in HHS 
Exchange.  responsible for execution of the program reported that these debates and uncertainties over 
the ACAs policies had ramifications throughout development and implementation of the 
Federal Marketplace.  As is often the case with complex legislation, many regulations and 
other guidance documents were needed to implement the ACA.  HHS had significant 
responsibility for implementing marketplace operations in accordance with statutory 
requirements and timeframes.  The ACA provided for significant flexibility in implementation, 
for example, leaving States to decide whether they would operate their own marketplaces or 
participate in a national, Federally-run marketplace.20 
HHS initially housed the marketplace project in a new office that made early gains, 
but was hindered by limited resources and competing expectations 
The first  HHS launched the HealthCare.gov Plan Finder website and established a new office to 
HealthCare.gov   manage the marketplaces.  From the outset, implementing the ACA required HHS to meet 
HealthCare.gov Plan  multiple priorities, including many provisions unrelated to the marketplaces.  A former HHS 
Finder was an early 
official reflected, There were hundreds of things that needed to be done, and the 
browsing website created 
Marketplace was just one.  A number of ACA provisions had early delivery dates for HHS.  
by HHS in 3 months  
and cost approximately   This included the initial HealthCare.gov website, Plan Finder, a browsing website to provide 
$17 million.  health plan information to consumers but without the functionality to purchase plans or apply 
for Federal financial assistance. 
The ACA required that HHS establish the browsing website by July 1, 2010.21  A small team of 
technical experts from HHS and the White House worked around the clock to complete the 
HealthCare.gov Plan Finder website.  The website launched in July 2010 with general plan 
information, and was upgraded in December 2010 with functionality for consumers to enter 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           4  Preparation & Development | March 2010December 2012 
information about themselves and receive an estimate of their health plan premiums.22  HHS 
reported that the website functioned smoothly and received approximately 2 million visitors 
from July 2010 to July 2011.23  HHS and CMS staff later said that the relative ease with which 
HHS built the browsing website may have contributed to underestimation of the resources 
and time required to build the much more complex full HealthCare.gov website planned for 
launch on October 1, 2013. 
HealthCare.gov Plan Finder Homepage, July 1, 2010. 
 
At the same time that HHS was developing the browsing website, it began planning for the 
establishment of the Federal and State marketplaces that would facilitate health plan 
coverage for consumers by January 1, 2014.  In April 2010, HHS created the Office of 
Consumer Information and Insurance Oversight (OCIIO) to oversee implementation of the ACA 
provisions related to private insurance.24  Meanwhile, HHS continued its broader focus on all 
provisions of the law.  OCIIO was to serve as a coordination point between the Department, 
issuers, and other Federal (e.g., IRS, SSA) and State partners, and to begin putting into 
operation the way in which individuals would purchase insurance.  
HHS officials indicated that the private 
insurance aspect led HHS to establish OCIIO  The single most important thing 
as its own staff division in the Office of the 
was the insurance part had to work. 
Secretary, rather than assign the program 
 
Former HHS official
to CMS, its Medicare and Medicaid 
operating division.  HHS staffed OCIIO with 
both long-time Federal employees, many 
from CMS, and those with expertise in the private insurance market, such as former officials of 
State Departments of Insurance.  OCIIO in 2010 had direct hiring authority that allowed 
flexibility to assist HHS in expeditiously filling vacant positions when facing a critical hiring 
need.  OCIIO focused largely on developing and obtaining approval for the many regulations 
required to implement private insurance reforms and establish the Federal and State 
marketplaces.25  It also worked with a contractor to create an initial blueprint of critical tasks.  
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           5  Preparation & Development | March 2010December 2012 
According to interview respondents who worked in OCIIO at that time, the focus was on policy 
development and not yet on operational issues, such as development of the full website 
intended for the 2013 launch. 
OCIIOs chief objectives, according 
to former officials, were to publish 
Of course issuers were going to participate.  
the regulations that laid out how 
It is a huge new market where people are 
the Federal and State 
marketplaces would work, and to  compelled to buy this new product.  There 
coordinate the participation of the  was money to be made.    
Former OCIIO official
partners necessary to make the 
marketplaces work, including 
States, issuers, consumers, other Federal agencies, and private entities.26  Former OCIIO 
officials reported in interviews they were most concerned with ensuring the participation of 
issuers, which were needed to submit health plans for purchase on the marketplaces.  Officials 
worried that issuers would be reluctant to submit plans due to concern about new 
requirements for coverage, the process for marketplaces to approve plans, and uncertainty 
about establishing premium rates not knowing the size of the population to be covered and 
what health care services this new insurance population required.  A former OCIIO official 
later discounted concern over issuer participation, stating, Of course issuers were going to 
participate.  It is a huge new market where people are compelled to buy this new product.  
There was money to be made.  Still, at a minimum, OCIIO was tasked with developing a new 
and complex system that required the collaboration of multiple entities, each with its own 
  incentives and requirements. 
We didnt want   HHS officials had competing predictions for whether marketplaces would be State or Federal.  
people to walk into the  Both HHS and States faced uncertainty regarding whether States would build their own 
store and not have 
marketplaces or default to the Federal Marketplace.  Interviews with former OCIIO officials 
anything on the shelf. 
indicated that the conventional wisdom among leadership at HHS in 2010 was that most 
Former HHS official 
States would choose to establish their own marketplace, with as few as eight States 
participating in the Federal Marketplace.  They reasoned that State leaders would want 
autonomy and to avoid participating in a large Federal program, and that States would be 
enticed by Federal Establishment Grants that provided money to build and operate State 
marketplaces. To build their own marketplace, States had to complete a considerable number 
of tasks, including passing State legislation in some cases.  Interviews and documentation 
indicated that some in HHS focused on encouraging States to build their own systems, while 
others in HHS predicted that it would be too difficult for State Governments to build individual 
marketplaces, politically and operationally, so they would default to the Federal Marketplace.  
HHS officials recalled an expected sense of failure among some States that opted to join the 
Federal Marketplace because they did not think they could pull it off, were too small, and 
didnt have the issuer base to finance the user fees.  In other cases, State legislatures did not 
authorize the creation of a State marketplace.  One HHS official explained that some States 
did not want to touch Obamacare.  (HHS delayed the deadline for States to make this 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           6  Preparation & Development | March 2010December 2012 
decision  several  times,  and  it  was  ultimately  early  2013  before  it  was  clear  which  States  would  
join  the  Federal  Marketplace  for  the  first  open  enrollment  period.27)  
HHS determined that OCIIO was illequipped to manage such a large and complex project.   
Although  OCIIO  provided  HHS  with  a  program  and  staff  focused  on  the  marketplaces,  it  had  a  
relatively  small  number  of  staff  and  lacked  the  infrastructure  and  budget  of  an  operating  
division.   OCIIO  did  not  possess  basic  inhouse  operations,  including  contracting  and  
technological  support.   It  also  had  few  technical  staff,  rendering  it  illequipped,  in  HHSs  view,  
to  manage  such  a  large  project.   CMSs  decadeslong  experience  administering  large  
Government  programs  made  it  the  frontrunner  to  replace  OCIIO  in  managing  the  
marketplaces.   CMS  had  a  large  infrastructure  from  which  the  Federal  Marketplace  program  
could  benefit,  including  technical  and  operational  staff.   Some  in  favor  of  assigning  the  Federal  
Marketplace  operations  to  CMS  wanted  to  employ  existing  expertise  and  infrastructure  rather  
than  build  the  Federal  Marketplace  program  from  scratch,  with  one  CMS  official  arguing  that  
the  Medicare  program  had  much  of  the  same  operational  DNA  that  would  be  needed  for  
the  Federal  Marketplace.   Other  officials  in  HHS  argued  against  the  move  to  CMS,  contending  
that  identification  with  CMS  (as  the  operator  of  Medicare  and  Medicaid)  might  cloud  the  
programs  objective  as  a  market  for  private  insurance.  
In  January  2011,  10  months  after  OCIIOs  inception,  HHS  transitioned  the  marketplace  project  
from  OCIIO  in  the  Office  of  the  Secretary  to  CMS.28   Those  involved  in  the  decision  reported  
that  it  was  driven  largely  by  the  idea  that  CMSs  available  funding  and  substantial  existing  
infrastructure  could  help  support  marketplace  functions.   The  ACA  provided  a  $1  billion  Health  
Insurance  Reform  Implementation  Fund  to  help  pay  for  the  administrative  costs  of  
implementation  of  ACA,  but  HHS  ceded  over  half  of  these  funds  to  the  IRS  and  other  
Departments  to  support  ACA  implementation.29   Given  that  the  Congressional  Budget  Office  
predicted  a  $510  billion  cost  of  implementation  to  HHS  over  the  20102019  period,  HHS  
faced  a  substantial  funding  shortfall.30   In  addition,  several  HHS  officials  perceived  that  
embedding  the  project  in  CMS  may  have  helped  those  responsible  for  developing  the  
marketplaces  to  avoid  distractions  from  ongoing  debate  about  the  ACA.   They  believed  that  
incorporating  the  project  into  the  larger  organization  might  help  to  avoid  lineitem  scrutiny  of  
its  budget  execution  by  critics.  
Integrating  into  a  large  organizational  structure  at  CMS  brought  new  challenges  to  
the  Federal  Marketplace  project,  primarily  caused  by  unclear  project  leadership  
The move to CMS separated marketplace staff into different divisions.   The  transition  to  CMS  
after  10  months  in  the  Office  of  the  Secretary  under  OCIIO  provided  the  expected  benefits  of  
greater  resources,  but  also  brought  drawbacks.   Most  OCIIO  staff  and  leadership  previously  
working  together  were  split  into  two  CMS  divisions:   (1)  policy  and  business  operations  
management  staff  went  to  the  newly  created  CMS  CCIIO,  responsible  for  establishing  Federal  
and  State  marketplace  policies;  and  (2)  technical  and  contract  management  staff  went  to  the  
existing  OIS,  which  coordinated  the  technical  aspects  of  the  HealthCare.gov  website  
development  (website  build).   Other  Federal  Marketplace  responsibilities  were  folded  into  
existing  CMS  divisions.   Most  notably,  the  CMS  Office  of  Acquisition  and  Grants  Management  
OEI061400350             HealthCare.gov:   Case  Study  of  CMS  Management  of  the  Federal  Marketplace            7   Preparation & Development | March 2010December 2012 
(OAGM) was responsible for managing the Federal Marketplace contracts, the Office of 
Financial Management (OFM) was responsible for the Federal Marketplace budget, and the 
Office of Communications (OC) was responsible for interaction with consumers.  CMSs Center 
for Medicaid and CHIP Services (CMCS) also played an important role in coordinating the 
application form that marketplace consumers would use to apply for assistance.  The 
application would enable consumers to apply for:  coverage in a health plan, advance 
payments of the premium tax credit, cost-sharing reductions, Medicaid, and CHIP.  Given the 
shared application form, CMCS had much to gain from a successful Federal Marketplace that 
effectively facilitated Medicaid enrollment. 
In addition, key Federal Marketplace staff were located in offices apart from either CMS or 
HHS headquarters, which contributed to communication problems and a sense of 
separateness.  The office space for both CCIIO and OIS staff assigned to the marketplaces 
remained in the existing OCIIO space in Bethesda, MD, a 45-minute drive from the main CMS 
campus in Baltimore, MD.  The decision to stay in the existing location rather than move to 
CMSs main campus was attributed to the desire to retain former private sector employees 
who lived in the greater Washington, DC, area and were unwilling to travel to suburban 
Baltimore.  Some CMS staff reported that the separate location in Bethesda may have 
exacerbated problems communicating with other CMS divisions, particularly in the case of OIS 
technical development staff located in Bethesda but reporting to the larger OIS organization in 
Baltimore.  The use of multiple locations in the Washington, DC metropolitan area was not 
new for CMS, with its agency headquarters in several separate buildings in Baltimore and 
senior officials in Washington, DC (in addition to substantial staff in 10 regional offices 
throughout the country).  Many rooms were equipped for video conferencing, and staff had 
access to daily shuttles between locations and remote access to computer systems.  Still, even 
with these technical capabilities, the Bethesda location for CCIIO, particularly in the critical 
first year following integration, appears to have contributed to a sense of separateness 
  between the new program and the larger agency. 
There were two   More important than integration of CCIIO staff within the larger CMS organization was a lack 
different conversations  of unity among those responsible for pieces of the marketplace development.  Marketplace 
going on and they were 
staff in CCIIO, OIS and the other divisions had different operating procedures, reporting 
not married up. 
structures, and lines of authority.  Project management documents indicated efforts 
CMS official 
throughout 20112012 to bridge these gaps, such as regular meetings between project 
leaders in the various divisions.  These documents also revealed major differences in 
understanding between the divisions regarding shared responsibilities and assessments of 
project progress.  In interviews, staff in CCIIO and OIS gave very different descriptions of each 
others tasks and in some cases could not identify the staff positions or subdivisions 
responsible for critical project tasks.  This lack of unity and dispersion of responsibility had 
serious consequences, resulting in difficulty tracking progress and enforcing accountability. 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           8  Preparation & Development | March 2010December 2012 
Merging a start-up organization with a large bureaucracy created tension.  The integration 
of the Federal Marketplace project into CMS provided a broader infrastructure for the 
program, but some marketplace officials reported that they struggled to fit into the CMS 
organizational culture.  One explanation offered by several officials was that the marketplace 
program and development of the HealthCare.gov website required a start-up mentality that 
encouraged creativity and innovation to support something new and unique.  Yet the CMS 
organizational culture was that of a more traditional government bureaucracy, based on rigid 
management methods and an established hierarchy.  In their view, those working on the 
marketplace program had to straddle the two cultures and this exacerbated difficulties 
meeting the already-challenging schedule and 
tasks.  As one long-time CMS policy official serving 
We were never fully 
in CCIIO reflected, We were never fully accepted 
accepted by CMS as a whole.   
by CMS as a whole.  Every new program feels that 
 
way, but this was a special case.  The objective was  CCIIO official
too different and not well understood [at CMS].  
Transition to CMS heightened differences regarding program mission.  The uncertainty about 
whether States would choose to operate their own marketplaces or join the Federal 
Marketplace continued after the Federal Marketplace project was transitioned into CMS.  The 
lines in this debate were drawn largely between long-time CMS staff who were in favor of 
building a large-capacity Federal Marketplace, and newer staff with State or private insurance 
experience who were in favor of encouraging States to establish their own systems.  Given 
that the marketplace program had already used almost a year of scarce implementation time, 
resolving these conceptual differences took on a new urgency.  CMS had yet to develop and 
publish a large number of pressing regulations, some of which were tri-departmental 
regulations with the Departments of Labor and Treasury and therefore potentially faced more 
hurdles in coordination.31 
CMS had also not begun planning in earnest for how the Federal Marketplace would operate.  
The debate about the role and need for State and Federal marketplaces played out in issues 
across the policy spectrum, affecting decisions about health plan requirements, benefits, and 
financial assistance.  Some of these differences involved ACA provisions about consumer 
protections that were not specific to the Federal Marketplace and HealthCare.gov, such as 
premium rate reviews and essential health benefits.32  Several CMS officials indicated that 
disagreement about these issues complicated decisionmaking about the Federal Marketplace, 
and may have contributed to delays in decisionmaking.  One CCIIO official from a State 
background complained that these ancillary issues caused CMS staff sometimes to lose focus 
on the key Federal Marketplace mission to facilitate buying and selling of insurance:  Weve 
built up the marketplace to be something grander than it is.  Its a store.  
Another point of contention between some long-time CMS staff and newer employees was 
that CMS staff tended to rely on past experience implementing Government health care 
programs, such as the launch of the Medicare Part D prescription drug program in 2006.  The 
Part D program was a large project that required collaboration with private issuers to create 
competition among plans, so it is reasonable that the launch of the Part D program provided 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           9  Preparation & Development | March 2010December 2012 
CMS staff with relevant expertise and experience.  Also, the Part D program had problems at 
launch that required quick correction.33  Still, creation of the Federal Marketplace was a larger 
and more complex project.  Further, prior programs, such as Part D and Medicare Advantage, 
were built for the existing Medicare population, rather than a new market.  Comparisons of 
the marketplaces to existing CMS programs were perceived by those new to CMS as 
demonstrating a lack of understanding about the larger scope and complexity of the Federal 
Marketplace program.  As one CMS official reflected, I learned to never bring up the Part D 
example. 
Who was in charge?  Lack of leadership caused problems integrating Federal Marketplace staff into CMS.  CMS 
When asked who led the  interviews and documentation indicated that the chief cause of organizational problems was 
Federal Marketplace 
CMSs failure to complete two critical tasks:  assign clear and dedicated leadership for the 
project, several staff 
marketplace program, and fully assess project needs to determine how to best establish the 
named positions that were 
marketplace program in CMS.  In CMS, there was no single official early on, below the CMS 
unfilled.  When asked 
what division led the  Administrator, responsible for the Federal Marketplace, and the officials responsible for 
project, staff named four  different pieces often reported to the CMS Administrator through several layers of 
different CMS divisions.  management.  Throughout this early development period and into implementation and the 
first open enrollment period, CMS senior leadership never declared a clear business owner 
with overall responsibility for the Federal Marketplace.  This issue of leadership was more 
straightforward for OCIIO as a small organization:  the Director of OCIIO reported directly to 
the HHS Secretary, whereas in CMS those active in planning for the marketplaces were spread 
throughout the organization and reported through the CMS Administrator. 
The marketplace program was a complex task for an organization that was already responsible 
for implementation of many other ACA provisions and managing Medicare and Medicaid, 
programs with combined expenditures exceeding $900 billion in fiscal year 2014about  
one-third of total U.S. health care expenditures.34  Adding to this, the CMS Administrator and 
the second-in-command Principal Deputy Administrator were both new to CMS, joining CMS 
in 2010 and 2011, respectively, in the first year of CMSs enormous tasks to implement the 
ACA.35  To undertake the project without assigning leadership was to underestimate the 
projects difficulty.  Lack of clear and effective leadership and thoughtful consideration of 
project and organizational structure characterized the preparation and development period, 
and had long-term negative consequences for the programs success.  This insight was not 
only true in retrospect.  In late 2011, the White House CIO expressed concern to HHS and CMS 
officials that leadership for the program was fragmented and recommended appointing a 
Chief Executive Officer (CEO) to oversee the project. 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           10  Preparation  &  Development | March 2010December 2012  
CMS struggled to  provide timely guidance  to States, secure  future  funding  for  the  
development of  HealthCare.gov, and retain key staff  for the project  
Lack  of clear direction and continued  uncertainty  delayed guidance to States.   Several State 
officials criticized CMS for being slow to  provide regulations and guidance for developing State  
marketplaces.   The ACA  required CMS to provide criteria for State marketplaces and required  
States to demonstrate their readiness to operate on the basis  of these  criteria.36   Pieces of the 
regulations and guidance were due throughout  2011 and 2012, but most were  delayed 
several times.  In October 2012, a State Governor wrote to the Secretary explaining that the 
delay in guidelines contributed to the State deciding to join the Federal Marketplace.   There 
is simply  not enough valid information to make an informed choice for  such an important  
decision.37   Around the same time, a State insurance commissioner  testified before Congress  
that  the lack  of detailed information  from HHS  has  put [us] in a very difficult position.38   In  
fact,  CMS did not finalize  its  regulations for  State  marketplace oversight and  program integrity  
standards  until October 2013,39  10  months after the  State deadline to submit plans  for a State  
marketplace.40  
CMS officials indicated that the delays were caused, in  part,  by efforts to provide greater  
flexibility for  States to establish their  own  marketplaces or to use the Federal Marketplace in a  
way that suited their  needs.  According to CMS staff, when States  objected to a  policy, CMS  
sometimes revised the policy to  provide additional flexibility.  This in turn  required changes to  
the business requirements  (provisions that articulate to developers the processes  the 
website  would support),  and potential delays in  software development.   CMS officials did  not 
have a good sense of which or the  number  of  States that would build their own marketplaces,  
causing consternation about scale, budget, and coordination with States.  As observed by an  
HHS official,  Whether  1 State or 50, we knew we had  to build the functionality, but the scale  
   shifted many times by an  order of magnitude and made it more complicated.  
CMS was convinced    Political  context and funding uncertainties also  affected CMSs  development of  the  Federal  
that [we] had to build  the  Marketplace program.  CMS officials and  staff  reported that they felt the  political importance  
perfect policy and did not  
of the  ACA throughout marketplace implementation.   CMS officials described White House  
leave time to implement.  
staff as  being substantially involved throughout  policy  development and as the clear  policy  
CMS  Administrator  
leads.   A Deputy Director at CCIIO reported,  There was constant contact with the  White  
House.  The White House was in charge.   It appears that for  high-level CMS officials the  
interaction was expected,  but lower-level CMS staff were unaccustomed to working so closely  
with  White House  staff  and the lack of clarity and experience around this process led to some  
confusion and delays.  Specifically,  some CMS  staff expressed frustration that the  close  
involvement of White House staff and  HHS  officials  resulted in a  complex process for making  
decisions and caused delays in completing policy work.  They were  particularly  frustrated  
when they  perceived heavy involvement about what they believed were relatively small  
issues.  For example, emails between  HHS  and CMS staff  revealed  CMS staff frustration  with  
the discussion around changing the term nationwide  health insurance to health insurance  
in official documents.  Others  defended the high level of involvement given the  importance of  
the ACA  to the White House.  The CMS Administrator  also reported perceiving the White 
House involvement as collaborative and helpful, particularly in  making policy decisions.  Other  
OEI-06-14-00350             HealthCare.gov:   Case Study of  CMS Management of the Federal Marketplace            11   Preparation & Development | March 2010December 2012 
high-level CMS officials agreed, indicating that most understood the strong interest in details.  
As one CMS official noted, This was the Presidents achievement.  It raised the stakes.  It 
meant that people at my level had a lot of bosses.  
Others at CMS identified occasional challenges in meeting White House and HHS direction 
during the development and implementation of the Federal Marketplace.  For example, 
according to some CMS policy officials, White House staff and HHS officials expressed concern 
that planning and educational documents for key stakeholders might be overly complex and 
could discourage participation.  CMS, CCIIO, and OIS staff reported that a White House policy 
official requested that some policy and technical documents be simplified or not used.  As an 
example of these requests, CCIIO produced a Federal Marketplace concept of operations 
document in 2011 to educate States and issuers about Federal Marketplace operations that 
was perceived by CCIIO staff as critical to these stakeholders understanding the planned 
operations, but the document was not distributed as planned.   
Another complication stemming in part from the political contention surrounding the ACA was 
lack of certainty regarding future Federal Marketplace funding.  Although initial resources 
were improved with the move to CMS, it was unknown to HHS and CMS officials how much 
funding Congress would provide for development in future budgets.  CMS officials reported 
that this uncertainty about future funding for implementing the Federal Marketplace and 
other ACA programs made decisionmaking more difficult, particularly determining how to 
prioritize different aspects of the website build and provide overall project direction.  CMS 
officials also reported that this uncertainty delayed establishing contracts and moving forward 
with the technical build.  As one CMS contracting official explained, I cannot put [a contract] 
on the street without [funding]. . . .  We didnt know when we were going to get [funding] or 
what we can use it for.  HHS officials expected user fees to make Federal Marketplace largely 
self-sustaining by the end of 2015, but ensuring that funding was available for the build was 
cause for concern.  One HHS official reported that CMS completed a budget in mid-2012 that 
showed a sizeable gap between the amount of money forecasted to complete the technical 
  build and the amount available for use. 
Every time I turned  CMS experienced high turnover in Marketplace staff.  The Federal Marketplace program 
around, I had to brief new  experienced significant staff turnover after the project moved to CMS in 2011, particularly in 
people.  This project is so 
CCIIO management positions.  Officials in Director- and Deputy-level positions in CCIIO in 
complex you have to be 
20112015 often had a tenure of less than 1 year.  One Director-level position was filled by 
immersed. CMS official 
seven different people during our study period.  As many as two-thirds (30 of 45) of the 
Director- and Deputy-level positions were vacant at some point during our study period, many 
for an extended time.  These vacancies were filled by staff who were temporarily moved from 
other parts of CMS to serve in an acting capacity.  In many instances, individuals in these 
temporary slots divided their attention with a second, or even third, leadership position.  One 
important position responsible for establishing premium rates was vacant for a total of 
24 months spanning 20112013.  CMS also experienced a high turnover of staff responsible 
for managing and overseeing key Federal Marketplace contracts.41  CMS staff reported that 
the high turnover and lack of permanent managers in key positions hindered program and 
organizational knowledge while making building relationships among management and staff 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           12  Preparation & Development | March 2010December 2012 
more difficult.  CMS officials later attributed the high turnover to a number of factors, 
including that former State and private sector employees intended to serve for only a short 
time to participate in the launch and that the workload was considered by some to be 
unsustainable.  A former CCIIO director stated that CCIIO was a rewarding place to work but 
was not sustainable based on the hours needed and timeframes to get the job done.  
IT contracting for the FFM encountered significant problems, including limited bids, 
uncertainty in funding, and disjointed CMS contract management 
92%  CMS selection of contractors and type of contracts limited the number of bids.  As CMS worked 
to finalize policies and standards for the marketplaces through 2012, it awarded key  
of contracts  IT contracts for the Federal Marketplace largely on the basis of existing contracts.  Prior OIG 
for the Federal  work on Federal Marketplace contracts found that of the 60 Federal Marketplace contracts, 
Marketplace were  55 were awarded under previously established contracts.42  CMS contracted the core pieces 
awarded under previously 
for the FFM and Hub in 2011, and the EIDM in 2012, using Indefinite Delivery Indefinite 
established contracts. 
Quantity (IDIQ) contracts.  Contracting through an IDIQ is generally more streamlined and 
faster than the processes for other types of contracts because CMS can choose from 
pre-qualified companies that are familiar with CMSs systems and procedures without having 
to follow all Government acquisition regulations, such as completing acquisition plans.43  CMS 
hired CGI Federal to build the core of the overall Federal Marketplace system, the FFM, which 
consisted of three main subcomponents:  eligibility and enrollment, plan management, and 
financial management subcomponents.  CGI Federals responsibility also included developing 
the website interface (functions that support consumer interaction) and online application for 
consumers.  CMS also hired QSSI to build the Hub and the EIDM.  The Hub enabled the Federal 
Marketplace to check application information such as income and citizenship, and the EIDM 
provided identity management services that enabled consumers to create accounts and verify 
their identities.  The EIDM was built for account-creation functions in multiple CMS programs, 
including Medicare. 
Use of the IDIQ contracts limited the number of companies allowed to submit proposals to the 
16 companies previously awarded an IDIQ contract, but uncertainty surrounding the ACA may 
also have limited contractor interest in the project.44  One CMS official later reflected that 
uncertainty about ACA requirements and funding may have further reduced the field of 
companies willing to vie for the contracts:  We had problems getting people to bid on 
contracts without assurances that the law would continue.  Ultimately, only four contractors 
submitted proposals for the FFM build, and only that of CGI Federal was deemed technically 
acceptable.45 
CMS did not use an acquisition strategy to develop contracts and select contractors.  In 
addition to the limited number of companies under consideration, the CMS process for 
establishing these key contracts included three factors that may have further hindered 
contractor selection and later results.  First, as OIG previously reported, CMS did not develop 
an acquisition strategy for Federal Marketplace contracts, nor did it perform a thorough 
review of contractor past performance beyond the basic requirement to consider past 
performance on prior task orders awarded under the IDIQ.46  An acquisition strategy is an 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           13  Preparation & Development | March 2010December 2012 
overall plan for satisfying the project mission in the most effective, economical, and timely 
manner.47  Having an acquisition strategy would have provided a framework for CMS to 
precisely assess project needs and make a systematic assessment of the contractors ability to 
meet those needs.  Not developing such a strategy likely limited CMSs ability to fully and 
systematically assess proposals.  Second, CMS chose to structure the FFM contract with CGI 
Federal as cost-plus-fixed-fee.  OIG work from January 2015 reported that this type of 
contract pays the contractor a prenegotiated award fee amount, requiring the contractor to 
bill as it incurs additional labor and material expenses.48  A cost-plus-fixed-fee contract is 
typically selected when the tasks are so uncertain that accepting a contract on the basis of an 
end product would pose undue risks for contractors.  The drawback is that it provides the 
contractor with less incentive to control costs and provide high quality products.49  Third, CMS 
did not define important aspects of the Federal Marketplace functionality in its original 
statements of work to some contractors.  For example, CGI Federal reported that their 
statement of work for the FFM contract . . .contained very broadly defined general technical 
requirements, task order management expectations, 
and work activities.  Officials and staff from both CMS 
October 2013 seemed a 
and CGI Federal indicated that this lack of specificity 
long way off.      
CMS official
created misperceptions about CGI Federals 
responsibilities, resulting in delays and additional work. 
Unclear division of staff responsibilities led to disjointed contract management by CMS.  In 
addition to the barriers presented by contract selection practices, confusion over the roles of 
contract managers hampered contract management.  Staff within OISs Consumer Information 
and Insurance Systems Group (CIISG) in CMS was responsible for managing the technical 
aspects of the contracts to ensure they met specifications; most importantly, the Government 
Task Leaders (GTL) within CIISG monitored contract progress and contractors deliverables.50  
At the same time, CMS contracting officers (CO) in OAGM were responsible for authorizing, 
administering, and terminating CMS contracts.51  COs appoint Contracting Officers 
Representatives (COR) to assist in the technical monitoring of a contract.52  The CO is the only 
CMS official authorized to make modifications to the contract.53  Contractors reported that 
they had difficulty determining where GTL responsibilities left off and the CO or COR picked 
up, and they received inconsistent direction when asking the same question of the GTL, CO, 
or COR.  
Such poor coordination between the GTLs and the contracting office also led to key staff 
discussing changes with contractors outside of formal channels, causing tension between CMS 
offices and lack of clarity to contractors.  Previous OIG work determined that CIISG and OAGM 
did not adequately maintain contracting records or always monitor contractor performance.54  
Part of the problem was that the CO did not always designate and authorize the COR in 
writing, and did not always document the specific duties and responsibilities assigned for each 
contract.  Regarding these challenges, the CMS Deputy CIO wrote, This is an epidemic of 
anemic management and leadership and it is much worse in our case because of the relative 
book of work that needs to be accomplished. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           14  Preparation & Development | March 2010December 2012 
Continued insufficient coordination and direction by CMS led to delays in guidance 
to contractors, and set the stage for problems with HealthCare.gov operations 
Solidifying business requirements and technical specifications was slow and involved much 
iteration.  Although the contractors had begun the website technical build, CMS was still 
making key regulatory and policy decisions and in many cases had not finalized the business 
requirements and the technical specifications needed to develop the software.  Writing the 
business requirements is an iterative process, with policy experts and technical staff refining 
the technical specifications to meet the functional needs.  In December 2011, a CMS technical 
official expressed frustration over the lack 
of a single leader to make decisions and 
We had an overall vision for 
sign off on business requirements, 
functionality, but we were not sure 
indicating that the lack of leadership 
resulted in mid-level staff and managers  how to get there.    
CMS technical staff
having difficulty coordinating and lacking 
  authority to make decisions.  
When they started   For the Federal Marketplace, the back-and-forth (between CMS policy and technical staff and 
the build, the requirements  CMS and contractors) was substantial because CMS was still making regulatory and policy 
they knew were in the 
decisions.  Contractors reported that they immediately identified the lack of business 
middle.  Things at the 
requirements as a problem and that the extent of iteration was unusual.  This likely reflects 
beginning, such as the 
CMSs lack of clarity in defining key marketplace functions, which traces back to conflicting 
application, were still being 
decided and were not  statutory interpretation and debates about policy choices.  In addition, the pending Supreme 
solved until very late in the  Court case regarding whether individuals would be mandated to purchase health insurance 
process. CMS contractor staff   created further uncertainty, given its potential to alter implementation of the ACA.55  One 
contractor noted that continued legal uncertainties surrounding the ACA also slowed 
progress.  For example, in the contractors view, regulations slowed to a trickle while waiting 
for the ruling.   
Supreme Court Upholds  This lack of clarity forecasted problems to come.  A CCIIO official sent the following email in 
Health Care Law, 5-4   late 2011 to officials in CCIIO and OIS:  I am growing increasingly worried that our 
The New York Times,   [marketplace] work is off track.  We have not been effective in communicating the importance 
06/28/2012 
of finalizing policy timely so that operational decisions can be made and processes built.  
 
Several CMS staff reported that they believed that stronger CMS leadership could have 
prevented some problems by mandating that no additional changes to business requirements 
be made at a certain point.  CMS technical staff were often caught in the middle between 
policymakers and contractors, moving forward on the basis of assumptions but without a clear 
plan.  By the time the June 2012 Supreme Court ruling confirmed the legality of the individual 
mandate, CMS had already lost critical months toward focused development of the Federal 
Marketplace and HealthCare.gov. 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           15  Preparation & Development | March 2010December 2012 
Agile is a method of  The selected method of software development made it easier for policymakers to seek frequent 
software development that  and late changes.  In its contract proposal to build the FFM, CGI Federal indicated its intent to 
breaks larger tasks into 
use the agile method of software development, which separates development of pieces of 
smaller increments that are 
the build such that developers can begin on some components while business requirements 
then completed and tested. 
for other components are still being finalized.  This method is commonly used when some 
pieces are not well-understood at the outset.  An advantage of agile development is that each 
increment, often completed in two-week sprints, results in a functioning product that can 
be tested, so that the business owners (in this case, CMS staff) responsible for a particular 
process can assess whether the software meets the projects needs and adjust business 
requirements accordingly.  This ability to adjust, however, enabled policymakers to frequently 
change business requirements and technical specifications on an ongoing basis.  Changes 
made through the agile development process must still be properly considered, documented, 
and communicated.  Managers at CGI Federal reported in interviews that the frequency of 
CMSs requests for change resulted in too much change too late in the process, contributing 
to delays.  For example, CGI Federal managers reported that CMS did not define business 
requirements at the beginning of each sprint and often made changes throughout the sprint, 
which inhibited the agile method and resulted in incomplete development.  An agile method 
also allows for pieces of the build to be completed out of order, but this also caused problems 
for the FFM in that some earlier pieces that were still undecided affected the build of 
subsequent components. 
Key technological choices and poor execution also inhibited the website build.  A number of 
technological decisions hampered development, including selecting a nontraditional 
technology that did not align with the expertise available.  CGI Federal indicated in its final 
FFM contract proposal that it would use a combination of two types of database platforms 
(the digital structure upon which the website is built):  a traditional relational platform such 
as Oracle that uses tables to store data; and a nontraditional NoSQL platform that uses 
non-tabular documents to store data.  Given that each type has different benefits, CMS 
indicated in its evaluation of CGI Federals contract proposal that use of both Oracle and 
NoSQL databases was a strength.  The key benefit of the traditional relational platform is its 
wide use, with most developers experienced in Oracle.  The key benefit of the nontraditional 
NoSQL platform is its potentially greater capability, in that it can allow more data to be 
transferred at a time and can be easily expanded to include more data or users.  However, 
NoSQL platforms were used less frequently and fewer developers had experience building on 
and maintaining them. 
Although CGI Federals final contract proposal, accepted by CMS, indicated partial use of a 
NoSQL platform, it did not specify the brand of platform it would useover 100 vendors 
provide NoSQL platforms.  In November and December 2011, CGI Federal met with one of the 
oldest NoSQL vendors, MarkLogic, to better understand the risks and benefits of MarkLogics 
platform.  Following these meetings, CGI Federal managers expressed concern to CMS about 
using MarkLogics platform.  Interviews and documentation indicated that few at CGI Federal 
and CMS had development experience with the platform.  In a presentation to CMS technical 
managers in January 2012, CGI Federal indicated that its lack of experience with MarkLogics 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           16  Preparation & Development | March 2010December 2012 
platform could affect the timely development of the FFM.  CGI Federal reported to CMS that it 
would be unable to fully replace staff expertise on traditional databases with equivalent 
expertise with MarkLogics NoSQL platform and recommended that CMS develop a 
contingency plan. 
Despite CGI Federals reluctance to use the platform, CMS notified CGI Federal of the 
selection of MarkLogic in a January 2012 Technical Direction Letter (TDL).  A TDL is used to 
clarify or give specific direction to the contractor within the scope of the statement of work 
and cannot alter the contract cost.  The TDL directed CGI Federal to use MarkLogics platform 
and to obtain staffing support from MarkLogic to help implement the change.  In a February 
2012 presentation to CMS, CGI Federal reiterated its concerns about the use of the platform.  
CGI Federal managers contended later 
that use of a TDL demonstrated that 
From a design, development, and 
CMS did not recognize the importance 
integration standpoint, MarkLogic is 
of the platform choice, and use of a 
more formal Contract Change Order  vastly different from Oracle [the prior 
would have allowed both parties to  relational platform].   
fully vet the decision and possible   
CGI Federal presentation
consequences.  A Contract Change 
Order directs the contractor to make a 
change that may increase the contract cost, change the terms of the contract (e.g., extend the 
length of the contract), or be outside the existing scope of the statement of work.  CGI Federal 
hired additional staff trained in the platform and subcontracted with MarkLogic for technical 
assistance, but staff from both CMS and MarkLogic reported that CGI Federal never retained 
the number of experts needed to configure and integrate the MarkLogic technology. 
Both CMS and CGI Federal staff reported that use of MarkLogics platform caused 
development problems throughout the remainder of the FFM build.  Although QSSI used the 
platform for development of the Hub, CMS and QSSI did not report the same problems as CGI 
Federal had using the platform to develop the FFM.  CMS documentation indicated that QSSI 
hired an adequate number of additional staff from MarkLogic to work on the Hub 
development and operations, as CMS had directed CGI Federal to do in the TDL. 
Another technical decision with the website build may have compounded problems with the 
platform.  CGI Federal included in its final contract proposal for the FFM the use of 
automatically generated software code called Model-Driven Architecture (MDA), typically 
used in conjunction with developer-written code to save time and reduce human error.  
According to a CMS official, CGI Federal was committed to this technique and referred to MDA 
as their bread and butter.  Although CMS approved the use of MDA in the contracting 
process, the CMS GTL responsible for monitoring the technical progress of CGI Federals 
contract later reported that it was one of the biggest culprits in the coding problems.  The GTL 
explained that CGI Federal used MDA for perhaps too much of the build (estimated at 
60-70 percent), and did not build the MDA code effectively or coordinate it properly with 
developer-written code.  The GTL also came to believe that use of MDA created additional 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           17  Preparation  &  Development | March 2010December 2012  
problems  in the last mile  of development because it was  difficult to modify  when late  
changes  were needed.  
Poor  CMS  management  of  changes  to  the  website  build  created  delays  and  confusion.   Each 
development team maintained detailed schedules to  manage system development, which 
were summarized into  an  integrated  master  schedule  to  track  development  of  all  Federal  
Marketplace  systems.   The integrated  master schedule  was maintained  by CIISG, which served  
as the CMS IT project management office for the Federal Marketplace.   CMS  required  that  
CMS  staff  and  contractors  working  on  the  website  submit  changes  in  business  requirements  
or  technical  specifications  for  approval  by  a  Change  Control  Board  if  the  change  could  alter  
the  project  cost,  scope,  or  schedule.   The  Change Control Board  was  mostly  comprised  of  
representatives  from  OIS,  CCIIO,  OC,  and  CMCS  who  were  to  make  these  decisions  and  log  
changes  in  a  central  repository  accessible  to  both  CMS  and  contractors.   CMSs  central  
repository  stored  a  variety  of  information  about  the  Federal  Marketplace  project,  including  
not  only  business  requirements  and  technical  specifications,  but  also  archived  software  code,  
infrastructure  descriptions,  testing  results,  and  past  defects  that  were  identified  and  
resolved.56   Part  of  the  purpose  of  the  central  repository  was  to  enable  CMS  staff  managing  
contracts  to  analyze  and  compare  technical  specifications  to  ensure  that  the  technical  build  
met  business  requirements.  
However,  the  Change Control Board  and  central  repository  processes  were  not  effectively  
managed,  leading  to  delays  and  confusion  about  tasks  and  progress.   In interviews, CMS  staff  
said  that difficulty tracking  and responding to revisions to the  business  requirements may  
have concealed problems  with production and schedule.   First,  project  documentation  and  
email  correspondence  revealed  that  CMSs Change Control Board  frequently  cancelled  
decision  meetings  and  did  not  promptly  address  change  requests.   Second,  when  changes  
were  approved,  contractors  were  not  always  informed  immediately,  resulting  in  further  
delays.   Third,  CMS  and  contractors  did  not  always  log  approved  changes  in  the  central  
repository, as required.   Fourth,  CMS  staff  did  not  appear  routinely  to  review  documents  in  
the  repository  to  identify  problems,  as  intended  by CMS.   Email  correspondence  indicated  that  
by  mid-2013  there  were  several  thousand  documents  in  the  central  repository,  and  CMS  staff  
raised  concerns  that  the  documents  were  not  well  organized  and there was no  evidence  of  
review  by the  Change Control Board.   This  failure  to  approve  changes  promptly  and  track  
changes  in  the  repository  also  inhibited  the  agile  development  process  that  was  used  to  
build  the  FFM.   As  part  of  the  Change Control Board  process,  developers  were  required  to  
submit  changes  before  they  could  begin  the  next  sprint,  so  delays  in  the  board  approval  
   process  delayed  the  daily  work  of  the  software  build.    
If we had the   Additionally, CMS  staff and  contractors  sometimes circumvented the  Change Control Board  
requirements sooner, we  process, making changes without  board  approval or the knowledge of affected CMS  staff and  
would had discovered the 
contractors.  Between March and December 2012, CMS  staff not authorized to  modify the 
problems sooner.  The flux  
contract added numerous  work items to CGI Federals contract.57   The CO and  initial  COR  
in the requirements  
responsible  for CGI Federals contract were unaware  of the additional items added to the  
allowed the developers to  
hide, to not come clean.   contract until  a newly assigned  COR  discovered the contract overrun, referred to  as an  
 CMS  technical  staff    unauthorized commitment, which violates  the  Federal Acquisition Regulation.58   However,  
OEI-06-14-00350             HealthCare.gov:   Case Study of  CMS Management of the Federal Marketplace            18   Preparation & Development | March 2010December 2012 
an unauthorized commitment may later be approved under certain circumstances.59  In this 
instance, CMS was able to modify CGI Federals contract to fund the unauthorized changes, 
but bypassing the Board created confusion among stakeholders about which changes were 
implemented. 
According to both CGI Federal and CMS staff, this continued alteration of business 
requirements through 2012 and into 2013 resulted in significant delays in website 
development.  Additionally, CMS was still making policy decisions late in development that 
changed technical specifications and created additional work.  One example was CMS OCs 
decision to change the website interface long after CGI Federal had begun development.  
Another late policy decision that changed the website build was the determination of whether 
only one member or all family members needed identity verification. 
CGI Federal advised CMS staff numerous times in 
weekly status reports beginning in February 2012 
We should have been more 
that delays in finalizing business requirements 
emphatic in warning CMS of 
were affecting the development timeline, yet the 
the risks of launching. 
changes continued.  CGI Federal managers 
 
indicated in interviews that they now believe they  CGI Federal official
should have alerted the CMS Administrator 
directly regarding the depth of the problems and 
spoken more candidly and earlier to other CMS officials.  A CGI Federal official stated, We 
should have been more emphatic in warning CMS of the risks of launching.  CGI Federal 
managers reflect now that they did not do so primarily in order to follow the standard chain 
of command in reporting problems to CMS. 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           19  Preparation & Development | March 2010December 2012 
KEY FACTORS CONTRIBUTING TO BREAKDOWN 
  POLICY DEVELOPMENT DELAYS 
Implementing the Federal Marketplace required substantial policy development and decisionmaking to inform 
technical planning and implementation of the website.  This included not only writing regulations to govern the 
marketplaces, but also establishing partnerships with other entities involved in implementation, such as other 
departments, States, and issuers.  This policy work was made more difficult and protracted by a lack of 
certainty regarding the mission, scope, and funding for the Federal Marketplace and website, which was 
caused in part by varying expectations for the marketplaces and a contentious political environment.  This time 
spent developing regulations resulted in further delays later in the process, such as States deciding whether to 
join the Federal Marketplace and technical needs for website contracts.  These delays used valuable time and 
made an already compressed timeframe more difficult. 
  POOR TRANSITION to CMS 
The transition of the Federal Marketplace to CMS after 10 months in HHS OCIIO resulted in problems that 
lasted long after the move.  HHS OCIIO made significant strides in establishing the policy framework, but did 
not focus attention on planning for the projects longer-term technical and operational needs.  CMS had to 
reconfigure roles and timelines, determine how it would leverage its resources, and begin work behind 
schedule.  Further, while CMSs infrastructure and experience provided greater resources for the project, it 
also required the Federal Marketplace to operate within a large bureaucratic structure that separated contract, 
policy, and technical staff, causing further diffusion to the project team and making implementation needlessly 
more complex.  Interviews and documentation indicate that CMS leadership failed to address this diffusion by 
fostering effective collaboration, particularly between CMS policy and technical staff and contractors. 
  LACK of PROJECT LEADER 
CMSs failure to immediately assign a project leader was particularly problematic for HealthCare.gov.  Clear 
leadership alone may have corrected many of the projects deficiencies.  As a new project with staff spread 
across CMS, the HealthCare.gov team needed unity and identity within the larger organization.  The project 
also needed quick decisionmaking and flexibility, made easier when a single lead entity is responsible rather 
than multiple entities with organizational layers.  Effective project leadership would have enabled a 
comprehensive view across the project to better assess progress, identify problems, and determine priorities.  
Leadership was also lacking beneath the senior executive level, with high turnover among officials in CCIIO and 
high-level CMS technical officials involved in the HealthCare.gov build. 
  MISMANAGEMENT of KEY WEBSITE CONTRACT 
Mismanagement of the FFM contract with CGI Federal was a key problem for CMS in development up through 
the launch.  The contracting process suffered from a limited number of bids and uncertainty about funding and 
technical specifications, and CMS contract oversight was disjointed and lacking.  CMS made frequent changes to 
contracted work, some of which represented questionable technical decisions, and did not communicate 
effectively with CGI Federal about the changes and any resulting effects on staffing and schedules.  Interviews 
and documentation indicate that CGI Federal made missteps as well; for example, the company did not 
adequately increase staffing and expertise when changes were made and progress began to deteriorate.  Still, 
poor CMS management of the contract substantially contributed to problems building the FFM, a critical 
component of HealthCare.gov. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           20  Final Countdown to Launch | January 2013September 2013 
Hope is not a strategy. 
Former CCIIO staff 
C 2 
HAPTER 
 
F C L  
INAL  OUNTDOWN TO  AUNCH
January 2013September 2013 
 
By January 2013, CMS knew that 34 of the eventual 36 States would participate in the Federal 
Marketplace for the first open enrollment period, finally giving contractors the knowledge of 
needed scope and capacity.  Still, inability to finalize business requirements continued to 
hamper the website build into 2013.  Some of these requirements were delayed because CMS 
had not yet completed the underlying program policies.  For example, CMS did not finalize 
decisions about some aspects of the programbenefits required for health plans and using a 
single, streamlined Medicaid applicationuntil February and June 2013.60  In other cases, 
such as Medicaid eligibility exceptions, CMS abandoned the goal of creating new policies and 
reverted to former practices because it lacked the time to solidify new policies. 
Several entities voiced concerns about the status of HealthCare.gov, but warnings 
were either not fully communicated or not acted upon 
18 A CMS technical advisor and two consulting firms identified specific problems that threatened 
  a successful launch.  Throughout the course of the HealthCare.gov build, staff at HHS and 
CMS, as well as outside entities, identified problems with the program and warned that these 
Documented 
problems warranted action.  By January 2013, the most common advice given to CMS senior 
Warnings 
leadership was that the program needed a single lead entity and that CMS should stop 
with concerns regarding 
the HealthCare.gov build  revising policy.  During JuneOctober 2012, the technical advisor hired by CMS to assess 
HealthCare.gov progress prepared six reports for the CMS Administrator and CCIIO leadership, 
laying out problems in explicit detail after reviewing project documentation.  CMS also hired 
outside firms to assess progress.  Throughout 2012 and 2013, a series of 11 technical reports 
from the firm TurningPoint Global Solutions gave scathing reviews, including a progress report 
in April 2013 that listed the Top 10 Risks of the website build, such as inadequate planning for 
website capacity and deviation from IT architectural standards.  In addition, in early 2013, the 
Secretary hired McKinsey Consulting to review the program and make recommendations to 
improve CMS management of the project.  In all, CMS received 18 documented warnings of 
concerns regarding the HealthCare.gov build between July 2011 and July 2013, all containing 
substantial detail about the projects shortcomings and formally submitted to CMS senior 
leadership or project managers at CMS. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           21  Final Countdown to Launch | January 2013September 2013  
Among  the  recommendations  was  to  assign  a  single  project  leader  for  the  Federal  
Marketplace,  the  same  advice  given  earlier  by  the  U.S.  CIO.   However,  these  reports  were  not  
shared  broadly  with  CMS  leadership  and  technical  staff.   The  TurningPoint  reports  were  
presented  only  to  CMS  technical  staff,  such  as  the  CMS  Deputy  CIO,  and  McKinseys  
recommendations  were  presented  only  to  senior  CMS  officials  such  as  the  Administrator.   The  
CMS  Deputy  CIO,  the  chief  CMS  technician  on  the  HealthCare.gov  build,  specifically  reported  
that  technical  staff  did  not  receive  or  even  have  knowledge  of  the  McKinsey  report  findings.61  
Attempts by CMS to take action on recommendations were poorly executed.   CMS  officials  
were  repeatedly  made  aware  of  problems  with  the  development  of  HealthCare.gov  and  
attempted  to  take  corrective  action,  but  these  efforts  were  largely  unsuccessful  because  they  
were  not  fully  and  diligently  executed.   For  example,  after  criticism  that  there  was  not  clear  
leadership,  CMS  assigned  its  newly  appointed  COO  (previously  the  Deputy  COO)  in  early  2013  
to  head  the  Federal  Marketplace  program,  but  the  assignment  was  not  formally  announced,  
the  position  was  not  supported  by  clear  responsibilities,  and  the  designee  had  an  already  large  
responsibility  as  CMS  COO.   In  addition,  the  Deputy  COO  position  remained  vacant  until  
November  2013,  which  meant  even  greater  responsibility  for  the  new  COO.   Reflecting  after  
the  launch,  CMS  officials  pointed  to  this  assignment  as  an  example  of  underestimating  the  
enormity  of  the  task.   The  COOs  assignment  also  was  not  formally  communicated  to  other  
CMS  leadership  and  staff,  although  staff  indicated  later  that  the  group  sort  of  knew.  
Who  is  tracking?   As  another  example,  a  CMS  advisor  recommended  that  the  project  hire  a  technical  systems  
Although  CMS  staff  in  both   integrator62  and  CMS  officials  and  contractors  discussed  this  need  at  several  points  in  the  
its  technical  and  policy   project.   However,  in  correspondence  and  congressional  testimony,  it  was  clear  CMS  technical  
divisions  were  assigned  to  
leadership  perceived  that  CMS  itself  was  already  serving  in  that  role.63   CGI  Federal  managers  
oversee  contracts,  
reported  that  the  lack  of  a  true  systems  integrator  created  extra  work  that  was  outside  the  
documentation  indicated  
little  formal  corrective   scope  of  their  contract.   For  example,  CGI  Federal  reported  having  to  assist  CMS  with  defining  
action  or  even   the  business  requirements  to  mitigate  problems  with  interdependency  of  various  Federal  
communication  with   Marketplace  computer  systems  and  avoid  losing  more  time  for  system  development  and  
contract  managers  in  2012  
testing.   Although  the  systems  integrator  need  not  be  a  contractor,  CMS  staff  and  contractors  
and  early  2013.  
later  identified  two  barriers  to  CMS  serving  in  this  capacity:   they  reflected  that  few  at  CMS  
had  the  necessary  experience  integrating  a  project  of  this  size  and  complexity,  and  that  CMS  
leadership  did  not  recognize  the  need  to  clearly  outline  responsibilities  and  delineate  this  role  
from  other  CMS  tasks.  
CMS failed to effectively manage poor contractor performance with HealthCare.gov, and did 
not take sufficient action when aware of problems.   In  February  2013,  independent  reviewer  
TurningPoint  Global  Solutions  determined  that  the  FFM  had  a  high  number  of  coding  defects.   
CMS  staff  later  reported  that  CGI  Federals  coding  quality  did  not  improve  later  in  
development.   In  an  onsite  review  in  August  2013,  just  over  1  month  before  the  
HealthCare.gov  launch,  CMS  staff  discovered  that  CGI  Federal  developers  did  not  follow  some  
best  practices  for  making  latestage  coding  changes,  resulting  in  software  code  conflicts  
between  some  systems.   First,  CMS  discovered  that  CGI  Federal  was  not  following  a  
standardized  process  for  documenting  development  of  code,  which  resulted  in  very  limited  
information  available  in  the  CMS  Central  Repository.   As  a  reviewer  reported,  [CGI  Federal]  
OEI061400350             HealthCare.gov:   Case  Study  of  CMS  Management  of  the  Federal  Marketplace            22   Final Countdown to Launch | January 2013September 2013 
was making changes on the fly without documenting them up until the launch.  They were 
breaking, from an industry perspective, every golden rule.  Second, CMS observed developers 
modifying the system without assessing the impact to other parts of the system, resulting in 
coding defects that required weeks to troubleshoot.  Third, CMS observed developers using an 
outdated version of code to continue building the website and deploying incorrect versions of 
code.  Fourth, CMS discovered that development teams were making system modifications 
that produced inefficiencies that required additional computing resources to process the 
code. 
In one technical example revealed in interviews, CMS discovered after the October 1 launch 
that in some instances the website software requested to access information from the FFM 
database over 100 times for a single operation that should require 1 or 2 requests.  
Compounding this problem was the fact that the requested information from the FFM 
database used what one CMS staffer called a bloated data model that made the information 
10 times the ideal size and larger with each request.  Therefore, each of the numerous 
requests made to the database would have retrieved ever larger records and required more 
capacity to process.  According to CMS documentation of its correspondence with CGI 
Federal, CMS noted delays and performance problems, but did not issue a Corrective Action 
Plan when performance did not meet the contractual commitments.64  Correspondence 
among CMS staff indicated that some 
believed CGI Federal needed to have more 
We are in bad shape.  Perhaps 
skin on the line to help ensure on-time 
worse than ever and we are not 
delivery of functionality, implying that the 
cost-plus-fixed-fee contract type or lax  even touching the hard stuff yet. 
contract oversight may have weakened  Email from CMS technical staff 
project management.  
CMS senior leadership failed to fully grasp the poor status of the website build, and to alter its 
course.  Interviews indicated that at this point in time, CMS senior leadership believed that the 
technical work was still on schedule or close enough that concerted effort would ensure 
delivery at launch.  An official in CCIIO reflected in an interview that [CMS senior officials] 
would sit in meetings across from me and not know there is an enormous fire burning behind 
them.  It is not clear why CMS senior leadership failed to grasp the poor status or why those 
who felt the project was in danger failed to communicate their opinions more forcefully.  
When CMS requested a live demonstration for leadership of the online health insurance 
application 1 month before first open enrollment, CGI Federal presented snapshots of the 
software rather than a demonstration of the functionality.  A CMS technical official 
commented, You cant test drive a Ferrari just by looking at pictures of a Ferrari going fast.  
CGI Federal ultimately staged two successful live demonstrations in the weeks before first 
open enrollment, but the failed demonstration the prior week, which did not show actual 
functionality, did not indicate to CMS leadership that the Federal Marketplace was in trouble. 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           23  Final Countdown to Launch | January 2013September 2013 
Path Dependency  CMS leadership, staff, and contractors became fixed or dependent on an organizational path 
is an unfounded reliance  to complete the website build, failing to adequately consider new information and alter 
on former ways of doing 
course as needed.  CMS has a long history of administering large programs, such as Medicare 
things that prevents 
and Medicaid, but the organizational structure used to manage those programs was 
adaptation to new 
insufficient for developing and implementing the innovative technology solutions required for 
conditions. 
the website build.  Part of the problem was poor communication across divisions and between 
CMS and contractors in an environment where project status changed quickly.  As previously 
stated, CMS technical staff often received different messages from policymakers and 
contractors, and were forced to develop ad hoc strategies that were then not well-
documented and unlikely to evolve further.  CMS staff and contractors reflected that they had 
failed to coordinate the work, did not adhere to a clear schedule, and failed to track progress 
and changes. 
The HealthCare.gov website build was alarmingly behind schedule, with CMS 
scrambling for minimal functionality 
Communication deteriorated further as problems worsened, with a critical early piece of 
HealthCare.gov failing in July 2013 and more problems arising through the summer.  By 
summer 2013, responsible CMS staff had been warned of problems repeatedly and knew the 
website build was in trouble.  Still, both leadership and responsible staff did not fully grasp the 
extent of problems and the degree to which the build was behind.  Reflecting afterward, those 
involved reported that communication deteriorated as the situation worsened.  Interviews 
and documentation revealed this was due to a number of factors:  the range and number of 
technical problems made it difficult for nontechnical staff and officials to gauge the enormity 
and impact of problems, negative reports about progress became so common that they lost 
their power to alert, and information was not communicated comprehensively to 
demonstrate the extent of the problems across the build. 
This lack of recognition changed somewhat in 
July 2013, when CMS received a tangible sign of  The level of complexity was 
problems:  an early component of the website 
greater than what we originally 
build failed immediately.  One of CGI Federals 
anticipated.    
significant deliverables for summer 2013 was  CMS official
the pre-enrollment account creation system, 
Account Lite.  In comparison to the overall 
website build, the functionality for Account Lite was fairly simple and straightforward.  Still, it 
posed a challenge to deploy, in part because CMS did not request this functionality until 
May 2013 with an expected launch date of July 1, 2013.  CGI Federal delayed delivery of the 
system past the July date, and then requested the assistance of CMS technical staff. 
When CGI Federal finally demonstrated the product, it performed poorly.  A top OIS official 
indicated that a week past the date Account Lite was supposed to launch, OIS found 
105 defects with the Account Lite system.  The extent of problems with Account Lite raised 
alarm throughout OIS, CCIIO, and OC, and began a shift in thinking that would lead to reducing 
planned functionality for HealthCare.gov.  As a CCIIO official later noted, That was our first 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           24  Final Countdown to Launch | January 2013September 2013 
inkling into how bad things were.  If they couldnt even deliver Account Lite, where were we 
on the build? 
CMS technical staff began to avoid reporting further bad news about the website build, leading 
to greater disconnection as problems worsened.  Despite knowledge that the project was 
going poorly, many of the CMS and contracted staff responsible for the Federal Marketplace 
build were averse to alerting those in leadership positions that there were problems with the 
build.  Correspondence indicated that this was driven in large part by the belief that they 
would be able to succeed in the end, and thus there was little benefit to causing alarm.  It also 
indicated that CMS technical staff were so busy attempting to complete the build that they 
were reluctant to take time reporting to executives and answering questions.  During this 
timeframe, HHS IT leadership requested more information from CMS, but did not receive the 
information requested.  The reluctance to convey information included the critical topic of 
website security, even though security testing ran well behind schedule and identified possible 
risks.  The top CMS security official later testified to Congress that [CMS leadership] was not 
properly briefed or properly portrayed, [about] the issues that were happening that week 
during security testing.65  During the Account Lite problems, a CMS officials correspondence 
indicated that those responsible kept thinking and hoping that the next thing will solve more 
issues and we'll be okay. 
117%  Despite the importance of the mission, a small number of CMS staff carried responsibility.  Had 
communication about the projects status been more open at this stage, leadership at CMS 
Increase   and contractors might have been prompted to add additional staff to the website 
in CMS technical staff to  development earlier; they did not add substantially to staffing until less than 2 months before 
support the Federal  launch.  CMS officials reflected later that in summer 2013, there were still relatively few 
Marketplace project 
people working on the project given its size and slippage in the schedule, and experienced 
during August 2013, 
technical staff and others in CMS were not called to assist.  Officials and staff not included in 
nearly all reassigned from 
the development of HealthCare.gov included CMSs CTO, responsible for technological 
other CMS projects. 
innovation and strategy, CMSs CIO, responsible for operating CMS technological systems, and 
most of the staff of CMS OIS.  Additionally, HHS technical officials and staff could seemingly 
have been called in to assist, including the HHS CTO, the HHS CIO, and others.  CMS 
documentation indicated that CGI Federal did not make requests during this period for more 
staffing or more time. 
By late July 2013, HealthCare.gov technical managers were requesting assistance from others 
in their divisions; as one stated, You know it has to be bad if I am requesting help.  Two 
months before the October 1 launch, CMS temporarily assigned 60 additional staff from other 
projects within OIS (39 staff) and in other parts of CMS (21 staff) to assist the 51 staff already 
working on the Federal Marketplace, a 117 percent increase (see Figure 1).  Some involved in 
the project reported that it felt too late to involve others, given the steep learning curve, and 
that they were embarrassed to add others into a mess.  As late as mid-September 2013, 
there were calls for establishing contingency plans, including pulling in additional staff and 
even CMS staff taking the project over from contractors. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           25  Final Countdown to Launch | January 2013September 2013 
Figure 1:  Increase in CMS staff dedicated to Federal Marketplace, JulyAugust 2013. 
 
Source:  OIG analysis of 2013 CMS organizational charts, 2015. 
Last minute attempts to correct and avoid further problems with HealthCare.gov 
 
were ad hoc and insufficient 
We contemplated   CMS continued to prioritize functions and cut those it could not complete, including the 
[as late as August 2013]  Spanish language website and the anonymous shopper function.  CMS focused by 
whether we would scrap the 
mid-August on determining the minimum that could be delivered by October 1, holding 
whole thing and start over.  
reprioritization meetings to further reduce the scope of the HealthCare.gov build to deliver 
CMS technical staff 
basic functionality.  Part of the reprioritization process involved collaborating with CGI Federal 
to establish a list of minimum essential capabilities and a timeline for FFM development.  
There was an attempt at this time to recognize problems and renew project unity and mission.  
The CMS COO, assigned to serve as the single project lead, announced at an August 2013 
meeting, This is a blame-free zone.  We are a team.  This is the Presidents number one 
priority and we will make it happen, but we must be open and honest with each other.  The 
revised scope and functionality of the website was reduced to only what CMS considered 
necessary:  accepting information, determining eligibility, and selecting a plan.  There was 
skepticism among CMS staff regarding whether even that could be completed:  Around the 
table, people were saying, how are we going to get this done?  There was also discussion 
about whether the existing pieces were so flawed that CMS should begin developing some 
pieces anew rather than improve upon the existing structure.  Reprioritization resulted in 
reductions to planned scope and functionality of the website.  In August 2013, CMS called on 
CGI Federal to develop a definitive plan for the final 40 days.  The CMS Deputy CIO instructed 
them, Dont dwell.  Dont debate.  Dont be in denial.  Come up with a plan, however thin it is 
because it certainly is better than an unrealistic plan.  The reprioritization included delaying 
the Spanish language website, CuidadoDeSalud.gov, a tool that correspondence indicated was 
particularly important to White House staff.  Meeting notes from this time period revealed 
that the CMS Administrator was notified of this decision, made by other senior CMS officials, 
on September 3, 2013. 
After consulting with technical staff, the CMS Administrator communicated to HHS and White 
House officials that CuidadoDeSalud.gov would be working by October 15, 2013. (It was later 
delayed further and was not working until early December 2013.)  The Secretary of HHS and 
the U.S. CTO expressed concern about losing consumers and asked CMS to provide additional 
call center support for Spanish speakers, which it did.  CMS also announced on September 26, 
2013, that it would delay until November 2013 its completion of the Small Business Health 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           26  Final Countdown to Launch | January 2013September 2013 
Options Program (SHOP) website, a companion website to HealthCare.gov.66  SHOP provides 
health plan selection and enrollment for businesses with 50 or fewer full-time-equivalent 
workers.  (The SHOP website was further delayed until second open enrollment, opening in 
November 2014 then adding further consumer tools in February 2015.) 
62   Needing even further reductions in technical scope, CMS also delayed a tool to identify and 
%
compare health plan information that CMS had dubbed anonymous shopper, as well as 
of Americans did not   parts of the Eligibility and Enrollment functionality.  The anonymous shopper tool would have 
know the exchanges were  allowed consumers to view some targeted health plan information, including premiums 
opening October 1, 2013. 
estimates, without completing a full application.  (The tool was not truly anonymous in that 
Kaiser Family Foundation 
consumers would still have to create an account.)  Testing of the anonymous shopper tool 
during AugustSeptember 2013 revealed that it did not provide accurate information and 
would require significant rework. 
Since CMS did not consider the tool to be critical for the launch, it delayed the completion 
until after October 1, 2013.  Parts of the Eligibility and Enrollment system that were delayed 
until later included the automated functionality enabling the FFM to send and receive 
enrollment information to issuers regarding enrollee status (e.g., payment of premium, plan 
cancellation, changes in circumstance) and the resolution of complex application 
inconsistencies (e.g., income, whether applicant is lawfully present in the U.S.), which can 
occur when Federal data available through the Hub do not exist or do not match an 
applicants information.  A CMS official reflected later that reducing scope seemed 
reasonable and normal before a launch of this size.  However, project documents and 
correspondence indicated the lateness and depth of these reductions was not planned. 
Final State Count  CMS and contractors recognized they would not finish system functionality testing before the 
For the first open  launch, but prioritized delivering the product on time over testing and resolving problems.  By 
enrollment period,   August it was clear to both CMS technical staff and contractors that the system would not be 
36 States, including  
fully tested for functionality before October 1, 2013.  As one contractor remarked, You cant 
7 State-partnership 
test what is not built.  Issuers complained about testing delays and problems conducting 
marketplaces, used the 
tests due to incomplete and malfunctioning software and unavailable testing environments 
Federal Marketplace,  
and 15 States had   (the computing and storage space to run tests).  This lack of system capacity to conduct 
established State  testing affected those who were building website pieces because the agile development 
marketplaces.  process relies on testing of each increment as it is completed to ensure it functions correctly.  
In addition, according to CMS and contracted staff, CMS was never able to complete full 
end-to-end testing that identifies problems in how the pieces work together, because the 
component pieces arrived too late.  Regardless of the development process used, complete 
end-to-end testing is the final, and a critical, step in simulating consumer use of all functions.  
One technology contractor involved in the build reported, End-to-end testing is critical.  Its 
suicide not to do it.  CMS staff did not appear to consider the lack of functionality testing to 
be a dire situation.  The CMS technical team reported they considered launching the website 
on time the priority over testing for and resolving performance problems.  One CMS technical 
official characterized the launch itself as a test of the system and indicated that CMS planned 
to resolve problems after launch, as CMS had done with other large programs, such as 
Medicare Part D. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           27  Final Countdown to Launch | January 2013September 2013 
CMS prepared to launch HealthCare.gov on October 1, 2013, as planned, optimistic 
in spite of problems and never seriously considering delay 
CMS leadership held no formal discussion of delaying the website launch date, despite poor 
progress.  There were many discussions in the months leading up to the launch that the 
HealthCare.gov build was behind schedule, including multiple presentations to CMS 
leadership.  Still, documents and interviews indicated that no one among CMS leadership, or 
seemingly even among CMS staff, seriously discussed delaying the October 1 launch date.67  
This may be in part because some CMS staff and contractors working on the build were under 
the misimpression that the deadline for website functionality was statutory when, in fact, the 
ACA required that health plan coverage begin by January 1, 2014.68  Moreover, some HHS and 
CMS staff feared that due to the high expectations and the contention surrounding the 
marketplaces, if HealthCare.gov did not launch as planned it might fuel efforts in Congress to 
repeal the ACA.  
Several key CMS technical staff reported they 
never discussed the launch date with CMS  People needed insurance.  In the 
leadership, neither when the date was set in  absence of major problems, it 
2012 nor leading up to the launch.  Some staff 
was important to move forward. 
at CMS complained in correspondence to each 
 
HHS official
other that the timeframe was unrealistic and 
that leadership was bent on moving forward 
despite the significant workload and problems.  CGI Federal reported that it did not request 
additional time or formally request that CMS delay the launch because it believed CMS would 
not delay due to the White Houses public commitment to launch on October 1, 2013.  Many 
CMS staff reported later that they were eager to launch HealthCare.gov despite concerns 
shortly before launch, and were optimistic about its success.  Some may have assumed the 
launch would be delayed at the last minute if functionality did not operate correctly; the 
Medicare Part D website launch was delayed several times, on its proposed launch date and 
two other promised dates, ultimately launching three weeks after originally planned.69 
 
The launch was   Not delaying the launch resulted in a race to complete what was possible before the deadline.  
alpha version 1 of the  By September, concern grew to the degree that officials with little prior involvement in the 
[Federal] Marketplace.   
Federal Marketplace, including managers from other programs at CMS, technical officials from 
You would expect version 1 
HHS and other Federal agencies, and the U.S. CTO, began asking responsible CMS and 
to do exactly what it did . . . 
contractor staff for more detailed progress reports and offered their assistance.  At this point, 
launching to 300 million 
Americans on day one was  CGI Federal and CMS technical offices were filled with staff and experts from multiple 
a crazy idea. HHS official  Government and contractor offices, including support staff from software vendors and CMS 
technical staff who had no prior knowledge of the Federal Marketplace program and 
therefore were not optimally prepared to contribute quickly.  CMS OC also began notifying 
stakeholders and consumers through messages on the website and outreach to media to 
potentially expect some problems.  As one OC official noted, It was never a matter of 
whether we moved forward, it was only how to message about what the public and others 
could expect to reduce poor reactions.   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           28  Final Countdown to Launch | January 2013September 2013  
CMS moved HealthCare.gov forward to launch with an interim  authorization to operate and 
concerns about incomplete  security testing.   Federal  guidelines  require  that  a  senior  official  or  
executive  assume  responsibility  for  operating  an  information  system  at  an  acceptable  level  of  
risk  by  signing  an  Authorization  to  Operate  (ATO).70   ATOs  are  typically  signed  by  the  CMS  CIO,  
last  for  13  years,  and  include  supporting  documentation  of  security  testing.71   Each  part  of  
the  Federal  Marketplace  system  (e.g.,  FFM,  Hub,  and  EIDM)  required  an  ATO  before  
operating.   Full,  3year  ATOs  for  the  EIDM  and  Hub  were  completed  and  signed  by  the  
CMS  CIO  (on  March  22,  2013,  and  September  6,  2013,  respectively).  
Authorization    On  September  24,  2013,  just  days  before  the  launch,  CMSs  chief  information  security  officer  
to  Operate    raised  concerns  that  the  FFM  did  not  reasonably  meet  CMSs  security  requirements,  citing  
is  a  declaration  stating  
specific  concerns  about  the  lack  of  security  testing.   Given  this  information,  and  the  high  
publicly  that  the  launching  
profile  of  HealthCare.gov,  the  CMS  Administrator,  CIO,  and  COO  determined  that  a  higher  
organization  (e.g.,  CMS)  
official  than  the  CIO  should  sign  an  interim  (shortterm)  ATO  that  would  require  completed  
has  deemed  the  
functionality  sufficient   security  testing  within  6  months  of  launch.   On  September  27,  2013,  the  CMS  Administrator  
and  is  accepting  any   signed  the  interim  ATO,  which  allowed  the  FFM  to  operate  for  6  months  provided  that  the  
associated  risk  to  the  
security  risks  were  reduced  by  employing  a  mitigation  plan  that  included  completing  
organizations  operations  
additional  security  testing  and  installing  continuous  monitoring.   Additional  security  testing  
or  to  others  involved.  
was  conducted  in  December  2013  and  a  full,  1year  ATO  was  signed  by  the  CMS  CIO  on  
March  12,  2014.    
Shortly before launch, CMS determined that the system had much lower capacity than 
anticipated and requested that the contractor double capacity in 3 days.   On  September  26,  
2013,  CMS  technical  officials  visited  CGI  Federal  offices  to  assess  progress  on  the  FFM  build  
and  conduct  testing.   They  conducted  limited  performance  testing  and  determined  that  the  
website  capacity  could  support  far  fewer  concurrent  (simultaneous)  users  than  planned.   The  
low  capacity  available  was  in  part  because  the  software  code  required  more  infrastructure  
capacity  for  execution  than  CMS  anticipated.  
According  to  CMS  and  contractor  staff,  the  CMS  officials  drove  immediately  to  the  offices  of  
72  
Terremark,  the  main  contractor  for  HealthCare.gov  computing  capacity  and  infrastructure,  
 hours 
and  informed  its  project  managers  that  the  project  would  require  double  the  capacity  already  
before  launch  of  
purchased  by  CMS  for  the  launch.   A  Terremark  manager  reported  in  interviews  that  The  
HealthCare.gov,  CMS  
requested  the   request  was  to  double  everything  72  hours  from  launch.   We  had  done  an  extremely  large  
contractor  to  double   build  for  months.   We  were  pulling  gear  from  all  over  the  world,  renting  planes  to  get  
computing  capacity   hardware  here  that  was  intended  for  other  clients.   The  request  for  a  72hour  buildup  was  
on  September  26,  2013.  
made  on  September  26  (4.5  days  from  launch)  to  allow  for  at  least  1  day  of  testing.   Terremark  
added  more  than  double  the  capacity  during  this  time,  and  by  September  29,  testers  indicated  
that  the  concurrent  user  limit  was  raised  substantially.   A  CMS  technical  official  communicated  
to  the  CMS  Administrator  and  U.S.  CTO  over  the  course  of  September  2930  that  capacity  was  
no  longer  a  critical  problem.  
   
OEI061400350             HealthCare.gov:   Case  Study  of  CMS  Management  of  the  Federal  Marketplace            29   Final Countdown to Launch | January 2013September 2013  
Technical  problems  continued  in  the  final  days  before  launch.   The  EIDM  account  and  identity  
management  system  suffered  outages  on  September  29,  and  contractors  reported  later  that  
they  anticipated  continued  problems.   (See  Figure  2  for  a  timeline  of  the  final  countdown  to  
launch,  JulyOctober  2013.)   Still,  the  CMS  Administrator  emailed  the  U.S.  CTO  on   
September  29  that  the  website  would  be  ready  to  launch  on  October  1.   CMS  officials  later  
reflected  that  they  were  nervous  about  the  launch,  but  were  still  excited  to  move  forward  and  
that  they  did  not  understand  the  depth  of  the  technical  problems  or  predict  the  poor  
outcome.   As  one  CMS  official  reflected  later,  there  was  a  sense  that  it  is  always  like  this  on  
major  projects,  with  tight  deadlines  and  complex  delivery.  
Figure  2:   Timeline  of  Final  Countdown  to  Launch,  JulyOctober  2013.  
Source: OIG analysis of CMS project management documents, 2015.  
OEI061400350             HealthCare.gov:   Case  Study  of  CMS  Management  of  the  Federal  Marketplace            30   Final Countdown to Launch | January 2013September 2013 
KEY FACTORS CONTRIBUTING TO BREAKDOWN 
  COMPRESSED TIMEFRAME for TECHNICAL BUILD 
The final months of development and implementation for HealthCare.gov were chaotic for CMS staff and 
contractors.  The 9 months from JanuarySeptember 2013 provided, from the outset, very little time to 
accomplish the tasks remaining.  These included tasks critical to success, such as testing website functionality 
and security, and ensuring adequate capacity for users.  Changes in policy and scope continued into early 
2013, with the States deadline to establish their own marketplaces or join the Federal Marketplace moved to 
December 2012 and many decisions remaining regarding the content of the website.  CMS made changes to 
business requirements and technical specifications well into 2013, delaying development to a point where it 
was not feasible to complete and test the website as initially planned. 
  RESISTANCE to BAD NEWS 
CMS leadership and staff were warned of trouble prior to the launch of HealthCare.gov, both formally with 
reports from outside entities hired to assess the project and informally, through meetings and emails.  
Despite this awareness, those knowledgeable at CMS did not ensure that the bad news prompted 
appropriate change.  CMS leadership and staff took little action to respond to warnings, remaining overly 
optimistic about the launch, and developing few contingency plans.  As the project degraded further and 
problems became more well-known, CMS officials and staff appear to have become desensitized to bad news 
about progress.  The problems were layered and complex, and information became unwieldy and difficult to 
prioritize.  Also, the CMS officials were used to problems implementing large projects, particularly with 
technology, causing them to fail to recognize the extent of problems with HealthCare.gov. 
  PATH DEPENDENCY 
In early 2013, problems with the HealthCare.gov build deepened and CMS did little to improve management.  
Through most of the year, CMS continued with the same plans for a full launch and even added an early 
implementation of the Account Lite creation function.  Given the technology and complex systems involved, 
changing the projects path would have required a leader or team to conduct a comprehensive assessment of 
status, and to either possess the authority to alter tasks and processes or to fully communicate that 
assessment to leaders with that authority.  Absent this, CMS staff and contractors continued with the initial 
strategy and goals, falling further behind schedule, with largely the same leadership, staff, and plan. 
  CORRECTIONS WEAK and LATE 
By the time CMS took action to change the projects path in August and September of 2013, it was too late to 
adequately affect change given the substantial need for progress and improved execution.  For example, the 
CMS Administrator placed the CMS COO as head of the project without establishing a clear agenda or 
communicating the decision to the full team.  CMS cut functionality that was at one time considered critical 
to a successful launch, such as the Spanish language and SHOP websites, to divert resources to the main 
build.  This occurred in the last few weeks before launch, when developers and testers reported they were 
months behind.  The rush to launch affected all aspects of the build, including moving forward with only an 
interim authorization to operate and requesting double computing capacity late in September.  Leaders 
sought to deliver minimal functionality but without a comprehensive and thoughtful strategy.  The 
corrections were too weak and late to avert the poor outcome. 
  
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           31  Launch, Correction, & First Open Enrollment | October 2013March 2014 
CMS didnt need a technological surge,  
we needed an organizational surge. 
CMS Official 
C 3 
HAPTER 
 
LAUNCH, CORRECTION, FIRST
   &   
OPEN ENROLLMENT 
 
October 2013March 2014 
 
The HealthCare.gov launch quickly revealed multiple problems with the website, 
and initial efforts to fix the problems were hampered by lack of coordination 
Problems with HealthCare.gov were apparent immediately after launch.  On October 1, 2013, 
HealthCare.gov experienced 250,000 concurrent users, much greater than the planned 
capacity.  Website outages began within 2 hours of launch, preventing many consumers from 
logging in and signing up for health insurance.  In reporting to the public, the U.S. CTO 
attributed the problems to high volume, which was five times the number of simultaneous 
users anticipated.72  It was soon clear to CMS and contracted operators that the 
HealthCare.gov issues were not caused solely by a higher number of visitors to the website 
but also by core problems in website 
performance.  In the end, only six 
consumers were able to submit an  We were sitting in the office at 
application and select a plan on the first 
midnight when it started running 
day of the first open enrollment.  The 
and it wasnt looking good.  
problems at launch created a firestorm of 
negative stakeholder response and media  Everything was turning red on our 
attention chronicling the website  screens.   
 EIDM contracted staff
problems.  Within days of the 
HealthCare.gov launch, CMS leadership 
dispatched additional CMS and contractor staff to be onsite at the CGI Federal command 
center in Herndon, VA to correct software defects and improve system performance.  Those 
onsite were largely staff from CMS and CGI Federal, joined gradually by additional staff from a 
number of contractors involved with the build.  
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           32  Launch, Correction, & First Open Enrollment | October 2013March 2014  
Homepage  of  HealthCare.gov  at  launch,  October  1,  2013.  
 
CMS  and  contractors  identified  the  most  immediate  performance  problem  as  the  EIDM,  the  
website  entry  system  used  for  establishing  accounts  and  verifying  consumer  identity,  a  
problem  caused  in  part  by  CMS  not  adequately  communicating  with  contractors  about  overall  
system  functionality.   The  lack  of  capacity  on  the  part  of  the  EIDM  created  a  bottleneck  to  
consumers  reaching  website  functions  and  information.   As  one  CMS  official  explained,  It  was  
like  having  a  small,  onelane  onramp  to  a  major  highway.   The  EIDM  bottleneck  also  caused  
problems  for  developers;  since  the  EIDM  served  as  an  entry  point  to  the  website,  coders  
assigned  to  fix  errors  could  not  easily  access  the  website  themselves  to  see  and  correct  other  
problems.   As  one  CMS  technical  official  reported,  The  FFM  was  actually  the  bigger  problem,  
but  we  could  not  see  the  magnitude  of  the  coding  problems  in  the  FFM  and  begin  fixing  until  
we  got  through  the  EIDM  problem.   CMS  directed  QSSI  to  fix  the  EIDM  and  at  the  same  time  
directed  CGI  Federal  to  construct  a  new  portal  for  website  entry  and  account  creation,  to  be  
used  if  QSSI  was  not  able  to  fix  the  EIDM.   The  work  on  a  new  portal  expended  CGI  Federal  
coding  time  that  could  have  been  devoted  to  other  website  fixes,  but  an  HHS  technical  advisor  
reflected  later  that  the  redundancy  was  a  reasonable  strategy  given  the  importance  of  the  
website  entry  function  and  uncertainty  about  the  viability  of  the  EIDM.   
Software,  Design   QSSI,  the  contractor  responsible  for  developing  the  EIDM,  reported  later  that  they  did  not  
Defects  Cripple    know  that  all  visitors  to  the  website  would  have  to  enter  through  the  EIDM  system  and,  
HealthCare  Website   therefore,  underestimated  the  capacity  needed.   QSSI  officials  reported  in  interviews  that  they  
The  Wall  Street  Journal   believed  the  anonymous  shopper  tool  would  enable  individuals  to  view  health  plan  
10/6/2013  
information  without  creating  an  account,  and  that  CMSs  decision  to  postpone  
implementation  of  the  tool  contributed  to  the  EIDM  experiencing  heavier  than  expected  
traffic.   CMS  officials  later  confirmed  that  the  (poorly  named)  anonymous  shopper  tool  would  
not  have  been  truly  anonymous  and  would  have  required  individuals  to  create  an  account  
through  the  EIDM  even  if  they  only  wanted  to  view  plans.   (Later  in  October,  CMS  launched  a  
limited,  temporary  shopping  tool  to  allow  consumers  to  view  health  plan  information  without  
establishing  an  account.)  
OEI061400350             HealthCare.gov:   Case  Study  of  CMS  Management  of  the  Federal  Marketplace            33   Launch, Correction, & First Open Enrollment | October 2013March 2014 
Initial efforts to fix the website were hampered by lack of information and coordination among 
CMS and contracted operators.  Also hampering diagnosis and correction of the website was a 
lack of coordination among CMS and contractors in monitoring website performance.  CMS 
technical staff later reported that ensuring comprehensive and coordinated monitoring was 
not a priority before the launch because resources were focused on completing the build of 
the website.  CMS, CGI Federal, QSSI and other contractors continued adding staffing 
resources prior to and following the launch, but there was no clear entity or system to 
coordinate and monitor their efforts.  A CMS technical official reported that prior to the 
launch, contractors were responsible for monitoring their own systems and unable to see 
other systems, which was a problem because the systems worked together.  As a result, there 
were teams from CMS and contractors in different locations functioning as separate 
command centers, including the CGI Federal command center in Herndon and a QSSI 
command center in Columbia, MD.  These centers had a variety of tools for monitoring 
outages, response time, and errors in loading and processing information, but there was not a 
single, systems-wide leader or team with an overview of the project and ability to take action.  
While the CGI Federal command center was still considered the primary location for operating 
HealthCare.gov, it did not house all CMS and contracted staff relevant to the websites 
operations.  It did not include development teams or contractors responsible for the Hub, 
EIDM, and data centers, and it did not have an entity serving formally as systems integrator to 
coordinate these functions.   
Federal Government-wide shutdown further complicated CMSs management of the launch.  
The morning of the launch, the Federal Government was shut down due to lack of funding.73  
The Government shutdown was not certain until late the prior evening.  Thus, CMS officials 
had to manage both the launch and implementing the shutdown, such as distributing notices 
to staff and managing orderly shutdown processes.  The shutdown lasted for 16 days and 
affected critical Federal Marketplace staff, complicating implementation.  CMS officials 
reported that staff in key offices responsible for Federal Marketplace functions, including 
CCIIO, OIS, and OC, was reduced to approximately one-third its pre-shutdown levels and in 
some subgroups, closer to one-tenth of pre-shutdown levels.  Contractors continued to work 
on HealthCare.gov during the shutdown, but they reported that they were hampered by CMS 
staff not able to work and by the general disruption of processes and communication brought 
on by the shutdown. 
After initial difficulties, CMS and contractors worked with outside experts to repair 
HealthCare.gov, instilling changes in the project culture and work processes 
CMS reconfigured HealthCare.gov operations to improve the website, establishing clearer 
leadership and consolidating technical 
operations.  On October 22, three weeks 
This was the hardest thing I have 
after the launch, HHS announced new 
ever done and I hope nothing ever 
leadership for the HealthCare.gov fix, 
appointing a well-known Federal manager  comes close to it again. 
to oversee efforts to improve the website   
Member of ad hoc technology team
and facilitate enrollment.74  At the same 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           34  Launch, Correction, & First Open Enrollment | October 2013March 2014 
time, the U.S. CTO recruited an ad hoc technology team comprised of several leaders and 
engineers from top technology firms who were not involved with the Federal Marketplace 
development.75  HHS and CMS officials made improving the website a top priority.  The 
Administration assigned an official with business operations experience as the project lead, 
responsible for managing daily operations and reporting back to White House staff.  CMS also 
filled the Deputy COO position, which had remained vacant leading up to October 1, to 
provide assistance to the COO.  Some from the ad hoc technology team also took lead roles, 
identifying problems and organizing daily tasks.  In combination with the HHS and CMS 
leadership already involved, this made for a multi-layered group of managers and advisors 
that could have resulted in difficulties determining which leader would make key decisions.  
Yet all interview respondents involved in this period reported that it was a productive and 
efficient environment, with few if any problems of delegation or workflow.  
According to CMS staff and contractors, the productive environment was due to several 
factors:  the willingness of CMS and CGI Federal staff to open their work processes to input 
and work side-by-side with the new ad hoc technology team; the tight structure and discipline 
brought by the new leadership and engineers, several with extensive expertise working with 
successful private sector companies; and the shared sense of mission and urgency involved to 
make the website work for consumers.  As observed by a leader of the ad hoc technology 
team, Some of the very best engineers and troubleshooters in the world willingly put their 
lives on hold to dedicate their time to this very difficult problem. . . .  They found Government 
officials and contractors, who also wanted nothing more than to fix the site and who were 
ready and willing to work together.76  Similarly, CGI Federal managers reported that their 
staff felt strongly about fixing the website and finishing the job we started.  Still, the task at 
hand was difficult and those onsite experienced many successes and failures in improving the 
software and systems. 
By late October, CMS and contractors began to move command center operations residing at 
the CGI Federal facility in Herndon and other locations to QSSIs Columbia location, 
establishing what would ultimately become the formal HealthCare.gov command centerthe 
Exchange Operations Center (XOC).  The structure at the XOC was based on active 
coordination between technical and policy staff, a key component missing during the website 
preparation and development.  The structure ensured that technical solutions aligned with the 
functionality consumers needed to apply for and select plans.  A member of the ad hoc 
technology team explained, We had policy folks figuring out the error messages. What did 
the code do or not do? What was supposed to happen?  Lack of tools for website monitoring 
was still a problem for those repairing the website, particularly as it concerned consumer use.  
A member of the ad hoc technology team 
noted that in the absence of comprehensive 
There was no place to look to find 
monitoring tools, There was no place to 
out whether the site was up today 
look to find out whether the site was up 
today or not except CNN, which was literally  or not except CNN. 
  
how we found out about problems a good   
Member of ad hoc technology team
part of the time in the beginning.77  
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           35  Launch, Correction, & First Open Enrollment | October 2013March 2014 
Contractors and the ad hoc team of technical experts recommended that CMS obtain 
monitoring tools for a variety of functions, including website traffic, capacity use, speed of file 
transfers to States, and website security.  During the period of October 1December 1, 2013, 
and shortly after, CMS purchased new monitoring tools that provided at-a-glance statistics 
(dashboard) of website performance. 
Badgeless Culture is  The technical staff were divided by function, with one group focused on capacity and speed, 
a term used by CMS to 
and another on defects in the software code.  Assignments were made by skill and availability, 
signify all CMS and 
irrespective of whether the person was employed by CMS or any of the contractors.  This 
contracted staff operating 
effort was the beginning of a badgeless, titleless culture at the XOC, meaning an 
as a team regardless of 
their employer status or  environment in which all staff were to operate as a team regardless of their job title or 
job title.  whether they were a CMS employee or a contractor.  Nontechnical CMS staff and contractors 
were also present at the XOC, or actively communicating with technical staff located at the 
center.  For example, the call centers were linked to the XOC to provide information about 
problems their staff heard about from consumers. 
CMS made key decisions to build on current systems rather than create new ones, to develop 
contingency plans, and to hire a technical systems integrator.  CMS leadership, in consultation 
with the ad hoc technology team, made a key early decision to refactor (correct and 
streamline the code for) the existing software code rather than rebuild from scratch.  Those 
involved reported that the decision was based on the tremendous time investment required 
to start over and the willingness of CMS and contractor staff to work together with the ad hoc 
technology team to identify and solve problems.78 
At the same time these repairs were underway, CMS began to develop redundancies for core 
operations in the event of future problems.  CMSs contingency plan up to this point was to 
ask consumers to fill out paper applications through call-center assistance if the website went 
down.  The new contingency planning focused on establishing redundant systems to keep the 
website up if the primary systems failed.  CMS hired separate groups of developers to begin 
creating new systems to replace three key components of HealthCare.gov.  These were an 
account creation system, the Scalable Log-In System (SLS) that, unlike the CMS-wide EIDM 
(the EIDM was developed to support multiple CMS programs), was created exclusively for 
HealthCare.gov; a streamlined application, Application 2.0 (App 2.0); and Plan Compare 2.0, a 
new shopping tool that provided more robust information about health plans and premiums, 
and did not require consumers to create an account.  Lastly, CMS hired three companies to 
run data centers to provide system capacity, placing various data functions at different 
centers so that, as a CMS official stated, CMS would not put all our eggs in one basket and 
have a greater range of data resources to call upon.  
A key decision by CMS during this time was to hire QSSI as technical systems integrator to 
serve as an advisor in coordinating technical tasks and resources.  By all accounts, this action 
led to greater coordination between and among contractors and CMS technical staff, and 
enabled project leaders to more quickly and clearly identify and correct problems and allot 
resources.  The job of the systems integrator is to coordinate operations, ensuring that those 
responsible for various aspects of the project communicate their activities, schedules, and 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           36  Launch, Correction, & First Open Enrollment | October 2013March 2014 
needs to each other.  Some in CMS and outside the agency had raised the idea of contracting 
a systems integrator at various times 
since 2011, but the concept was not 
widely discussed.  According to CMS  We put eyes on everything.  We 
officials, for past projects such as the  deployed tools that enabled us to see 
Medicare Part D implementation, 
ahead of time if the server would get 
CMS coordinated contract activities 
overloaded.  We threw everything up on 
themselves, but the agency had not 
implemented a project with the  a monitor to see what was going on. 
scope and complexity of the Federal   
QSSI, technical systems integrator
Marketplace.  A CMS official reported 
that, in hindsight, CMS had a difficult 
time performing the systems integrator role and that it plans to keep a HealthCare.gov 
systems integrator for the foreseeable future. 
HHS officials provided information to Congress and the public.  At the same time as CMS staff 
and contractors were fixing the website, the HHS Secretary and White House staff were 
managing much of the public fallout following the launch.  The website problems were 
front-page news and generated substantial debate among all manner of stakeholders, 
including Congress.79  This culminated in 10 congressional hearings before the end of 
  November, with testimony from the HHS Secretary, CMS staff, IRS, and contractors.80 
 Nobody could   During this time, the CMS OC was responsible for providing updates to the White House, HHS, 
have felt worse than the  and the public, holding daily briefings to indicate project status.  OC also managed the call 
people at CMS did.  It was 
centers, adding thousands of call center operators after the first week of operations and 
their responsibility to 
transmitting information to technical staff about problems reported by website users to the 
stand it up.  They want to 
call-center staff.  The website difficulty also affected the call centers and staff in external 
never have this happen 
again. HHS Official  organizations that helped consumers enroll, such as navigators.  In interviews and external 
studies, navigators reported that technical difficulties with the website and long wait times at 
the call centers increased the amount of time needed with each applicant.  One external study 
found that approximately 25 percent of assisters (such as navigators) spent more than  
2 hours, on average, with each applicant.81  Developers also made improvements that 
instructed consumers when the online application was unavailable and provided the call 
center number as an alternative for consumers to apply for coverage. 
HHS announced an improvement benchmark, that the website would relaunch and work 
smoothly for the vast majority of users by the end of November, a date many believed was 
necessary, given that the ACA required coverage by January 1, 2014.82  Public reports 
indicated that the new target date was made on the basis of assurances that the ad hoc 
technology team made to the White House.83 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           37  Launch, Correction, & First Open Enrollment | October 2013March 2014 
HealthCare.gov consumer message, November 10, 2013. 
 
Those responsible for repairing the website reported that although daily operations of fixing 
and operating the website were going fairly smoothly, lack of communication among some 
HHS and CMS senior leadership was still a problem.  For example, CMS leaders at the XOC 
reported that they were not involved in discussions about whether the end of November was 
reasonable from a technological perspective, and that HHS informed them only after the 
decision was made.  The deadline created further pressure to improve, with one HHS official 
later reflecting in an interview that the December 1 assurance for improvement seemed 
ambitious.  Sporadic outages of Terremarks equipment challenged these efforts, including 
two outages in late October 2013 that led to website downtimes of 24 and 36 hours. 
HealthCare.gov downtime message, October 30, 2013. 
 
H  ealthCare.gov 
 
 
 
The system is down at the moment. 
 
We are experiencing technical difficulties and hope to have them resolved soon.  Please try again later. 
 
In a hurry? You might be able to apply faster at our Marketplace call center.  
 
Call 1-800-318-2596 to talk with one of our trained representatives about applying over the phone. 
 
 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           38  Launch, Correction, & First Open Enrollment | October 2013March 2014  
Inadequate  website  capacity  continued  to  be  a  challenge,  limiting  computing  and  storage  
infrastructure  for  developers  to  make  changes.   By  midNovember,  Terremark  had  increased  
capacity  to  a  point  at  which  these  challenges  were  lessened  and  progress  accelerated.84   The  
work  to  improve  website  performance  continued  at  a  strenuous  pace,  with  some  CMS  staff  
and  contractors  reporting  they  slept  in  nearby  hotels  and  worked  24hour  shifts.   In  the  
meantime,  HealthCare.gov  went  down  several  times,  and  press  reports  and  public  dialogue  
were  highly  critical  of  the  website.   As  one  CMS  official  stated,  those  were  dark  days.   CMS  
also  continued  to  reduce  the  scope  of  the  HealthCare.gov  build,  for  example  announcing  on  
November  27,  2013,  that  the  SHOP  website  promised  for  that  month  would  be  delayed  until  
the  second  open  enrollment  period  a  year  later.85  
CMS  and  the  expanded  technical  team  improved  HealthCare.gov  by  December  1,  
2013  and  continued  improvements  through  early  2014   
Even with  HealthCare.gov substantially improved, CMS faced large challenges to further 
improve operations.   CMS  improved  website  performance  by  December  1,  2013,  as  promised  
by  HHS.   Also  in  December,  CMS  enhanced  its  shopping  tool  that  allowed  consumers  to  shop  
for  health  plans  without  creating  an  account.   CMS  staff  and  contractors  continued  website  
corrections  through  the  end  of  first  open  enrollment  (March  31,  2014).   The  proportion  of  
time  that  the  website  was  functioning  went  from  42  percent  in  early  November  to  over  90  
percent  at  the  end  of  November  (see  Figure  3).86  
Figure  3:   Percent  of  time  HealthCare.gov  functioned  during  November  2013.  
 
Source: CMS, HealthCare.gov Progress and Performance Report, December 1, 2013.  
The  website  could  now  handle  more  than  35,000  concurrent  users  without  crashing,  and  
technical  glitches  were  less  frequent.87   The  XOC  had  projectwide  monitoring  systems  and  
website  performance  dashboards.   When  capacity  was  overloaded,  CMS  could  place  
consumers  in  waiting  rooms  that  inhibited  navigating  further  to  complete  tasks  until  website  
traffic  was  reduced.   When  placed  in  a  waiting  room,  consumers  were  given  an  option  to  
receive  email  notification  to  return  to  the  website  when  capacity  was  available  or  to  contact  
the  call  center  to  apply.  
   
OEI061400350             HealthCare.gov:   Case  Study  of  CMS  Management  of  the  Federal  Marketplace            39   Launch, Correction, & First Open Enrollment | October 2013March 2014 
HealthCare.gov waiting room message, December 23, 2013. 
 
HealthCare.gov 
 
HealthCare.gov has a lot of visitors right now! 
 
We need you to wait here, so we can make sure theres room for you to have a good experience. 
  If you are trying to enroll by the December 23rd deadline for coverage beginning on January 1 
and cant finish because our system is busy, we will still help you get the coverage you need. 
 
While you wait, here are some things you can do to get ready to enroll: 
  Gather important documents and numbers:  heres a list 
Browse:  health plans available in your area 
Find out if you might be able to get lower costs using our quick calculator 
 
 
Cant wait? 
   Leave your email  and well send you a one-time message when HealthCare.gov is ready for you to return. 
 
  Email address   
SKIP TH E WAIT 
   
Remember, you can apply right now at our Marketplace call center. Call 1-800-318-2596 to talk with one of our trained 
  representatives about applying over the phone. 
 
 
As the technical staff fixed defects and improved performance, staff at CCIIO and OC were 
focused on retaining and assisting consumers by resolving problems with individual cases 
identified by call centers, navigators, and issuers.  CMS also extended the period for signing up 
for health insurance coverage that would be effective beginning January 1, 2014, by 9 days 
(shifting from December 15, 2013, to December 24, 2013) so that those who encountered 
difficulty enrolling could have more time.88  On the final day, HealthCare.gov experienced 
49,156 concurrent users, and approximately 1.2 million consumers selected a plan through 
HealthCare.gov by December 28, 2013.89 
As the end of the first open enrollment period neared, other issues emerged.  As of February 
2014, CMS reported that the Federal Marketplace was unable to resolve about 2.6 million of 
2.9 million application inconsistencies, which occur when a marketplace cannot verify an 
applicants information (e.g., Social Security number, citizenship, income, family size) through 
available data sources, such as IRS or SSA.90  CMS reported that as of February 2014, the 
Federal Marketplace computer systems lacked the capability to resolve inconsistencies and 
that CMS would retain any documents submitted to resolve inconsistencies until CMSs 
eligibility system had that capability.91,92,93  In addition, subsequent OIG work determined that 
not all of the Federal Marketplaces internal controls were effective in ensuring that 
consumers were properly determined eligible for health plans and insurance affordability 
programs.94  For example, OIG found that the Federal Marketplace did not always validate an 
applicants Social Security number when the applicant provided the number at the end of, 
rather than the beginning of, the application process.95 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           40  Launch, Correction, & First Open Enrollment | October 2013March 2014 
After January 1, 2014, and through the remainder of the first open enrollment period, CMS 
and contractors continued to work on system improvements, preparing for the expected 
surge of HealthCare.gov users enrolling before the March 31, 2014, deadline.  After meeting 
the improvement benchmark on December 1, 2013, CMS finalized the designation of the XOC 
in Columbia as the command center for CMS staff and contractors supporting HealthCare.gov.  
CMS developed a new and more standardized routine for monitoring the website and 
managing contracted work, with the XOC serving as the focal point for monitoring and 
coordinating systems during the remainder of the first open enrollment period.  The XOC was 
staffed by CMS technical staff and contractors involved with multiple aspects of the system 
(e.g., FFM development, EIDM operations, website capacity, security), and operated 24 hours 
a day during open enrollment.  The XOC also employed new monitoring tools, including an 
application monitoring tool that updates information about website functionality every 
5 seconds.  These tools allowed the staff to identify problems more quickly and acquire 
baseline data to track further performance improvements. 
CMS and contractors reflected later that this period represented a shift away from putting 
out fires and toward establishing standard operating procedures that would give the work 
more form and structure.  As during policy development in 20122013, CMS temporarily 
reassigned staff from other divisions to the Federal Marketplace to supplement technical and 
contractor staff.  Some technical problems continued throughout the first open enrollment 
period, with another outage of EIDM in March 2014 placing incoming users in waiting rooms 
for up to 45 minutes.  To better address challenges as they arose, CMS reorganized its 
management structure to create formal chains of communication for task areas such as 
eligibility and data centers to help address these concerns.  CMS and contractors set up 
integrated teams, called towers, centered on a single project area, such as eligibility. 
CMS also more tightly coordinated changes with contractors, in particular aligning contract 
obligations to meet changing needs, what HHS refers to as change control.  The essence of 
change control is that the contractor and agency both agree to changes in the contractors 
scope of work and document how those changes play out in cost and deliverables.  The 
Change Control Board began meeting daily (compared to initial development, when meetings 
were infrequent and often cancelled) to review system changes, and coordinated with 
contractors to prioritize fixes that would have the highest positive impact.  As a CMS technical 
official stated of the prelaunch time period, In the beginning, we were too busy making sure 
there was even a product to launch to properly track changes. . . .  [After the launch,] the 
problems came from so many directions . . . we worked with the Change Control Board to 
make a list of the most important things to fix. 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           41  Launch, Correction, & First Open Enrollment | October 2013March 2014 
CMS hired a new FFM software contractor, restructuring the FFM contract to better account 
for high-quality delivery.  At the same time CMS hired QSSI as the HealthCare.gov systems 
integrator, CMS officials decided to consider other contractors to replace CGI Federal.  CMS 
expedited the contracting process by issuing a letter contract that was awarded without a 
full and open competition.  Ultimately, the letter contract was finalized into a sole source 
contract that describes justifications for other than full and open competition.  The new FFM 
contractor, Accenture, had experience with several State marketplaces and had previously 
built the IRS connection to the Hub.  The company 
presented a proposal to CMS in December 2013 
We were equal partners. . . .  
that focused on increasing the technical discipline, 
We knew that we were all 
clarity of leadership, and decisionmaking authority 
of leaders.  They began work in January 2014 by  successful or no one was 
shadowing CGI Federal staff.  CGI Federal  successful.    
Accenture staff
officials reported that they learned of the switch 
from reading the Washington Post just days before 
Accentures arrival. 
The transition operated in three phases:  first, CGI Federal continued in the lead role through 
February with Accenture in support; second, Accenture acted as the lead with CGI Federal in 
support for March; and third, after the close of first open enrollment, Accenture fully 
performed tasks and consulted with CGI Federal as technical experts as needed.  Managers 
from both contractors reported the transition went smoothly, with CGI Federal dedicated to 
assisting and Accenture to learning and improving the project.  Staff from both companies and 
CMS reported pressure from the public conversation.  As one contractor noted, This was the 
most visible project in the world.  Part of Accentures task was to refactor the software and 
institute a way of overseeing and running the project to facilitate rapid decisionmaking. 
The Federal Marketplace project and development of HealthCare.gov came at a 
substantial cost financially and organizationally to CMS  
CMS met key enrollment goals for first open enrollment, but incurred higher than expected 
costs in contracts and fees paid for correcting defects.  CMS ended the first open enrollment 
period with 5.4 million individuals having selected a plan through the Federal Marketplace, a 
technologically workable system (albeit with limited functionality), a clearer policy framework, 
and a stronger management structure.96  However, the attention to the Federal Marketplace 
effort (including reconstruction of the website systems and other compensatory actions 
resulting from the launch) had costs for CMS, its programs, and staff. 
CMS contract costs for the Federal Marketplace project were ultimately much higher than 
initially estimated by CMS, but the total contract amount would have been expected to rise 
from the initial allotment.  CMS originally estimated the contract value for six key Federal 
Marketplace contracts to be $464 million.97  As of early 2014, CMS had updated the estimated 
value of these contracts to $824 million.98  The value more than tripled for the FFM contract 
awarded to CGI Federal, from $58 million to $207 million.99  In addition, the value for the Hub 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           42  Launch, Correction, & First Open Enrollment | October 2013March 2014 
contract more than doubled, from $69 million to $180 million.  The remaining four contract 
values increased between 1 and 54 percent.100 
The initial estimated value of a contract may 
increase after award for a number of reasons,  We are defined by the person 
including tasks added to the contract or 
who couldnt enroll, not the  
increases in the cost of scheduled work.  CMS 
99 percent who could enroll. 
reported that contract requirements changed 
 
during the implementation of the Federal  CMS official
Marketplace and that not all of these 
requirements were known at the time of 
award.  That said, some of the increased costs were fees that CMS paid to contractors for 
correcting defects.  Also, CMSs decision to select a cost-plus-fixed-fee contract with CGI 
Federal for the FFM contract under which CMS assumed the risk for cost increases meant that 
it was obligated to pay all allowable costs associated with correcting defects.  Previous OIG 
work determined that from October 2013February 2014, CMS paid CGI Federal for charges 
associated with hours worked to correct defects for work associated with the FFM contract.101  
The website breakdown and recovery effort also affected CMS staff, many of whom worked 
under high pressure for long periods or were redirected from other CMS programs.  CMS staff 
in CCIIO and OIS in particular had been working long hours since receiving the Federal 
Marketplace program from HHS in 2011, working under tight deadlines on difficult aspects of 
policy development and the technical build.  Now in 2014, both CMS staff and contractors 
reported difficulty maintaining focus and energy.  Compounding this were increased staff 
turnover and vacancies, causing a greater workload for those that remained and also a loss of 
organizational knowledge and relationships.  For example, as previously stated, one Directors 
position in CCIIO was filled by seven different people during our review period. 
Focus on HealthCare.gov also required CMS to redirect staff from other CMS programs.  CMS 
assigned staff from other divisions to help build and repair the Federal Marketplace.  CMS 
officials said this resulted in a fairly modest cost to other programs, with one senior official 
observing:  We have enough capacity to absorb extra work for a while.  We were not cut to 
the bone in an organization as big as ours.  In assessing this impact, it is important to note 
that the effect of diverted staff may have been somewhat amplified by more general 
reductions in CMS staff.  According to agency documentation, the number of staff in all CMS 
offices decreased by about 20 percent between 2011 and 2014, due in large part to a 
CMS-wide hiring freeze (and normal attrition, such as retirements).  Whether due to diverted 
staff or overall staff reductions, CMS officials noted that fewer staff required that 
management strategically prioritize what had to be done by postponing some items. 
A sense of CMS staff fatigue is unlikely to be abated in the near future with an aggressive 
schedule for the Federal Marketplace planned through 2016.  CMS staff and contractors 
reported great difficulty completing FFM-related tasks under current time and resource 
constraints and expressed concern that the prolonged high intensity work could hamper 
successful operations in the short- and long-term.  CMS leadership indicated an awareness of 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           43  Launch, Correction, & First Open Enrollment | October 2013March 2014 
these problems and have encouraged staff to take earned leave, communicate more openly, 
and set realistic deadlines that recognize limitations.  Leaders have also taken steps to 
improve morale by recognizing staff who performed well in challenging positions and by 
establishing and maintaining clearer program objectives and guidance.  Still, some key staff 
have left CMS or transferred out of the Federal Marketplace program to avoid the strain, and 
  others reported they may do so after they feel the program is more fully established. 
  The imperative   Many CMS officials and staff reported in interviews that they stayed at CMS despite the large 
for CMS staff was not   workload, pressure, and reassignments because they were committed to seeing a successful 
the publicity and the 
Federal Marketplace project.  HHS and CMS staff reported that one of the most painful 
embarrassment, it was 
aspects of the post-launch fallout during the first open enrollment period was that they 
the mission.  CMS is 
accustomed to taking  believed much of ACA implementation had gone smoothly, as had, in their view, 
hits.  We wanted to   nontechnological aspects of the Federal Marketplace, such as issuer participation and 
fulfill the mission.  establishment of benchmarks for essential health benefits.  While press reports and outside 
 CMS official  
stakeholders appeared overwhelmingly to perceive the launch as an unacceptable failure, a 
number of CMS technical staff in interviews defended not only the correction period and 
ongoing work on the website, but also the efforts leading up to the 2013 launch.  They alluded 
to the projects difficulty, arguing that the complex technology, fixed deadline, and multiple 
systems and stakeholders would have made execution difficult for any entity responsible.  
Several CMS officials and staff members who were brought into the Federal Marketplace 
program after the launch suggested that the difficulty of the project itself was as much a 
cause of the initial website failure as poor communication and project management, believing 
that people by-and-large did their best under tough circumstances.  Current CMS officials 
acknowledged CMS made major missteps, but they focused on learning from the experience 
rather than casting blame on individuals responsible. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           44  Launch, Correction, & First Open Enrollment | October 2013March 2014 
KEY FACTORS CONTRIBUTING TO RECOVERY 
  QUICK PIVOT to NEW STRATEGY 
CMS and contractors quickly brought in new staff and expertise following the launch, developing an 
all-hands environment wherein fixing problems with HealthCare.gov was the key agency mission.  Most of 
the additional staffing came to the project within 3 weeks, including technological and project management 
experts from CMS, contractors, and the private sector.  Working collaboratively under new leadership, the 
team simplified processes and consolidated operations.  These changes allowed CMS to make quick 
progress in identifying the source of problems and developing a strategy forward.  The team demonstrated 
a strong sense of urgency to take action, and quickly accepted new work processes.  The widespread 
attention to the launch and the number of parties involved could have created bureaucratic paralysis, but 
those working on the repairs directed their attention to immediate action and improved the HealthCare.gov 
website substantially in 2 months. 
  ADOPTION of a BADGELESS CULTURE 
The enhanced team of CMS staff, contractors, and technological experts correcting problems with 
HealthCare.gov included people at all levels of CMS and contracted entities, and with varied experience on 
the project.  Before the launch, artificial distinctions and divisions among staff contributed to poor 
collaboration, lack of communication, disjointed management, and slow progress.  Following the launch, 
first with the technological team and then more broadly, CMS promoted a horizontal culture that was 
badgeless and titleless, meaning all of those on the Federal Marketplace project were encouraged to 
collaborate as a single team, regardless of employer or job title.  CMS leadership promoted a culture 
wherein all team members could speak out about problems and develop creative solutions.  In interviews, 
CMS leaders and staff later reflected that this change in culture fostered a greater sense of mission and 
teamwork that further improved daily operations. 
  INTEGRATION of ALL FUNCTIONS 
The Federal Marketplace needed expertise and personnel across CMS, including policy, technical, 
contracting and communications staff, as well as many contractors.  Prior to the launch, some functions had 
no formal connection, despite their interdependency.  Key to the correction, CMS integrated the various 
functions both operationally and technically, improving daily work and promoting the larger project mission.  
CMS assigned clear project and technical leadership, and restructured its divisions to allow for greater 
visibility and oversight of technical staff and contractors by senior leadership.  This integration allowed CMS 
to identify and address problems more quickly, make informed decisions, and provide clearer direction to 
those involved in the website development and operations. 
  PLANNING for PROBLEMS 
CMS began to plan for and mitigate potential problems by considering contingencies, building redundant 
systems, and increasing capacity.  CMSs lack of contingency plans before the launch meant that CMS had 
few options when the functionality and computing capacity of HealthCare.gov encountered problems.  
Given limited resources, CMS leadership had to analyze past problems with HealthCare.gov and carefully 
consider how and to what extent it would develop new systems and strategies, such as enhancing training 
for call center staff.  Key to success was identifying all possible problems and developing systems and 
strategies specific to the concern. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           45  Turnaround & Second Open Enrollment | April 2014March 2015 
We dont only have a better product.   
We are a better organization. 
CMS Official 
C 4 
HAPTER 
 
TURNAROUND SECOND OPEN
 &     
ENROLLMENT 
April 2014March 2015 
In April 2014, CMS turned to preparing for the next open enrollment and added 
functionality, sharply prioritizing to limit scope and focus resources 
Ruthless  CMS worked to limit the scope of work required for second open enrollment.  By the end of the 
Prioritization   first open enrollment period, CMS had a stable website that functioned well at high capacity, 
is a CMS term for  but planned components had yet to be completed.  CMS needed to make significant upgrades 
determining the most 
to the account creation, application, and plan selection subcomponents, and to complete the 
critical elements of a 
financial management system that would track effectuated (paid) enrollments and manage 
project to be completed 
given the available  payments to issuers for Federal financial assistance and premium stabilization.  The day after 
time and budget.  first open enrollment closed, April 1, 2014, CMS began substantive planning for the second 
open enrollment period of HealthCare.gov to occur 7 months later on November 15, 2014.  
Federal Marketplace managers from OIS, CCIIO, OC, and the Office of the [CMS] Administrator 
met in a 3-day session for what they termed ruthless prioritization to consider which 
elements were most important to include for the second open enrollment period and what 
funds were available.  CMS leadership requested that managers provide a list of technological 
needs, then technical and business management staff evaluated the personnel, time, and 
other resources needed to complete each task.  According to officials, the group debated and 
then cut about half of the items requested. 
50%  These cuts included key elements of the Federal Marketplace system, including completion of 
the automated financial management sytem and tools for existing plan holders to reenroll.  
Estimated Cut   CMS initially planned for the automated financial management system to be completed for 
from list of additional  the first open enrollment period, then delayed the project to focus resources on items 
functionality for second  considered essential for the websites 2013 launch.  In the interim, CMS established a manual 
open enrollment in 
process to track effectuated enrollments and manage payments to issuers, which an OIG audit 
April 2014. 
later found did not have effective internal controls in place to detect possible payment 
errors.102  In addition to cutting automated financial management, CMS cut components of 
the automated reenrollment functionality that would have redetermined the eligibility of 
existing enrollees who did not make an active plan selection on or before December 15, 2014.  
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           46  Turnaround & Second Open Enrollment | April 2014March 2015 
CMS had not yet developed this component because the first open enrollment had only new 
customers and did not require reenrollment.  CMS also scoped out planned website tools 
intended to help consumers make health plan purchasing decisions. 
This process for strategic and organized prioritization marked a significant improvement over 
the reprioritization meetings that occurred prior to the launch.  The 2013 prelaunch 
reprioritization was performed late (August 2013, less than 2 months prior to launch), 
resulting in rushed and poorly informed decisions, and did not cut enough functionality to 
alter the negative outcome.  For 2014, CMS officials and contractors allowed for more time, 
fully engaged officials from across the organization, and conducted a close examination of 
resource costs and program implications. 
Pens Down is the  Informed in part by problems in 2013, CMS set an aggressive schedule to make technical 
term CMS used for  improvements in every area.  Also in April 2014, CMS leaders went over the schedule with 
signifying the final date 
Accenture, QSSI, and other key contractors to ensure adequate time for testing before second 
for technical staff to 
open enrollment.  Recalling the 2013 problems with policy changes requiring changes to the 
make system coding 
software late in the build, CMS established a more formal process for change management, 
changes. 
the task of completing and communicating changes in policy, business requirements, and 
technical specifications.  Part of this process was to bring problems to managers more quickly 
and to include more detail in technical reports and presentations.  A CMS technical official 
indicated that during this time period, CMS and Accenture were still correcting coding 
problems that were not fully resolved during the recovery period after first open enrollment.  
CMS also set a firm pens down date of October 7, 2014, for system coding, meaning no 
additional system changes were to be made other than to address problems found during 
testing.  Another barrier to testing in 2013 was coordinating testing environments to provide 
enough computing and storage space to run tests.  To improve this for the second open 
enrollment, CMS increased its infrastructure available for testing, and staff at the XOC worked 
in conjunction with QSSI, the systems integrator, to schedule use of testing environments 
among the CMS staff and contractors working on various pieces of the build. 
CMS also refined processes for the XOC, which was now able to shift its focus from fixing 
problems to training and establishing longer-term processes.  As a CMS official at the XOC 
stated, We tried to step away from heroics.  CMS managers conducted readiness reviews 
that identified weaknesses in the XOCs ability to operate during open enrollment.  Two key 
changes resulted, both regarding leadership:  CMS designated the role of the Pit Boss, a 
contracted employee who manages incidents and ensures that system changes are 
coordinated across teams; and instituted a Floor General position, a CMS employee who 
reports status and problems to a CMS Executive On Call and coordinates the XOC workflow. 
Other changes to management of XOC operations included hiring a permanent team of  
site reliability engineers to provide technical guidance focused on maintaining service; 
additional training, such as simulation drills to existing staff; and enhanced monitoring tools 
for real-time tracking of functions, such as the application process.  They also coordinated 
closely with other Federal agencies, such as SSA and IRS, regarding system downtimes and 
technical changes in interrelated systems.  CMS established a new Open Enrollment 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           47  Turnaround & Second Open Enrollment | April 2014March 2015 
Coordination Center to coordinate the nontechnological aspects of open enrollment, such as 
consumer and issuer experiences, and relay information from this vantage point to the XOC to 
improve operations (see Figure 4).  Overall, these enhancements strengthened CMSs ability 
to respond to operational issues with the website. 
Figure 4:  Description of Open Enrollment Coordination Center. 
TYING IT ALL TOGETHEROECC 
 
In September 2014, CMS created the Open Enrollment Coordination Center (OECC), an 
  entity that serves as the business version of the XOC and operates out of the policy 
office, CCIIO.  The OECC placed the communication and coordination of all project 
  components into a single office with a small group of staff.  The key tasks of the OECC are 
to monitor and coordinate resolution of all open enrollment issues by translating 
information to and from the policy, operations, communications, and technical teams.  
 
The creation of the OECC allowed the XOC to focus only on technical systems.  The OECC 
works closely with the call centers that receive information from consumers and issuers 
 
about problems, then feeds the information back to the systems operators at the XOC.  
The OECC keeps an open bridge telephone line with the XOC and determines how 
 
consumers and issuers might be affected by technical problems. 
Se cond open enrollment was complicated by the need for reenrollment of existing plan holders, 
requiring new website functions and communication to consumers.  As previously stated, CMS 
had to implement new technology for the second open enrollment that enabled existing plan 
holders to reenroll, either in their prior plan or a new plan by December 15, 2014.103  Existing 
plan holders could access the site to compare and consider selecting a different plan for their 
second year, but CMS officials wanted an automated reenrollment process that could also 
redetermine whether the plan holder was eligible to receive Federal financial assistance.  This 
would be convenient for consumers who did not want to reapply to stay enrolled in their 
current, or equivalent, plan and continue to receive financial assistance.  Auto-reenrollment 
would also reduce the number of users on the HealthCare.gov website, therefore reducing 
  capacity needs, which had been such a problem during the first open enrollment period. 
Year two was   Reenrollment and redetermination of existing plan holders raised other complexities beyond 
arguably tougher than 
the technical development; existing plan holders would require different information from 
year one because of the 
CMS than did new consumers.  Updated regulations required the Federal Marketplace to send 
complexity of adding 
notices to all individuals who received financial assistance and describe the annual 
reenrollment. CMS staff 
redetermination process for financial assistance.104  During summer 2014, CMS provided 
enrollees with the benefits of each option, (i.e., automatically reenrolling or reviewing plan 
options) by letter and email.  This information explained that if a consumers income or 
household size had not changed, then they could choose to do nothing and HealthCare.gov 
would automatically renew their coverage.  (Changes in income or household size could result 
in new monthly premium rates or changes to eligibility for financial assistance.)  Several media 
reports criticized CMS for encouraging consumers to do nothing.  Eventually, CMS leadership 
came to believe that consumers would be best-served by logging back into HealthCare.gov to 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           48  Turnaround & Second Open Enrollment | April 2014March 2015 
decide whether to change plans.  In October 2014, CMS revised its message on the website 
and in other materials to emphasize that consumers would benefit from updating their 
personal information on the website and reconsidering health plan options.105   
Timeframes were tight for final website improvements, but CMS execution was much improved 
in comparison to 2013.  In addition to addressing the new challenge of auto-reenrollment, 
CMS made other changes, including improving the waiting room system during heavy-use 
periods.  CMS also implemented changes to accommodate two States (Nevada and Oregon) 
that began using the Federal Marketplace for certain functions and one State (Idaho) that left 
the Federal Marketplace to build a State marketplace.106  (Thirty-seven States used the Federal 
Marketplace during the second open enrollment.107)  Other aspects of the HealthCare.gov 
operations were still in flux weeks before the second open enrollment period, and contractors 
conducted some system testing behind the original schedule.  For example, testing was 
delayed for the new function to notify issuers whether their existing plan holders would be 
automatically reenrolled in their previous health plan or chose a different plan through 
HealthCare.gov; this testing was delayed from August to October 2014, just a month before 
the second open enrollment period.  To handle the additional tasks, CMS augmented its staff 
in the final weeks before second open enrollment, including adding staff to the call center and 
redirecting some technical staff from other divisions, as in 2013. 
CMS management of the project leading up to the second open enrollment period again stood 
in contrast to the 2013 launch.  Project documentation indicated that in 2013, CMS and 
contractors were frantic to establish basic website functionality.  As a result, they pushed 
forward faulty and untested functionality and hoped to fix it after the launch.  Project 
documentation indicated that in 2014, CMS maintained a more disciplined project schedule, 
meeting deadlines with a goal to implement only technology that had what project 
documentation referred to as perfect execution.  When this standard could not be met in 
time, CMS identified problems more quickly to allow time to employ contingency plans.  For 
example, the new account creation and identity verification system, SLS, was deemed unready 
in late summer 2014, so CMS and contractors re-engineered a portion of the existing system, 
EIDM, to serve as a dedicated account creation and identity verification system called 
  Insurance Marketplace Authentication System (IMAS). 
App 2.0 went   One of the causes of problems with the EIDM in 2013 was that it was designed to provide 
into production 30 days  identity management services to multiple CMS programs rather than exclusively for 
early so we could see it in 
HealthCare.gov.  CMS technical staff and contractors re-engineered IMAS specifically for 
the wild (during special 
HealthCare.gov so that it allowed consumers to create an account, but avoided the EIDM 
enrollment).  It was very 
entry bottleneck.  IMAS was in testing by September and complete by October 2014.  CMS 
successful. CMS staff 
continued to develop and test the SLS, launching it in February 2015 and completing the 
transition in March 2015.  CMS also deferred a new tool called Plan Compare 2.0, which was 
designed to provide a more comprehensive comparison of health plans and premiums than 
the temporary shopping tool established after the launch.  CMS suspended work on Plan 
Compare 2.0 in summer 2014 to focus on what they perceived to be more critical:  the new 
streamlined App 2.0 consumer application.  CMS conducted a soft launch of App 2.0 to 
special enrollment period consumers in September 2014, then used App 2.0 for all new 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           49  Turnaround & Second Open Enrollment | April 2014March 2015 
consumers during second open enrollment with high performance results, according to CMS 
documentation.  As an illustration of improved planning and organization, CMS made the 
decision to defer a portion of Plan Compare 2.0 fully 5 months before second open enrollment 
began as compared to a decision regarding the similar anonymous shopper tool that CMS 
  deferred 11 days prior to first open enrollment. 
Year one we  CMS focused increasingly on consumer outreach and assistance, including improvements to 
focused on fixing the 
the navigation and content of HealthCare.gov.  Leading up to and during the second open 
technology, for year 
enrollment period, CMS placed greater focus on expanding outreach to eligible consumers 
two we focused on the 
and to assisting consumers with enrollment.  There is less happening now on the policy side, 
consumer.  
so our focus can be on consumers, explained a CCIIO official.  The direction for these efforts 
CMS marketing staff 
was led in part by feedback CMS received from navigators and call center representatives, and 
from information gleaned by HHS and CMS officials from stakeholders such as issuers, 
community organizations, and the public.  CMS OC also conducted market research to identify 
barriers to enrollment. 
Strategies differed for existing HealthCare.gov plan holders and potential new consumers.  
CMS sent existing plan holders letters, email, texts, and telephone calls to encourage 
re-review of plan options, and remind them of the auto-reenrollment process to take place on 
December 16, 2015 and other key dates.  CMS also reached out directly to consumers who 
started applications on the website but left them incomplete, encouraging them to return to 
the website to purchase plans.  To encourage new consumers, CMS conducted public 
enrollment events, many featuring the HHS Secretary and other officials, purchased 
advertising, and invested further in local navigators.  In addition to information about the 
Federal Marketplace and HealthCare.gov, these efforts sought to address needs in consumer 
health literacy, such as instruction in health plan and coverage terminology.  In interviews, 
CMS senior leadership indicated this was in part to increase enrollment in the marketplaces 
but also to improve overall public knowledge of health care and insurance, regardless of 
where it is purchased.  As a senior CMS official observed, We could be a resource for 
300 million people, not just 10 million. 
To improve the usefulness of the HealthCare.gov 
website, CMS sought to sharpen its visual  Now that operations are stable, 
appearance, navigation, and content.  For 
that frees us up to do more. 
example, CMS improved navigation tools to 
 
CMS consumer staff
engage consumers to access additional 
information without leaving the home page.  
CMS also replaced question-based categories (Am I enrolled?) with simpler topic-based 
labels to reduce confusion.  The HealthCare.gov home page also clarified what actions 
consumers could take immediately and on key future dates, and updated these instructions 
(e.g., from Get Ready steps prior to open enrollment to Act Now steps during open 
enrollment).  CMS sought to improve website navigation in particular for complicated 
households, such as blended families covered in part by employer insurance or government 
programs, and added substantially to HealthCare.gov website content, including more 
thorough explanations of coverage, improved tools to compare plans, and consumer case 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           50  Turnaround & Second Open Enrollment | April 2014March 2015 
examples.  CMS also launched the SHOP website, which enables small businesses and their 
employees to browse plan offerings and enroll in coverage online, to apply for Small Business 
Health Care Tax Credits, and to obtain personalized assistance.108   
HealthCare.gov homepage encouraging consumers to get ready for the second open 
enrollment period, October 3, 2014. 
 
CMS also sought to address prior difficulties that many consumers had regarding premium tax 
credits.  CMS regional office staff reported increasing communication in 2014 to make 
consumers aware of the importance of making updates to their income estimates.  Early 
external reports indicated that the income estimates of half of consumers were too low during 
the first year of enrollment and these consumers consequently owed income taxes for 2014 
following the first year of reconciling estimated to actual income.109  In interviews, a sample of 
navigators indicated that they were not equipped to field questions regarding the tax 
implications of enrollment decisions, and that CMS could have provided more information and 
guidance about tax-related issues.110,111  Also, according to the navigators, CMS did not 
provide consumers with adequate guidance about the need for consumers to update their 
income information.   
Second open enrollment operations of HealthCare.gov ran smoothly, with high use and no CMS 
system outages.  The second open enrollment period began November 15, 2014, and CMS 
documentation indicated the technical aspects of the website and supporting systems 
performed well, with no system outages and few consumer reports of problems applying or 
selecting plans.  On the first day of second open enrollment, the website had approximately 
34,000 accounts created, approximately 60,000 applications submitted, and over 650,000 
unique visitors.112  HealthCare.gov response times were also quicker with a 3.21 second 
median time for logging in during the second open enrollment compared to 18.46 seconds 
during the first open enrollment. 113  The highest number of users on a single day to 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           51  Turnaround & Second Open Enrollment | April 2014March 2015 
HealthCare.gov during second open enrollment was 1.86 million,114 with capacity to support a 
high of 250,000 concurrent users without an outage.  See Figure 5 for a comparison of 
selected statistics for HealthCare.gov between first and second open enrollment. 
Figure 5:  Comparison of selected HealthCare.gov statistics. 
 
CMS highlighted what it considered key successes across the first two open enrollments.  A 
June 2015 analysis showed that approximately 9.9 million consumers had signed up and paid 
the premium for a health plan on HealthCare.gov or a State marketplace during second open 
enrollment, surpassing the Departments revised projections of 9 million enrollees but falling 
short of the Congressional Budget Offices projection of 13 million enrollees.115  During that 
same period, CMS reported that increased competition in the Federal Marketplace provided 
consumers, on average, with 25 percent more health plans to choose from while minimizing 
premium increases to 2 percent for the benchmark plan in each State used to calculate 
premium tax credits.116 
Changes made by CMS during preparation for second open enrollment also focused 
on longer-term improvements to the Federal Marketplace and throughout CMS 
CMS further formalized HealthCare.gov project leadership.  Improved technical execution was 
due in part to changes made by CMS to Federal Marketplace management.  CMS continued its 
contract with QSSI as technical systems integrator in September 2014, and the CMS 
Administrator sustained daily, hands-on leadership through mid-2014.  In June 2014, the 
agency hired a Principal Deputy Administrator who had served as a key contracted manager 
during the website recovery period and was knowledgeable about the project and its 
responsible staff.117  The responsibilities of the position extended beyond HealthCare.gov, but 
interviews with the then-new Principal Deputy Administrator indicated a focus on 
HealthCare.gov through the second open enrollment period.  In August 2014, CMS hired a 
formal Marketplace CEO, selecting the former director of a State marketplace.118  These new 
officials, in conjunction with the CMS Administrator, coordinated Federal Marketplace tasks 
across CMS divisions responsible for various aspects of CMSs functions, such as OIS for 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           52  Turnaround & Second Open Enrollment | April 2014March 2015 
information technology, OC for communications, and OAGM for contracting.  In interviews, a 
senior CMS official described this as a re-boot of its prior enterprise (organization-wide) 
management strategy that relies on sharing services across the organization and its programs.  
The CMS COO reflected that the prior enterprise strategy itself was not faulty, given the need 
for specialization across a large, complex organization, but in this case, the unclear leadership 
and communication across divisions had diminished its effectiveness, made worse by the lack 
of integration among multiple contracted entities.  Other CMS officials and staff stated similar 
views, contending that the success of the enterprise structure is dependent on leaders 
managing projects globally across functions so that they can assess overall progress and 
identify gaps. 
CMS renewed its focus on contract management, particularly emphasizing the agency and 
contractor relationship.  HHS and CMS changed core contracting policies that had 
compounded problems with contract management to work more closely with contractors and 
better ensure project tracking and performance.  To improve its management of contracts, 
HHS instituted in April 2014 new acquisition planning guidance and is amending the HHS 
Acquisition Regulation, the Departments rules for conducting acquisitions, with the goal of 
improving efficiency and effectiveness of various phases of the acquisition process.119  CMS 
officials reported several changes to its contract procurement and oversight strategy, 
including better defining individual roles and responsibilities and transitioning to a program 
management structure for managing all IT-related investments, including the FFM.  This 
structure requires program managers to maintain responsibility for the overall success and 
management of the IT systems that support the program.120  According to CMS, the goal of 
the policy is to develop a program management culture that ensures everyone involved is 
working to meet the needs of the project and organization.  CMS had already addressed 
another key problem in contract management:  adding to the scope of the contract 
(unauthorized commitments); prior to the first open enrollment period in April 2013, the 
Director of OAGM issued a guidance memorandum in response to CMS staff having modified 
the scope of contractor work without authority to do so.  
Part of CMSs problem in managing 
contracts before the launch may have  We had to change the mindset 
been a lack of understanding by CMS 
without changing good governance . . . 
staff regarding the agency-contractor 
[to emphasize that you need] good 
relationship.  According to interviews 
and correspondence, some CMS staff  relationships with contractors. 
were reluctant to work too closely with  CMS official 
contractors for fear of violating 
Government contracting rules.  For 
example, CMS staff working on the website sometimes requested that contractors not 
participate in meetings or receive information.  These actions resulted in a sense that CMS 
and contractors were not a team working toward the same goals and hampered the agency 
and contractor relationships that would promote communication and progress.  As one CMS 
official noted, We had to change the mindset without changing good governance . . . [to 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           53  Turnaround & Second Open Enrollment | April 2014March 2015 
emphasize that you need] good relationships with contractors.  To achieve this, CMS 
leadership more clearly instructed staff who were coordinating technological work and 
supervising contracts about rules and provisions related to contracted work, and also 
instructed contractors to work more collaboratively with each other and the CMS divisions.  
For an agency in which much of the work is performed through contractors, this represented 
more of a cultural shift than a change in policy. 
Communication improved among CMS divisions as the agency focused on merging policy with 
operations, and encouraging identification of problems and sharing bad news.  CMS 
management focused on better blending the policy and technical components of CMS with a 
greater sense of what CMS leaders called the physics of operations or operational 
awareness.  Policymakers acquired a better understanding of the effort required to 
effectuate policy decisions, both in terms of time and resources, so that those considerations 
could better inform decisionmaking.  According to interviews with long-time staff members, 
CMS has always had a bifurcation between 
operations and policy, but during our report 
You cant over communicate.  
period, leadership appeared to be changing 
Over communication is never a 
this.  One long-time staff member observed:  
[CMS] leadership now is more focused on how  problem.  HHS official 
to think of the end-to-end process. 
This close interaction between CMS officials and staff also required a willingness to solicit and 
accept bad news, such as negative assessments of progress and performance.  CMS officials 
recognized that they needed to actively look for problems, and CMS employed a policy of 
encouraging staff and contractors to do so.  CMS staff shifted from following known processes 
to continually assessing outcomes and progress.  For example, CMS identified a number of 
problems in the lead-up to the second open enrollment through deep dives, assigning staff 
to scrutinize the performance of a specific area or function (such as eligibility) and bring to 
CMS leadership their assessments of weaknesses.  This approach provided a way for staff to 
move bad news to leadership and provided a more formal record of problems to better 
ensure resolution.  Seeking bad news and changing course as needed takes, according to one 
CMS senior leader, a conscious effort with buy-in and follow-through from all levels. 
Another change in approach was to conduct financial budgeting for the Federal Marketplace 
project as a single process rather than manage separate budgets across the various CMS 
divisions and functions.  Leading up to the launch of HealthCare.gov, CMS had separate 
budgets for the policy and technological work in the two key divisions of CCIIO and OIS.  This 
led to confusion over which division covered which costs and responsibilities and resulted in 
inefficiencies.  In preparation for the second open enrollment period, the Federal Marketplace 
budget was combined into a single process, meaning that both IT and non-IT costs and 
benefits would be more clearly assessed together, and each division better understood the 
activity of the other and potential tradeoffs for decisions and additional expenditures.  As a 
CMS financial officer noted, If the call centers needed more money, we could push back on 
some IT activities. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           54  Turnaround & Second Open Enrollment | April 2014March 2015 
CMS management focused on realistic alignment of project goals and resources, and 
straightforward measurement of outcomes to avoid artificial progress.  Two key aspects of 
CMSs operational strategy during this period were to more carefully align project needs with 
organizational resources and to monitor use of resources and progress more closely.  CMS 
officials spoke in interviews of avoiding the prior problems of CMS staff and contractors 
working at cross-purposes and on tasks that did not clearly promote core objectives.  They 
observed that the prior approach created an environment of artificial progress that created 
unwarranted optimism and masked problems.  [We are] outcome driven . . . [meaning] no 
  hiding, noted a CMS official. 
If we have a   QSSI, the systems integrator, played a large role in increasing rigor in aligning project goals 
challenge, we talk about  and resources.  QSSI continually assessed project progress and weak points and connected 
it daily.  If we have a 
with staff to resolve discrepancies or potential breakdowns.  As a CMS official reflected, 
really big challenge, we 
[QSSI] lifted up the specifics to flag problems and bring them to leadership, preventing silos 
talk about it twice a day. 
and poor communication.  People dont always want to take problems to [leadership], so the 
CMS official 
systems integrator did so.  CMS officials credited QSSI with easing the process of executive 
decisionmaking when decisions required the input of CMS leadership.  QSSI handled much of 
the executive reporting function 
previously held by CMS division leaders, 
People dont always want to take 
which saved time for those working on 
problems to [leadership], so the 
various pieces of the website and 
program, and also provided an easier and  systems integrator did so. 
more objective method for bringing forth  CMS official 
problems.  
CMS senior leadership was actively involved in daily project work for HealthCare.gov, easing 
aspects of project management but likely not sustainable over time.  Another noteworthy 
difference in project management during this period was the degree to which senior CMS 
executives handled day-to-day operations for the Federal Marketplace.  The CMS C Suite of 
Administrator, Principal Deputy Administrator, COO, CIO, Chief of Staff, and other leaders 
routinely attended Federal Marketplace meetings.  Senior leaders alternated as the Executive 
On Call, serving 24-hour shifts to make decisions more rapidly, regardless of the time of day.  
As a result, even specific problems reached the top; for example, an issue that concerned a 
single health plan in one State was raised to the Administrator and resolved within hours.  For 
the most part, CMS staff touted the senior-level involvement as positive and welcome, 
enabling quicker decisionmaking and greater unity across divisions.  CMS senior leadership 
and staff also noted that this involvement 
likely raised the sense of urgency to 
If people know that problems will 
collaborate and to complete tasks timely 
be elevated to the top levels quickly, 
and well.  As one CMS official noted, If 
people know that problems will be  the incentive to reach consensus and 
elevated to the top levels quickly, the 
move forward is very high. 
incentive to reach consensus and move 
 
CMS official
forward is very high.  
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           55  Turnaround & Second Open Enrollment | April 2014March 2015 
CMS officials reflected that there were also drawbacks to such concentrated involvement by 
senior CMS leaders and that it was likely not sustainable indefinitely.  A few officials and staff 
expressed concern that CMS senior leadership had overcorrected their prior lack of 
involvement, and that continued heavy involvement could lead to a narrow focus on daily 
chores and managing to a punch list at the expense of broader organizational needs such as 
strategy and goal-setting.  As one official stated, We got a more granular operational 
awareness, but sometimes at the expense of a 
broader view.  CMS officials noted an awareness of 
this drawback and indicated they would reassess the  We got a more granular 
level of senior leadership involvement as the Federal  operational awareness, but 
Marketplace system matured.  As one CMS official 
sometimes at the expense of 
reflected, When you have the problems we had, 
a broader view. 
you are going to see more time and depth of senior 
 
leadership involvement.  As we move to a more  CMS official
mature program, we will see less of that.   
CMS expects its restructuring to improve operations across the organization.  CMS changed its 
organizational structure in February 2015 to improve governance (oversight of processes) and 
make more efficient use of resources.  These changes included segregating IT operations and 
governance to preserve impartiality in making IT decisions, promoting shared use of services 
such as IT and contracting, and elevating the usefulness of its data analytics functions.  To 
preserve impartiality in making IT decisions and governing those decisions, CMS divided two 
functions from OIS into separate groups:  IT operations (e.g., networking and hardware) and 
IT governance (e.g., software architecture and security).  The new Office of Technology 
Solutions is responsible for IT operations while the broader Office of Enterprise Information is 
responsible for IT governance, under the leadership of the CMS CIO.  Also, CMS established an 
information security team under the CIO that monitors and tracks corrective plans for security 
  vulnerabilities and ensures the plans are completed.121,122 
Marketplace has   To promote the enterprise structure and sharing use of services such as IT and contracting, 
quickly become an 
CMS leadership formed a Strategic Planning and Management Council comprised of five 
incubator wherein we lead 
workgroups:  appeals, eligibility and enrollment, plan oversight, security and privacy, and 
change to other areas of  
workforce planning.  The workgroups include leaders from each of CMSs programs with the 
the organization.   
CMS official  goal of mapping out operational similarities and developing opportunities to share resources.  
For example, several CMS programs, including the Federal Marketplace, provide support for 
beneficiaries, providers, and suppliers to appeal coverage and payment decisions.  The 
appeals workgroup outlined similar appeals processes across programs and combined 
operations where appropriate.  Finally, CMS placed its new Chief Data Officer and data 
analytics and research group within the Office of the Administrator to further integrate the 
use of data into CMS management and decisionmaking.123 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           56  Turnaround & Second Open Enrollment | April 2014March 2015 
Challenges remain for CMS in operating HealthCare.gov, with public scrutiny high 
and the website and other automated functions not yet complete 
Public scrutiny of HealthCare.gov is still high, and periodic problems continue to raise concern 
from stakeholders.  Public attention to the performance of HealthCare.gov diminished even 
before the start of the second open enrollment period.  The website was perceived as working 
well, but criticism of the launch and the cost of recovery remains in the public dialogue.  Some 
CMS officials reflected this was likely due in part to the continuing political contention over 
the ACA and marketplaces.  An April 2015 Kaiser Family Foundation poll found that the 
publics opinion of the ACA remains divided, but the overall view turned favorable for the first 
time since November 2012 with 43 percent 
reporting a favorable view of the law and 
42 percent an unfavorable view. 124  The  We are in a fishbowl.  In the 
margin between positive and negative views 
private sector, no one knows 
remains slight; January 2016 results from the 
your mistakes, but everything we 
same survey indicated that overall public 
opinion turned negative again, with 44 percent  do is visible.  CMS official 
reporting an unfavorable view and 41 percent 
a favorable view.125  
Much of the public dialogue has surrounded the extent to which Government officials and 
contractors should be held accountable for mistakes leading up to and at the launch.  There 
were calls from Congress and the media for Government officials to be fired over the website 
failures.  Most notably, CGI Federal lost its role as the primary contractor for the FFM.126  
Additionally, some HHS and CMS officials and staff did resign or retire following the launch, 
and others were reassigned to different positions or their responsibilities were revised.  CMS 
staff reported that these changes were due to a range of factors, including differences in 
approach regarding the projects direction and management, concerns about poor 
performance, exhaustion following the intense work leading up to and following the launch, 
and changes that occur in the normal course of business during a reorganization.  CMS 
officials indicated in interviews that the fact that more CMS staff involved in the launch did 
not leave their positions immediately was due in part to CMSs need to implement post-
launch corrections and retain already low staffing levels. 
CMS continued through 2014 to face issues related to HealthCare.gov, which led to 
substantial media interest and congressional inquiry.  Most were resolved, although they likely 
led to some continued public concern about HealthCare.gov operations.  For example, in 
mid-2014, a hacker breached a HealthCare.gov test server, causing CMS and observers to 
question security, although there was no known compromise of private information.127  In 
another example, media discovered that CMS allowed third-party content providers, hired by 
CMS to monitor consumer use of HealthCare.gov, to share personal information of 
HealthCare.gov users with other entities.128  Although CMS contended that this was a fairly 
common practice for public websites, the agency curtailed sharing of information in response 
to the concern.  Errors also continued to occur in providing enrollment and other data to 
stakeholders.  The most publicized of these errors was an overstatement by the CMS 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           57  Turnaround & Second Open Enrollment | April 2014March 2015 
Administrator in testimony to Congress in September 2014 of the number of individuals 
enrolled in health plans, an error that CMS later attributed to a staff error in interpreting the 
data.  And in early 2015, the FFM sent incorrect tax forms, later corrected, to thousands of 
consumers who received tax credits, prompting the IRS to provide more time for consumers 
to file their taxes.129  These errors resulted in questions about the accuracy of HealthCare.gov 
enrollment figures overall and more stakeholder inquiry.130 
CMS has not fully implemented the Federal Marketplace automated financial management 
functions.  The most significant technical challenge facing CMS is completing implementation 
of the automated financial management system.  CMS planned to complete the automated 
system prior to the first open enrollment period, then delayed the systems projected 
completion date several times to prioritize other aspects of the Federal Marketplace project.  
OIG audits found that with the manual system CMS used in the interim, CMS could not 
confirm the accuracy of payments at the individual, policy-based level131 and could not ensure 
that payments are made only for enrollees who paid their premiums.132  
CMS has continued work on the automated system, now called Enrollment and Payment 
System (EPS).  When fully implemented, EPS will automate all financial functions of the 
Federal Marketplace, including tracking effectuated (paid) enrollments, managing payments 
to issuers for financial assistance and premium stabilization, and managing user fees.  In 
January 2016, CMS transitioned most issuers to the portion of EPS that calculates payment 
amounts and enrollment numbers, replacing the manual calculation method with a more 
precise, policy-based method.133  CMS continues to add issuers to the automated system as 
they meet the agencys criteria for readiness to transition.  The agency plans to complete the 
remaining EPS functions within 2016, hoping to make financial management of the Federal 
Marketplace more efficient and lower cost, and to improve the accuracy of payments and 
data.  Even with the full automation, CMS staff acknowledged in interviews that there will 
always be some need for data reconciliation between Federal Marketplace and issuer data to 
ensure that issuers have accurate consumer information.  CMS reported that it plans to 
continue conducting internal validation checks to ensure issuers submit accurate information. 
In addition to completing the website build, CMS must continue to address technical and 
operational challenges.  CMS has continued to correct technical problems with the website, 
some dating back to the original 2013 framework.  As with earlier problems, changes to the 
system still require development of business requirements, technical development, website 
performance and security testing, and reconfiguration of monitoring and operations, such as 
website capacity.  In interviews, CMS officials indicated they perceived the third cycle of open 
enrollment as the first full enrollment period because this is the first open enrollment 
period for which they will have 
data to predict consumer use of 
The complexity of the systems is 
the website for both first-time 
surprising. . . .  You could have a Ph.D. in 
enrollees and re-enrollees.  These 
estimates of website user behavior  [Federal] Marketplace data file transfer. 
should allow for more precise 
CMS technology official 
measurement of needs for 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           58  Turnaround & Second Open Enrollment | April 2014March 2015 
website capacity and consumer support.  There are other, ongoing challenges, such as 
improving the transfer of account information to issuers and consumers.  As one CMS 
technology official who was new to the project noted, The complexity of the systems is 
surprising. . . .  You could have a Ph.D. in [Federal] Marketplace data file transfer.  Depending 
upon the data issue, this task is shared by CMS policy and technical staff, with CCIIO focused 
on conducting outreach and casework with issuers and consumers, and the Office of 
Technology Solutions focused on data transmittal.  CMS reported that it also hopes to 
continue increasing enrollment among eligible consumers who have not purchased health 
plans.  Prior to the third open enrollment period, an estimated one-third of eligible consumers 
had enrolled in health plans through the Federal or State marketplaces.134 
CMS must also improve the accuracy of critical Federal Marketplace functions such as 
determining who is eligible and amounts paid to issuers.  Previous OIG work based on data 
from the first open enrollment period concluded that CMS should strengthen its internal 
controls for determining eligibility for enrollment and Federal financial assistance, and for 
resolving inconsistencies in enrollment information submitted by applicants.135  In response, 
CMS reported that it works continuously to ensure that the Federal Marketplace accurately 
determines eligibility and resolves inconsistencies, including making regular updates to the 
system to resolve issues.  As an example, CMS created a new pop-up message in the 
HealthCare.gov application to encourage consumers to enter a Social Security number on the 
application, which CMS believes should decrease the number of data-matching issues from 
that of the first open enrollment. 
Open enrollment for the third coverage year of the Federal Marketplace was originally to 
begin on October 1, 2015 (see Figure 6 for open enrollment dates);136 CMS moved the start 
date by 1 month to November 1, 2015, in order to complete the build of additional 
functionality and for new issuers to submit and refine plan data.  CMS also extended the end 
date of open enrollment to January 31, 2016, to provide 3 months, the same duration as the 
second open enrollment period.137  CMS plans to continue open enrollment for annual 
3-month periods, unfolding a multi-year IT approach to continue improvement, such as 
enhancing plan selection tools for consumers. 
Figure 6:  Periods of firstthird open enrollment for the Federal Marketplace, 20132016. 
 
 
 
 
 
Source:  45 CFR  155.410. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           59  Turnaround & Second Open Enrollment | April 2014March 2015 
KEY FACTORS CONTRIBUTING TO RECOVERY 
  RUTHLESS PRIORITIZATION 
Because the timeframe and resources available to prepare for the second open enrollment period were fixed, 
CMS focused on reducing scope to meet deadlines.  The day after first open enrollment closed, CMS 
leadership met to employ ruthless prioritization of tasks to focus on the most urgent needs and 
functionality.  These decisions and resulting changes were then locked down and measured for progress and 
results.  CMS was not able to deliver some functionality as planned, with full automation of the financial 
management system still in development at the end of second open enrollment.  Ruthless prioritization 
served, though, to align goals with the resources available, guide daily work and accountability, and temper 
unrealistic expectations about results. 
  QUALITY OVER ON-TIME DELIVERY 
CMS adopted a project management approach of going live with website functionality only when it could 
ensure what one CMS official called perfect execution.  This was in contrast to the launch of 
HealthCare.gov, wherein CMS delivered what it knew was faulty functionality, planning to improve the 
website later.  In the case of the new HealthCare.gov consumer application, App 2.0, delivery was tested 
through a soft launch prior to open enrollment.  As with the prioritization process, this approach meant 
that CMS did not always deliver according to schedule.  For example, CMS did not launch its new account 
creation system as planned when problems arose.  CMS leaders and contractors said in interviews that this 
policy of requiring optimal functioning before delivery led to improved practices overall, such as targeting 
earlier deadlines for delivery and imposing stricter testing standards. 
  SIMPLIFYING PROCESSES 
Large organizations are vulnerable to creating unneeded organizational structures that can cause staff to lose 
sight of project goals.  During the 2013 launch of HealthCare.gov, CMS divisions, particularly policy and 
technical teams that were responsible for various pieces of the project, operated separately and did not 
communicate well with other teams.  This led to delays and lack of accountability.  CMS simplified both 
technical aspects of the build and the organizational structure of the agency itself by closely monitoring 
progress and results with daily reports and close communication with contractors.  This made the work more 
transparent and aided in prioritizing goals, reducing the common problem of artificial progress that large, 
complex projects havemany parties completing tasks, but not moving forward toward the project goals.  
Reduced complexity in tasks and organizational structure made it easier for CMS to identify those responsible 
for carrying out tasks and to track progress toward goals. 
  CONTINUOUS LEARNING 
A culture of continuous learning encourages open communication and active monitoring of performance and 
progress, allowing for a change in course as the facts dictate.  In preparation for the second open enrollment 
period, much about the HealthCare.gov project was still unfolding.  For example, CMS did not know how 
much website capacity consumers would require, and it was still developing and testing new and improved 
functionality in the final weeks before open enrollment.  Given that the design and proportion of the project 
was evolving, it was critical to CMSs success that the organization continuously learn as the project 
progressed.  As the HealthCare.gov project matured, CMSs knowledge and experience became more 
concrete and its planning more effective, but the project continued to require adaptation. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           60  CALL FOR CONTINUED PROGRESS
 
We need a sense of urgency without  
crisis.  It is a marathon, not a sprint. 
CMS official 
 
 
Does real change  CMS continues to face challenges in implementing the Federal Marketplace, and in improving 
require a crisis?  
operations and services provided through HealthCare.gov.  As of February 1, 2016, CMS 
We asked this of CMS 
reported that over 9.6 million consumers had selected a health insurance plan through the 
officials, and the general 
Federal Marketplace or had their coverage automatically renewed.138  As CMS moves forward, 
consensus was that 
change does not require a  challenges include improving the website and systems as planned, such as completing the 
sentinel event, but that  automated financial management system and improving consumer tools to select plans.  CMS 
the website breakdown  must also continue to address areas OIG has identified in past reports as problematic or 
expedited organizational 
needing improvement, including contract oversight, the accuracy of payments and eligibility 
changes already 
determinations, and information security controls.   
underway.  Still, some 
thought the high visibility  CMSs experience with HealthCare.gov provides lessons for HHS and other organizations in 
of the breakdown was 
navigating program implementation and change.  These lessons comprise core management 
critical to prompting 
principles that, had they been applied earlier, could have avoided problems in execution.  
change, such as the CMS 
CMSs use of these principles following the breakdown enabled the organization to recover 
senior official who stated, 
Sometimes an  the website and improve management and culture.   
organization has to get a 
wake-up call.  Given CMSs large organization and complex mission, prior management problems could 
resurface and new problems could emerge.  CMS placed intense organizational focus on the 
Federal Marketplace during the recovery of the website.  This level of focus will, by necessity, 
change in the face of new challenges and priorities within CMS, and inevitably officials and 
staff with key expertise and deep knowledge of the Federal Marketplace will leave CMS or the 
project.  Such changes in priorities and resources reinforce the need for CMS to fully embed 
core management principles in its daily work.  CMSs continued application of these lessons 
will promote further improvement to the Federal Marketplace and also foster future success 
in managing other large projects and CMS programs. 
OIG calls for CMS to continue applying lessons from the HealthCare.gov recovery in 
its management of the Federal Marketplace and broader organization 
LESSON 1  Assign clear project leadership to provide cohesion across tasks and a comprehensive view 
of progress. 
CMSs failure to assign a project leader below the Administrator hobbled the preparation and 
launch of HealthCare.gov.  Personnel across CMS were needed, including policy, technical, 
contracting and communications staff, and a range of contractors.  Lack of clarity about roles 
and the absence of a clear project structure led to staff working at cross-purposes and to 
managers and leaders receiving poor and incomplete information.  Clear visibility, what one of 
the ad hoc team of technical experts called viewing through a single pane of glass, is central 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           61  CALL FOR CONTINUED PROGRESS 
to leadership.  This requires that leadership have a view that includes all functions, and that it 
have the authority to implement and enforce changes when needed. 
To enable this single view and authority during recovery, the CMS Administrator stepped into 
the role of Federal Marketplace leader, attending detailed briefings and making large and 
small decisions about correcting the websites management.  Later, an experienced new CMS 
Principal Deputy Administrator largely filled that role.  Finally in August 2014, 3 months before 
the start of the second open enrollment period, CMS hired a Marketplace CEO to serve in that 
role and also as Director of CCIIO.  The program CEO structure was unusual for CMS, but it 
answered the call for a single leader to manage across the various project operations. 
LESSON 2  Align project and organizational strategies with the resources and expertise available. 
Sound planning for a major project begins, at its earliest stages, with an analysis of project 
needs and how best to align them with the organization and resources.  In its planning stages, 
the HealthCare.gov project faced considerable challenges, including a fixed deadline and 
uncertain funding.  However, in developing policy and establishing goals early in the Federal 
Marketplace project, CMS did not adequately assess the technical and operational tasks 
required.  Poor early decisions included underestimating operational requirements, selecting 
technical components not previously tested on a similar scale, and not securing technology 
capable of increasing website capacity.  CMS was continually correcting for problems, using 
resources to make up ground rather than move forward. 
The lack of effective planning was caused in part by project uncertainties.  The Department 
invested substantial time resolving policy issues that reduced time for the website build.  CMS 
reported that funding uncertainties made it difficult to determine and prioritize scope in 
contracting, in staffing, and in providing overall direction to the project.  Government projects 
commonly face funding uncertainty given the nature of Federal budget decisions.  Thus, it is 
imperative to develop management strategies and contingency plans to account for these 
uncertainties. 
In the crisis of the recovery period, CMS prioritized getting the website functioning well 
enough to enroll consumers in time for them to gain coverage.  Immediately after the first 
open enrollment period ended, CMS made a systematic effort to assess and prioritize 
operational needs, further develop contingency plans, tie policy to operations through 
establishment of the XOC and OECC, and deploy resources to meet goals for the second open 
enrollment period.  CMSs effort to align resources with needs and ruthlessly prioritize was 
critical to improving problems with HealthCare.gov.  Improvement required leadership to gain 
clear and accurate information about costs and benefits, and make well-informed use of the 
limited time and resources available and embed prioritization decisions in all aspects of 
planning, execution, and measuring results. 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           62  CALL FOR CONTINUED PROGRESS 
LESSON 3  Identify and address factors of organizational culture that may affect project success. 
Developing the Federal Marketplace within CMSs enterprise structure both helped and 
hindered the project.  It was useful to gain the expertise of policy and technical staff across 
divisions, but difficult for a new program to establish the needed relationships and lines of 
communication.  When the project was placed in CMS, insufficient attention was paid to the 
cultural shift required to facilitate a new type of program and development approach.  CMSs 
cultural preference for established structures, contained groups, and inflexible procedures 
was often at odds with the needs of a major technological start-up project, which required 
more creativity and flexibility.  Also, divisions among CMS staff and between CMS staff and 
contractors inhibited collaboration and slowed progress. 
Once CMS established clear project leadership, it made a cultural shift toward improved 
communication and transparency, quick assessment of problems, and openness to change.  A 
key to success was incorporating these values in daily work, such as encouraging a badgeless, 
titleless culture that allowed CMS staff and contractors to work together regardless of their 
employer or rank, and the use of data to define results, so that information was tangible and 
objective.  This horizontal structure extended to CMS leadership as officials became more 
deeply engaged with daily staff and contractor work.  CMS documentation and interviews 
indicated a deliberate effort by leadership to engage with all parties, create organizational 
unity, and increase interaction between CMS leadership and staff.  Maintaining this 
movement toward a more open work environment will require CMS to continually assess 
cultural factors and seek feedback from staff and other stakeholders as the project matures, 
CMS takes on new tasks, and the organization continues to evolve. 
LESSON 4  Seek to simplify processes, particularly for projects with a high risk of failure. 
Large and complex IT projects need constant attention to simplify design and operations.  
From inception, nearly every aspect of the Federal Marketplace project was complex and the 
risk of failure was high.  CMSs missteps further complicated the project both conceptually and 
technologically.  Conceptual examples included placing policy and technical staff in separate 
CMS divisions and using many contractors and subcontractors to complete aspects of the 
website build.  Technological examples identified by experts included use of an unsuited, 
inefficient identity management system and an overly complex application process, as well as 
employing poor coding practices and using multiple entities to monitor different aspects of 
the websites performance while not communicating with each other. 
In contrast, CMS leadership, staff, and contractors emphasized simplicity during the recovery.  
CMS established a single, comprehensive command center with robust and accessible 
monitoring tools.  CMS also simplified processes when developing new systems.  For example, 
App 2.0, the new application process, reduced the maximum number of web pages required 
to submit an application from 72 to 16.  The new Scalable Login System for identity 
management was built exclusively for the Federal Marketplace project and therefore did not 
need to include functionality for other purposes that had made the EIDM difficult to modify 
and repair when needed. 
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           63  CALL FOR CONTINUED PROGRESS 
LESSON 5  Integrate policy and technological work to promote operational awareness.  
Throughout the development of HealthCare.gov, progress was thwarted by changing policy 
and business decisions.  This began in the early stages with problems and delays in issuing 
program regulations and guidance and continued as CMS policy and technical staff revisited 
decisions throughout the website build.  Further, CMS policy and technical staff and 
contractors often did not communicate decisions and problems promptly, resulting in later 
complications. 
In the preparation for second open enrollment, CMS systematically determined and prioritized 
desired functionality and quantified the labor required.  This led to a more even distribution of 
work with greater efficiency and less need for rework.  CMS also sought to more effectively 
communicate to program staff what was required technologically to execute policy changes.  
For example, the XOC and OECC enabled policy and technical staff to identify and solve 
problems together during the second open enrollment period. 
LESSON 6  Promote acceptance of bad news and encourage staff to identify and communicate 
problems. 
Key officials failed to recognize the enormity and range of problems with the HealthCare.gov 
websites development and execution.  Communication was fragmented, meaning that not all 
officials received the same information at the same time, but warnings were significant 
enough to warrant further inquiry and action.  However, CMS leadership became desensitized 
to bad news.  CMSs history of overcoming problems likely increased the desensitization.  
Despite tasking multiple entities to assess the projects status, many of whom reported 
potential failure, CMS leadership did not collectively take action or share that with the 
technical staff who might have been able to correct the problems.  CMS staff who were aware 
of problems were reluctant to sound the alarm bell to leadership because they overestimated 
their ability to correct the problems and meet project deadlines. 
During the website recovery, CMS leadership and staff moved to solicit bad news from all 
levels at CMS and contractors.  CMS staff and contractors were encouraged to find and 
communicate problems.  More straightforward communication enabled leaders to better 
assess needs and problems.  It also enabled leaders to set more practical and realistic goals for 
progress, prioritize problem areas, and to better align resources with project needs.  
LESSON 7  Design clear strategies for disciplined execution, and continually measure progress. 
At several junctures, CMS was made aware of problems with the development of 
HealthCare.gov and attempted to take corrective action, but these efforts were largely 
unsuccessful because they were not fully executed.  For example, after criticism that there 
was not clear leadership, the CMS Administrator assigned its COO to head the Federal 
Marketplace project, but the assignment was not formally announced, the position was not 
supported by clear responsibilities, and the designee had an already very large scope of 
responsibility.  The action was taken, but was not executed successfully.  As another example, 
several key officials and entities advised CMS to use a technical systems integrator.  CMSs 
solution was to continue serving as its own systems integrator, but it did not sufficiently 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           64  CALL FOR CONTINUED PROGRESS 
delineate or execute this role.  In both cases and in other examples, CMS made decisions with 
incomplete information, inadequate execution, and insufficient monitoring of results. 
During the website recovery, CMS took steps toward establishing clearer strategies and higher 
standards for execution, and also implemented routine and objective monitoring.  CMS 
rectified key deficiencies by hiring a systems integrator.  CMS also revamped its organization, 
staffing, and monitoring of the website at the XOC, and created stronger lines of 
communication between responsible parties in policy, technical, and communications 
divisions.  This interconnectedness and commitment to measurement led to greater 
accountability for completed tasks and a sense of shared ownership critical to execution and 
success. 
LESSON 8  Ensure effectiveness of IT contracts by promoting innovation, integration, and rigorous 
oversight. 
The most publicly discussed aspect of the HealthCare.gov launch in its aftermath may have 
been the perception that HHS and CMS did not contract and hire for the degree of 
technological expertise required for such a large and complex project.  For HealthCare.gov,  
IT procurement decisions limited the number of companies that CMS solicited for contract 
proposals, and CMS may have over-specified technological approaches in the contracts that 
resulted in use of technology that was poorly matched to the project.  CMS did not fully assess 
the projects IT needs and did not strategize in a way that emphasized innovation and current 
practices; CMSs contract management failed to assess the effectiveness of technological 
decisions, comprehensively plan for coordinating technological work across contractors, and 
sufficiently react to late and deficient products. 
In contrast, in fixing the website, CMS management promoted communication and integration 
among its team of technical experts, CMS, and contracted staff.  In addition, CMS redesigned 
its command center, the XOC, to implement cutting-edge monitoring tools and methods of 
detecting and resolving problems.  CMS also made changes to its management and oversight 
of contracts by establishing new acquisition planning guidance that more clearly defined 
responsibilities of the CMS contracting office and staff; CMS pursued a program management 
culture that ensures that work meets the needs of the project and organization. 
LESSON 9  Develop contingency plans that are quickly actionable, such as redundant and scalable 
systems. 
Problems with complex projects are likely inevitable, yet contingency planning for 
HealthCare.gov was almost nonexistent prior to the breakdown and was late in some aspects 
during and after the website recovery.  This lack of planning meant that CMS had few options 
when HealthCare.gov failed.  Contingency planning enables a realistic assessment of work to 
be completed, and better ensures meeting program objectives, despite problems.  
Contingency plans are only effective, though, if they are practical, evolve as a system matures, 
and are adequately funded for speedy approval of contract changes and other costs. 
Following the launch, CMS more rigorously prepared to mitigate potential problems by 
considering contingencies, building redundant systems, and increasing capacity.  Additional 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           65  CALL FOR CONTINUED PROGRESS 
contingency planning should include clear and actionable plans, stakeholder communication 
strategies, and also funding, given that the process for approving additional funds is often 
delayed and may not be available mid-project due to budget fluctuations. 
LESSON 10  Promote continuous learning to allow for flexibility and changing course quickly when 
needed. 
HealthCare.gov was a novel and complex project that operated with multiple, sustained 
uncertainties.  As CMS moved through development and launch of HealthCare.gov, it relied on 
an existing management and operations structure that could not easily incorporate new 
information and strategies.  CMS staff and contractors continued to carry out plans made 
early in the process, and change did not begin until lack of progress on the project made the 
status quo untenable.  At that point, changes were made in haste, without careful 
consideration, and too late in the process.  CMS leadership neither recognized that changes 
were needed nor employed strategies to change course quickly and thoughtfully. 
Following the launch, CMS and contractors were faced with an urgent need for widespread 
and deep change to processes and technology.  CMS adopted a more open culture of 
continuous learning and quicker acceptance of change, using the website breakdown as an 
inflection point to create a new path.  Leaders also redefined the project scope to set more 
realistic expectations and continued to revise scope as they prepared for the second open 
enrollment period, better ensuring that staff and contractors could execute tasks effectively.  
An environment of continuous learning is especially important when course correction is so 
integral to the project results. 
AGENCY COMMENTS AND OIG RESPONSE
 
CMS concurred with OIGs call for continued progress in applying the lessons that CMS 
learned from the HealthCare.gov recovery in its management of the Federal Marketplace and 
CMSs broader organization.  CMS stated that since the OIG review, it has implemented 
several initiatives to improve its management, striving to incorporate principles aligned with 
this reports lessons learned in its culture, operations, and daily work.  These principles include 
a focus on leadership and accountability, continuous reevaluation of priorities and how the 
project could be more efficient, program measurement, and a flexible and evolving IT strategy 
aligned with policy requirements.  Additionally, CMS stated that it is further developing a 
culture wherein it embraces bad news to help identify and address risks.  CMS notes that 
these guiding principles are likely applicable for other large organizationsprivate or public
that undertake large, complex projects with limited time and resources. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           66  APPENDIX A
 
LIST OF KEY ENTITIES IN IMPLEMENTATION AND OPERATION OF THE FEDERAL MARKETPLACE, 20102015 
U.S. Department of Health and Human Services (HHS):  The ACA tasked HHS to develop the Federal Marketplace. 
  OCIIOOFFICE OF CONSUMER INFORMATION AND INSURANCE OVERSIGHT:  responsible for overseeing implementation 
of ACA provisions related to private insurance, coordination between HHS, issuers, and other Federal and State 
partners, and development of the Federal Marketplace.  OCIIO dissolved in January 2011 when HHS moved the 
Federal Marketplace project to CMS. 
Centers for Medicare & Medicaid Services (CMS):  In January 2011, HHS transferred responsibilities of the 
marketplaces to its largest operating division, CMS, which also administers Medicare and Medicaid. 
  CCIIOCENTER FOR CONSUMER INFORMATION AND INSURANCE OVERSIGHT:  responsible for establishing Federal and 
State marketplace policies and developing business requirements for the website build. 
  OAGMOFFICE OF ACQUISITION AND GRANTS MANAGEMENT:  responsible for developing and overseeing CMS 
acquisition efforts and awarding and administering Federal Marketplace contracts. 
  OCOFFICE OF COMMUNICATIONS:  responsible for CMS internal and external communications, including managing 
call-center operations and HealthCare.gov design and appearance. 
  OISOFFICE OF INFORMATION SERVICES:  responsible for coordinating the technical aspects of the HealthCare.gov 
website build and for implementing and supporting IT needs and enterprise (organization-wide) services 
throughout CMS.  During the January 2015 reorganization, CMS divided OIS responsibilities for IT operations  
(e.g., computer networks and hardware) and IT governance (e.g., software architecture and security) into two 
separate groups rather than placing both in OIS.  The Office of Technology Solutions now has responsibility for  
IT operations and the Office of Enterprise Information has responsibility for IT governance. 
Contractors:  CMS relied extensively on contractors for most of the design, development, testing, software licensing,  
IT security, and support services in the development of the Federal Marketplace.  Key contractors included: 
  AccentureACCENTURE FEDERAL SERVICES, LLC:  responsible for developing the Federally-facilitated Marketplace 
(FFM), in January 2014, including the FFMs three components, Eligibility and Enrollment, Financial Management, 
and Plan Management, as well as website support and operations, from January 2014present. 
  CGI FederalCGI FEDERAL SERVICES, INC.:  responsible for developing the FFM from award of the initial contract 
through the launch and early months of the first open enrollment period, September 2011early January 2014, 
then serving as consultants to Accenture for the FFM, JanuaryMarch 2014. 
  QSSIQUALITY SOFTWARE SERVICES, INC.:  responsible, from September 2011present, for developing the 
Enterprise Identity Management System (EIDM) that enables consumers to create accounts and verify their 
identities on HealthCare.gov and the Hub that routes information requests from the Federal Marketplace to other 
Federal agencies.  Also became the HealthCare.gov systems integrator following the launch, from  
October 2013October 2015. 
  TerremarkTERREMARK FEDERAL GROUP, INC.:  responsible, from November 2012present, for ensuring adequate 
computing capacity and for hosting the infrastructure of large components of the Federal Marketplace, including 
the FFM and the Hub. 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           67  APPENDIX B
 
TIMELINE OF KEY EVENTS IN CMS IMPLEMENTATION OF THE FEDERAL MARKETPLACE, 20102015 
2010  March 23 ACA signed into law 
  April 19 HHS created OCIIO 
  July 1 HHS launched HealthCare.gov Plan Finder website 
   
2011  January 26 HHS moved Federal Marketplace from OCIIO in the Office of the Secretary into CMS 
  September 30 CMS hired CGI Federal to build the FFM and QSSI to build the Hub 
   
2012  January 3 CMS notified CGI Federal of the selection of MarkLogic in a TDL 
  June 18 CMS hired QSSI to build the EIDM 
  June 28 Supreme Court upheld ACA individual mandate 
  September 7 TurningPoint issued first of 11 assessment reports 
  December 14 Deadline for States to submit plans to operate a State marketplace 
   
2013  January 1 Deadline for CMS to approve or conditionally approve State marketplaces 
  March 22 CMS CIO signed 3-year ATO for the EIDM 
  April 30 CMS finalized HealthCare.gov application and released to States 
  May CMS requested CGI Federal deliver Account Lite 
  July 26 HealthCare.gov technical managers requested assistance from other divisions 
  July 30 Failed launch of Account Lite 
  August CMS conducted onsite review of CGI Federal and found poor management practices 
  August 20 CGI Federal presented snapshots of software rather than a live demonstration 
  August 20-23 CMS meeting to reduce scope of HealthCare.gov including CuidadoDeSalud.gov 
  September 6 CMS CIO signed 3-year ATO for Hub 
  September 20 CMS meeting to reduce scope of HealthCare.gov including anonymous shopper tool 
  September 26 CMS requested double computing capacity 
  September 27 CMS Administrator signed 6-month interim ATO for the FFM 
  October 1 Beginning of first open enrollment; HealthCare.gov launch 
  October 116 Government shutdown 
  October 24 CMS hired QSSI as technical systems integrator 
  December 1 CMS improved HealthCare.gov performance 
   
   
   
   
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           68  APPENDIX B 
2014  January 11 Accenture Federal Services began work as FFM contractor 
  March 12 CMS CIO signed 1-year ATO for FFM 
  March 31 End of first open enrollment 
  April 810 CMS meeting to prioritize elements most important to build for second open enrollment 
  June 20 CMS hired new Principal Deputy Administrator 
  August 5 CMS identified concerns at XOC 
  August 26 CMS hired Marketplace Chief Executive Officer 
  September CMS created OECC 
  September 7 CMS soft launch of Application 2.0 for most enrollment consumers 
  September 27 CMS launched IMAS 
  October 7 Pens Down for technical system changes; CMS began end-to-end testing of the FFM 
  November 15 Beginning of second open enrollment 
  December 16 Beginning of auto-reenrollment 
   
2015  February CMS made substantial organizational structure changes 
  February 15 End of second open enrollment 
  February 21 CMS launched SLS 
  March 15April 30 CMS provided a SEP to consumers who did not understand implications of tax penalty 
  June 25 Supreme Court allowed premium tax credits for insurance purchased through all marketplaces 
  November 1 Beginning of third open enrollment (closed January 31, 2016) 
 
 
Listed Acronyms 
ACA:  Patient Protection and Affordable Care Act  IMAS:  Insurance Marketplace Authentication System 
ATO:  Authorization to Operate  OCIIO:  Office of Consumer Information and Insurance Oversight 
CMS:  Centers for Medicare & Medicaid Services  OECC:  Open Enrollment Coordination Center 
HHS:  Department of Health and Human Services  SLS:  Scalable Log-In System 
Hub:  Data Services Hub  SEP:  Special Enrollment Period 
EIDM:  Enterprise Identity Management  TDL:  Technical Direction Letter 
FFM:  Federally-Facilitated Marketplace  XOC:  Exchange Operations Center 
 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           69  APPENDIX C
 
GLOSSARY OF SELECTED FEDERAL MARKETPLACE TERMS 
Term  Definition 
System used to create accounts in HealthCare.gov prior to the first open 
Account Lite 
enrollment period 
An overall plan for satisfying the project mission in the most effective, economical, 
acquisition strategy 
and timely manner 
Group of technology experts recruited by the White House to help repair problems 
ad hoc technology team 
with HealthCare.gov after the October 1, 2013 launch 
Advance Premium Tax Credit  Tax credit for qualifying marketplace consumers, paid monthly to the issuer by the 
(APTC)  Federal Government to offset a portion of the enrollees premium cost   
Method of software development that breaks larger tasks into smaller increments 
agile development 
that are then completed and tested  
Legislation that required establishment of a health insurance exchange for each 
Affordable Care Act (ACA) 
State 
Tool enabling consumers to view some health plan information on the 
anonymous shopper tool 
HealthCare.gov website without creating an account  
Declaration stating publicly that the launching organization (e.g., CMS) has 
Authorization to Operate 
deemed functionality of a system to be sufficient and is accepting any associated 
(ATO) 
risk to the organizations operations or to others involved 
Term used by CMS to signify that all CMS and contracted staff operate as a team 
badgeless culture 
regardless of their job titles or employer status 
Timespan from passage of the ACA in March 2010 through the HealthCare.gov 
breakdown 
launch on October 1, 2013 
Provisions that articulate to developers the program goals, processes, and 
business requirements 
functionality needed for an IT project such as a website 
Group comprised of representatives across CMS divisions who review and approve 
Change Control Board   project changes submitted by CMS staff and contractors that could alter the 
project cost, scope, or schedule of work 
Written order, signed by the contracting officer, directing the contractor to make a 
Contract Change Order 
change that may affect the cost, scope, schedule, or other conditions of the work 
Term used to collectively describe the highest leadership in CMS, including the 
CMS senior leadership  CMS Administrator, Principal Deputy Administrator, Chief of Staff, Chief Operating 
Officer and Chief Information Officer, and in some cases division Directors 
Website-reporting measurement indicating the number of simultaneous users 
concurrent users 
accessing a website at a given time 
Individual using the HealthCare.gov website to create an account, obtain 
consumer  information about health plans, apply for Federal financial assistance, and 
purchase a plan 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           70  APPENDIX C 
Term  Definition 
Contract management document that includes improvements required by 
Corrective Action Plan 
contractors to meet deliverables with adequate quality and timeliness 
Federal financial assistance for qualifying marketplace consumers that lowers  
cost-sharing reductions  out-of-pocket expenses for health care, including deductibles, coinsurance, and 
copayments  
Type of contract that pays the contractor a prenegotiated award fee amount, 
requiring the contractor to bill as it incurs additional labor and material expenses; 
typically selected when the tasks are so uncertain that accepting a contract on the 
cost-plus-fixed-fee contract 
basis of an end product would pose undue risks for contractors, but also thought 
to provide the contractor with less incentive to control costs and provide high 
quality products 
Spanish translation of the HealthCare.gov website that operates at its own web 
CuidadoDeSalud.gov 
address and contains separate provisions for functionality and capacity 
Physical location containing computer servers that provide data storage and 
data center 
processing capacity for HealthCare.gov 
System that routes information requests from the Federal and State marketplaces 
Data Services Hub (Hub) 
and Medicaid and CHIP agencies to other Federal agencies and back 
Number of individuals that are enrolled in marketplace health plans and have paid 
effectuated enrollment 
their first months premiums 
System that enabled consumers to create accounts and verify their identities 
Enterprise Identity 
before they applied for Federal financial assistance and purchase a plan; used by 
Management (EIDM) 
CMS during first and second open enrollment of the Federal Marketplace 
enterprise management  Management strategy that relies on sharing services, such as technology, financial 
structure  management, and contracting services, across the organization and its programs 
System designed to fully automate the financial functions of the Federal 
Enrollment and Payment 
Marketplace, including payments to issuers for Federal financial assistance and 
System (EPS) 
premium stabilization 
Facility in Columbia, MD that serves as the primary HealthCare.gov technological 
Exchange Operations Center 
command center, staffed by CMS and contractors to  coordinate system 
(XOC) 
development, operations, maintenance, and testing 
Marketplace operated by the Federal Government for consumers in States that do 
Federal Marketplace 
not operate a website for residents to enroll in qualified health plans 
System that serves as the core of the Federal Marketplace system, including three 
subsystems that (1) determine consumer eligibility for health plans and Federal 
Federally-facilitated  financial assistance and facilitate enrollment in health plans (Eligibility and 
Marketplace (FFM)  Enrollment); (2) manage health plans with issuers (Plan Management); and  
(3) track and facilitate payments to issuers, including any insurance affordability 
payments (Financial Management) 
Range of operations that can be performed on a computer or other system; 
examples of HealthCare.gov functionality include operations that enable 
functionality 
consumers to obtain information about health plans, apply for Federal financial 
assistance, and purchase a plan 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           71  APPENDIX C 
Term  Definition 
Date on which CMS first opened HealthCare.gov for consumer use to enroll in 
HealthCare.gov launch 
health plans and apply for Federal financial assistance, October 1, 2013 
Type of contract that provides for an indefinite quantity of services for a fixed time 
Indefinite Delivery Indefinite  period and is used when the Government cannot determine above a specified 
Quantity (IDIQ) contract  minimum the precise quantities and/or delivery times of supplies or services that it 
will require 
Insurance Marketplace 
Re-engineered portion of the EIDM that served as a dedicated account creation 
Authentication System 
and identity verification system during second open enrollment 
(IMAS) 
Issuers  Insurance companies that offer health plans to consumers through HealthCare.gov 
Written preliminary contractual agreement that authorizes a contractor to begin 
letter contract 
immediately manufacturing supplies or performing services 
Health insurance exchange wherein individuals can obtain information about 
marketplace 
health plans, apply for Federal financial assistance, and purchase a plan  
CMS program to subsidize the costs of prescription drugs and prescription drug 
Medicare Part D  insurance premiums for Medicare beneficiaries; Medicare Part D was implemented 
in 2006 
HealthCare.gov functionality that CMS considered necessary for the first open 
minimum essential  enrollment period, including allowing consumers to create an account, obtain 
capabilities  information about health plans, apply for Federal financial assistance, and 
purchase a plan 
Approach to software development that uses models to automatically generate 
Model-Driven Architecture 
computer code for system development; typically used in conjunction with 
(MDA) 
developer-written code to save time and reduce human error 
Individuals or organizations, funded through Federal grants, that help consumers 
navigators  enroll in health plans through the Federal Marketplace or State marketplaces, and 
provide guidance and education to raise awareness about the marketplaces 
Nontraditional, document-oriented database platform that uses nontabular 
NoSQL database platform 
records, unlike a relational database that uses tables to store and index data 
Subgroup within CMS CCIIO created by CMS in September 2014 to monitor and 
Open Enrollment 
coordinate resolution of all open enrollment issues by translating information to 
Coordination Center (OECC) 
and from the policy, technology, and operations teams 
Period of time during which individuals may enroll in a health plan; dates included: 
  First open enrollment:  October 1, 2013March 31, 2014 
open enrollment period 
  Second open enrollment:  November 15, 2014February 15, 2015 
  Third open enrollment:  November 1, 2015January 31, 2016 
Unfounded reliance on former ways of doing things that prevents adaptation to 
path dependency 
new conditions 
Term CMS used for signifying the final date for technical staff to make system 
pens down 
coding changes prior to testing 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           72  APPENDIX C 
Term  Definition 
Browsing website created in 2010 that provided health plan information to 
Plan Finder website  consumers but without the functionality to purchase plans or to apply for Federal 
financial assistance 
Private health insurance plan offered through a marketplace and certified by CMS 
qualified health plan  
or States as meeting certain standards and that cover a core set of benefits, 
(health plan) 
including doctor visits, preventive care, hospitalization, and prescriptions 
Analysis by experts to ensure that proposed rate increases of marketplace health 
rate review 
plans by issuers are based on reasonable cost assumptions 
recovery  Timespan from October 1, 2013 through the end of second open enrollment 
Technique used to restructure existing computer software code in order to correct 
refactor 
and streamline the code 
Meetings held prior to the first open enrollment period in summer 2013 to further 
reprioritization meeting 
reduce the scope of the HealthCare.gov build to deliver only essential functionality 
Method for determining the most critical elements of a project to be completed, 
given the available time and budget; used to make dramatic cuts to the 
ruthless prioritization 
HealthCare.gov project scope in preparation for the second open enrollment 
period 
New account creation and authentication system created exclusively for 
Scalable Login System (SLS)  HealthCare.gov to improve website performance and its ability to accommodate 
large changes in number of users (replaced EIDM) 
Small Business Health  Program that provides health plan selection and enrollment for employees of 
Option Program (SHOP)  companies with fewer than 50 full-time-equivalent workers 
Contract used when an agencys need for certain supplies or services is of such an 
unusual and compelling urgency that the Government would be seriously injured 
sole source contract 
unless the agency is permitted to limit the sources from which it solicits bids or 
proposals 
Time period outside of normal open enrollment period during which consumers 
special enrollment period   who experience certain life changes or other circumstances can purchase health 
insurance 
Type of work environment or culture that encourages creativity and flexibility over 
start-up 
rigid management methods and an established hierarchy 
Marketplace operated by a State for its residents to obtain information about 
State marketplace 
health plans, apply for Federal financial assistance, and purchase a plan 
Contract management document that includes contractor work to be performed; 
location of work; period of performance; schedule for completion (delivery) of 
statement of work 
work; applicable performance standards; and any special requirements  
(e.g., security clearances, travel) 
Technical guidance provided to a contractor that is meant to clarify, define, or give 
technical direction letter 
specific direction within the scope of the contract as written; does not result in 
(TDL) 
changes to the cost, terms, or conditions of the contract 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           73  APPENDIX C 
Term  Definition 
List of the exact functions, derived from the business requirements, that are used 
technical specifications  by developers to write software code that creates and supports the website 
systems 
Entity that coordinates operations, ensuring that those responsible for various 
technical systems integrator  technical aspects of the project communicate their activities, schedules, and needs 
to each other and that work aligns with project goals 
testing environment  Computing and data storage resources devoted to website system testing 
CMS teams that include staff and contractors from various offices working 
towers 
together regarding a particular aspect of the project 
Contract agreement that is not binding solely because the Government 
unauthorized commitment  representative who made it lacked the authority to enter into that agreement on 
behalf of the Government 
Website function used when HealthCare.gov website traffic overloaded capacity; 
waiting room  consumers placed in a waiting room were unable to navigate further in the 
website until website traffic had reduced 
Source:  HHS and CMS project management documentation and correspondence, 2015.  
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           74  APPENDIX D
 
COMMENTS FROM THE CENTERS FOR MEDICARE & MEDICAID SERVICES, OCTOBER 7, 2015 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           75  APPENDIX D 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           76  ACKNOWLEDGEMENTS
 
 
This report was prepared under the direction of Kevin Golladay, Regional Inspector General for Evaluation and 
Inspections in the Dallas regional office, and Ruth Ann Dorrill, Deputy Regional Inspector General. 
Ruth Ann Dorrill served as the team leader for this study, and Ben Gaddis and Jennifer Hagen served as the lead 
analysts.  Other Office of Evaluation and Inspections staff from the Dallas regional office who conducted the study 
include Amy Ashcraft, Malinda Hicks, and Jeremy Moore.  Office of Counsel staff who provided support include  
Juliet Hodgkins, Elizabeth Holahan, Lonie Kim, Andrew VanLandingham, and Paul Westfall.  Other OIG staff who 
provided support include Assistant Inspector General Erin Bliss, Heather Barton, Rose Folsom, Evan Godfrey, and  
Maria Maddaloni.
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           77  ENDNOTES
 
 
 
1 P.L. No. 111-148 (Mar. 23, 2010), as amended by the Health Care and Education Reconciliation Act of 2010,  P.L. No. 111-152 (Mar. 30, 2010), 
collectively referred to as the Affordable Care Act (ACA). 
2 Ibid.  1501, 1513. 
3 Ibid.  1311(a), (b). 
4 Ibid.  1321(c). 
5 Private health insurance plans certified as meeting certain standards and covering a core set of benefits including doctor visits, preventive care, 
hospitalization, and prescriptions. 
6 ACA  1401, 1402. 
7 The Henry J. Kaiser Family Foundation, State Decisions on Health Insurance Marketplaces and the Medicaid Expansion, December 17, 2015.  Accessed at  
http://kff.org/health-reform/state-indicator/state-decisions-for-creating-health-insurance-exchanges-and-expanding-medicaid/ on January 6, 2016.  CMS, 
Hawaii: For 2016 insurance coverage, use HealthCare.gov to apply and enroll.  Accessed at https://www.healthcare.gov/hawaii-2016/ on January 6, 2016. 
8 Ibid. 
9 ACA  1311(c)(6)(A),(B). 
10 45 CFR  155.410. 
11 Ibid. 
12 CMS, Special Enrollment Period.  Accessed at https://www.healthcare.gov/glossary/special-enrollment-period/ on February 12, 2016. 
13 For example, it allowed those facing possible tax penalties to purchase health plans during a SEP from March 15April 30, 2015.  CMS, CMS Announces 
Special Enrollment Period for Tax Season (English and Espanol), February 20, 2015.  Accessed at 
https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-releases/2015-Press-releases-items/2015-02-20.html on February 8, 2016. 
14 National Federation of Independent Business v. Sebelius, 132 S. Ct. 2566 (June 28, 2012). 
15 King v. Burwell, 135 S. Ct. 2480 (June 25, 2015). 
16 76 Fed. Reg. 4703 (Jan. 26, 2011). 
17 HHS, HHS Fiscal Year (FY) 2016 Budget in Brief.  Accessed at http://www.hhs.gov/about/budget/fy2016/budget-in-brief/index.html#overview on  
August 14, 2015. 
18 ACA  1411, 1412. 
19 Fox News, 26 States Join Suit Against Obama Health Law, January 19, 2011.  Accessed at http://www.foxnews.com/politics/2011/01/18/states-join-
obama-health-care-lawsuit-fla/ on August 14, 2015.  The New York Times, House Votes to Send Bill to Repeal Health Law to Obamas Desk, January 6, 
2016, Accessed at http://www.nytimes.com/2016/01/07/us/politics/house-votes-to-send-bill-to-repeal-health-law-to-obamas-desk.html?_r=0 on 
February 16, 2016.  
20 ACA  1321. 
21 ACA  1103. 
22 CMS, HHS Launches New Consumer Focused Health Care Website www.HealthCare.gov, July 1, 2010.  Accessed at http://wayback.archive-
it.org/3926/20131018160711/http://www.hhs.gov/news/press/2010pres/07/20100701h.html# on September 14, 2015. 
23 OIG, Oversight of Private Health Insurance Submissions to the HealthCare.gov Plan Finder, OEI-03-11-00560, April 2013. 
24 75 Fed. Reg. 20364 (April 19, 2010). 
25 These regulations were often tri-department regulations issued in conjunction with other Federal agencies. 
26 45 CFR part 155 and part 156 generally describe marketplace establishment standards and functions and issuer standards and responsibilities. 
27 CMS, Technical Implementation Letters.  Accessed at https://www.cms.gov/CCIIO/Resources/Technical-Implementation-Letters/ on August 14, 2015. 
28 76 Fed. Reg. 4703 (Jan. 26, 2011). 
29 The $1 billion Health Insurance Reform Implementation Fund (HIRIF) was created and funded by Section 1005 of Health Care and Education 
Reconciliation Act of 2010. 
30 Congressional Budget Office, Updated Estimates of the Effects of the Insurance Coverage Provisions of the Affordable Care Act, February 2014.  
Accessed at https://www.cbo.gov/sites/default/files/cbofiles/attachments/45010-breakout-AppendixB.pdf on February 8, 2016. 
31 The Departments of Labor, HHS, and Treasury published joint regulations relating to group health plans and health insurance issuers in the group and 
individual markets. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           78  ENDNOTES 
 
32 Section 1302(b) of ACA directs the Secretary of Health and Human Services to define essential health benefits. Non-grandfathered plans in the 
individual and small group markets both inside and outside of the marketplaces, Medicaid benchmark and benchmark-equivalent, and Basic Health 
Programs must cover the essential health benefits beginning in 2014.  Section 1302(b)(1) provides that essential health benefits must include items and 
services within at least the following 10 categories: (1) ambulatory patient services; (2) emergency services; (3) hospitalization; (4) maternity and 
newborn care; (5) mental health and substance use disorder services, including behavioral health treatment; (6) prescription drugs; (7) rehabilitative and 
habilitative services and devices; (8) laboratory services; (9) preventive and wellness services and chronic disease management; and, (10) pediatric 
services, including oral and vision care. 
33 Kaiser Health News, Rollout Resembles Some of the Problems of Medicare Part D, July 12, 2013.  Accessed at http://khn.org/morning-
breakout/health-reform-implementation-and-outreach/ on August 26, 2015. 
34 CMS, CMS Fast Facts, July 2015.  Accessed at https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/CMS-Fast-
Facts/index.html on January 11, 2016.  CMS, National Health Expenditures 2014 Highlights.  Accessed at https://www.cms.gov/Research-Statistics-Data-
and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/Downloads/highlights.pdf on January 11, 2016. 
35 The White House, President Obama Announces Recess Appointments to Key Administration Positions, July 7, 2010.  Accessed at 
https://www.whitehouse.gov/the-press-office/president-obama-announces-recess-appointments-key-administration-positions-0 on October 26, 2015.  
The White House, President Obama Announces More Key Administration Posts, November 23, 2011.  Accessed at https://www.whitehouse.gov/the-
press-office/2011/11/23/president-obama-announces-more-key-administration-posts on October 26, 2015. 
36 ACA  1321. 
37 Office of the Governor of the State of Alabama, Press Release, Governor Bentley Declines Decision on ACA Component; Cites Lack of Proper Options & 
Information, October 1, 2012.  Accessed at http://governor.alabama.gov/newsroom/2012/10/governor-bentley-declines-decision-on-aca-component-
cites-lack-of-proper-options-information/ on August 21, 2015. 
38 U.S. House of Representatives Committee on Ways and Means, Subcommittee on Health, Hearing on Implementation of Health Insurance Exchanges 
and Related Provisions, September 12, 2012.  Accessed at http://waysandmeans.house.gov/hearing-on-implementation-of-health-insurance-exchanges-
and-related-provisions/ on August 14, 2015. 
39 78 Fed. Reg. 65046 (Oct. 30, 2013). 
40 The deadline for States to submit a Blueprint application for State marketplaces was December 14, 2012.  See CMS Technical Implementation Letters, 
Letter to the Governors on Exchange Blueprints, November 9, 2012.  Accessed at https://www.cms.gov/CCIIO/Resources/Technical-Implementation-
Letters/Downloads/exchange-blueprint-letter.pdf on August, 21, 2015. 
41 OIG, CMS Did Not Always Manage and Oversee Contractor Performance for the Federal Marketplace as Required by Federal Requirements and Contract 
Terms, A-03-14-03001, September 2015.  Interviews with CMS staff and CMS correspondence. 
42 OIG, Federal Marketplace:  Inadequacies in Contract Planning and Procurement, OEI-03-14-00230, January 2015. 
43 Ibid. 
44 Ibid.  Although CMSs contracting approaches were permitted under Federal regulations, its procurement decisions may have limited the number of 
qualified companies that competed for contracts and the number of technically acceptable proposals from which CMS could choose.   
45 Ibid. 
46 Ibid. 
47 Federal Acquisition Regulation (FAR)  34.004.   
48 OIG, Federal Marketplace:  Inadequacies in Contract Planning and Procurement, OEI-03-14-00230, January 2015. 
49 FAR  16.306(a). 
50 CMS, Active Contributors.  Accessed at https://www.cms.gov/Research-Statistics-Data-and-Systems/CMS-Information-
Technology/SystemLifecycleFramework/downloads/ActiveContributors.pdf on August 10, 2015. 
51 FAR  1.602.  See also HHS Acquisition Regulation (HHSAR), 48 CFR subpart 342.70.  See also CMS, Active Contributors.  Accessed at 
https://www.cms.gov/Research-Statistics-Data-and-Systems/CMS-Information-
Technology/SystemLifecycleFramework/downloads/ActiveContributors.pdf on August 10, 2015. 
52 FAR  1.602-2(d), 1.604. 
53 FAR  43.102(a). 
54 OIG, CMS Did Not Always Manage and Oversee Contractor Performance for the Federal Marketplace as Required by Federal Requirements and Contract 
Terms, A-03-14-03001, September 2015. 
55 National Federation of Independent Business v. Sebelius, 132 S. Ct. 2566 (June 28, 2012). 
56 CMSs central repository is referred to as the Collaborative Application Lifecycle Tool (CALT). 
57 CGI Federal initially estimated a cost overrun of $36 million, but ultimately the overrun was $28 million.  See OIG, CMS Did Not Always Manage and 
Oversee Contractor Performance for the Federal Marketplace as Required by Federal Requirements and Contract Terms, A-03-14-03001, September 2015. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           79  ENDNOTES 
 
58 FAR  1.602-3(a). 
59 FAR  1.602-3.  HHSAR 301.602-3. 
60 78 Fed. Reg. 12834 (Feb. 25, 2013); CMS, Guidance on State Alternative Applications for Health Coverage, June 2013.  Accessed at 
https://www.medicaid.gov/Federal-Policy-Guidance/Downloads/SHO-06192013.pdf on January 22, 2016. 
61 U.S. House of Representatives, House Energy and Commerce Subcommittee on Oversight and Investigations, Security of HealthCare.gov,  
November 19, 2013. 
62 Entity that coordinates operations, ensuring that those responsible for various aspects of the project communicate their activities, schedules, and 
needs to each other. 
63 U.S. House of Representatives, House Energy and Commerce Committee, PPACA Implementation Failures: Answers from HHS, October 30, 2013.  
Interviews with CMS staff and CMS correspondence. 
64 A Corrective Action Plan is a contract management document that includes improvements required by contractors to meet deliverables with adequate 
quality and timeliness. 
65 U.S. House of Representatives, Committee on Oversight and Reform, Behind the Curtain of the HealthCare.gov Rollout, September 17, 2014, p. iii.  
Accessed at http://oversight.house.gov/wp-content/uploads/2014/09/Healthcare-gov-Report-Final-9-17-14.pdf on August 24, 2015. 
66 CMS, Launching the Small Business Health Option Program (SHOP) Marketplace, September 26, 2013.  Accessed at 
https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-releases/2013-Press-releases-items/2013-09-26.html on November 3, 2015. 
67 The initial website launch date of July 1, 2013, was delayed to October 1 while CMS was still early in the process (mid-2012), but between January
October 2013 there was no formal discussion of delay. 
68 ACA  1311(b)(1). 
69 The Washington Post, The last time the government expanded health care, it was also kind of a disaster, October 11, 2013.  Accessed at 
http://www.washingtonpost.com/news/wonkblog/wp/2013/10/11/the-last-time-the-government-expanded-health-care-it-was-also-kind-of-a-disaster/ 
on October 27, 2015.  The Washington Post, Web-Based Comparison of Prescription Plans Delayed, October 14, 2005.  Accessed at 
http://www.washingtonpost.com/wp-dyn/content/article/2005/10/13/AR2005101301861.html on October 27, 2015. 
70 National Institute of Standards and Technology.  Guide for Applying the Risk Management Framework to Federal Information Systems.  Accessed at 
http://csrc.nist.gov/publications/nistpubs/800-37-rev1/sp800-37-rev1-final.pdf on January 21, 2016. 
71 CMS, Office of the Chief Information Security Officer, CMS Information Security (IS) Authorization To Operate Package Guide, version 3.0,  
December 1, 2011.  Accessed at https://www.cms.gov/Research-Statistics-Data-and-Systems/CMS-Information-
Technology/InformationSecurity/downloads/ATO_Package_Guide.PDF on August 29, 2015.  Interviews with CMS staff and CMS correspondence. 
72 USA Today, Obama Adviser: Demand overwhelmed HealthCare.gov, October 6, 2013.  Accessed at 
http://www.usatoday.com/story/news/nation/2013/10/05/health-care-website-repairs/2927597/ on August 14, 2015.   
73 The White House, Remarks by the President on the Affordable Care Act and the Government Shutdown, October 1, 2013.  Accessed at 
https://www.whitehouse.gov/the-press-office/2013/10/01/remarks-president-affordable-care-act-and-government-shutdown on September 9, 2015. 
74 Reuters, Obama turns to trusted aide for tech surge to fix healthcare website, October 22, 2013.  Accessed at 
http://www.reuters.com/article/2013/10/23/us-usa-healthcare-idUSBRE99K0M220131023 on January 20, 2016.  HHS, A Technology Surge for 
HealthCare.gov, October 22, 2013.  Accessed at 
http://web.archive.org/web/20140820173427/http://www.hhs.gov/digitalstrategy/blog/2013/10/technology-surge-for-healthcare-gov.html on  
February 11, 2016. 
75 The technical experts recruited by the White House largely volunteered their time, but in keeping with Federal employment standards were employed 
at an hourly rate by QSSI. 
76 The White House, An Improbable Public Interest Start Up, January 2015.  Accessed at https://www.whitehouse.gov/blog/2015/01/21/improbable-
public-interest-start on August 14, 2015.   
77 Recording of keynote address, Velocity, Building a Faster, Stronger Web, September 16, 2014.  Accessed at http://velocityconf.com/velocityny2014 
on June 11, 2015. 
78 Time, Code Red: Nightmare Launch of HealthCare.gov and the Team that Figured Out How to Fix It.  Issue 183, no. 9.  March 4, 2014.  Interviews with 
CMS staff and CMS correspondence. 
79 The New York Times, From the Start, Signs of Trouble at Health Portal, October 12, 2013.  Accessed at 
http://www.nytimes.com/2013/10/13/us/politics/from-the-start-signs-of-trouble-at-health-portal.html?_r=0 on October 13, 2015.  The Washington Post, 
Meet CGI Federal, the company behind the botched launch of HealthCare.gov, October 16, 2013.  Accessed at 
http://www.washingtonpost.com/news/wonkblog/wp/2013/10/16/meet-cgi-federal-the-company-behind-the-botched-launch-of-healthcare-gov/ on 
October 13, 2015.  The Wall Street Journal, Federal Health Site Stymied by Lack of Direction, October 28, 2013.  Accessed at 
http://www.wsj.com/articles/SB10001424052702304682504579158043537719338 on October 13, 2015. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           80  ENDNOTES 
 
80 U.S. House of Representatives, Committee on Oversight and Government Reform, Examining the IRSs Role in Implementing and Enforcing Obamacare, 
October 9, 2013.  Accessed at https://oversight.house.gov/hearing/examining-irss-role-implementing-enforcing-obamacare/ on November 18, 2015.   
U.S. House of Representatives, Committee on Energy and Commerce, PPACA Implementation Failures: Didnt Know or Didnt Disclose? October 24, 2013.  
Accessed at https://energycommerce.house.gov/hearings-and-votes/hearings/ppaca-implementation-failures-didn-t-know-or-didn-t-disclose on 
November 18, 2015.  U.S. House of Representatives, Committee on Ways and Means, Affordable Care Act Implementation, October 29, 2013.  Accessed 
at http://waysandmeans.house.gov/UploadedFiles/102913_Tavenner_Testimony.pdf on November 18, 2015.  U.S. House of Representatives, Committee 
on Energy and Commerce, PPACA Implementation Failures: Answers from HHS, October 30, 2013.  Accessed at 
https://energycommerce.house.gov/hearings-and-votes/hearings/ppaca-implementation-failures-answers-hhs on November 18, 2015.  U.S. Senate, 
Committee on Health, Education, Labor and Pensions, The Online Federal Health Insurance Marketplace: Enrollment Challenges and the Path Forward, 
November 5, 2013.  Accessed at http://www.hhs.gov/asl/testify/2013/11/t20131105.html on November 18, 2015.  U.S. Senate, Committee on Finance, 
Health Insurance Exchanges: An Update from the Administration, November 6, 2013.  Accessed at 
http://www.finance.senate.gov/hearings/hearing/?id=3dd91089-5056-a032-5290-f158359b9247 on November 18, 2015. U.S. House of Representatives, 
Committee on Oversight and Government Reform, Obamacare Implementation: The Rollout of HealthCare.gov, November 13, 2013.  Accessed at 
https://oversight.house.gov/hearing/obamacare-implementation-rollout-healthcare-gov/ on November 18, 2015.  U.S. House of Representatives, 
Committee on Energy and Commerce, Obamacare Implementation Problems:  More than Just a Broken Website, November 14, 2013.  Accessed at 
https://energycommerce.house.gov/hearings-and-votes/hearings/obamacare-implementation-problems-more-just-broken-website on November 18, 
2015.  U.S. House of Representatives, Committee on Energy and Commerce, Security of HealthCare.gov, November 19, 2013.  Accessed at 
https://energycommerce.house.gov/hearings-and-votes/hearings/security-healthcaregov on November 18, 2015.  U.S. Senate, Committee on Small 
Business and Entrepreneurship, Affordable Care Act Implementation: Examining How to Achieve a Successful Rollout of the Small Business Exchanges, 
November 20, 2013.  Accessed at http://www.sbc.senate.gov/public/index.cfm?p=Hearings&ContentRecord_id=ba0fad5b-78ce-4c50-ad93-
ed326595eb52&ContentType_id=14f995b9-dfa5-407a-9d35-56cc7152a7ed&Group_id=43eb5e02-e987-4077-b9a7-
1e5a9cf28964&MonthDisplay=11&YearDisplay=2013 on November 18, 2015. 
81 The Center on Health Insurance Reforms, Report from the First Year of Navigator Technical Assistance Project:  Lessons Learned and Recommendations 
for the Next Year of Enrollment, October 2014.  Accessed at http://www.rwjf.org/content/dam/farm/reports/issue_briefs/2014/rwjf416166 on August 14, 
2015. 
82 USA Today, Zients: HealthCare.gov problems to be fixed in a month, October 25, 2013.  Accessed at 
http://www.usatoday.com/story/news/nation/2013/10/25/zients-update-healthcaregov-website/3187867/ on August 25, 2015.  White House, 
Statement by the President on the Affordable Care Act, November 14, 2013.  Accessed at https://www.whitehouse.gov/the-press-
office/2013/11/14/statement-president-affordable-care-act on February 16, 2016.   
83 Time, Code Red: Nightmare Launch of HealthCare.gov and the Team that Figured Out How to Fix It.  Issue 183, no. 9.  March 4, 2014. 
84 HHS, Progress on Capacity, November 22, 2013.  Accessed at http://wayback.archive-
it.org/3909/20140515212425/http://www.hhs.gov/digitalstrategy/blog/2013/11/progress-on-capacity.html on October 14, 2015. 
85 HHS, A Direct New Path to SHOP Marketplace Coverage, November 27, 2013.  Accessed at 
http://web.archive.org/web/20150905170440/http://www.hhs.gov/healthcare/facts/blog/2013/11/direct-new-path-to-shop-marketplace.html on 
November 3, 2015. 
86 CMS, HealthCare.gov Progress and Performance Report, December 1, 2013.  Accessed at 
https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-releases/2013-Press-releases-items/2013-12-01.html on August 31, 2015. 
87 The Washington Post, Can HealthCare.gov actually handle 50,000 users at a time?  December 2, 2013.  Accessed at 
http://www.washingtonpost.com/news/wonkblog/wp/2013/12/02/can-healthcare-gov-actually-handle-50000-users-at-a-time/ on October 5, 2015. 
88 45 CFR  155.410(c)(1); see also HHS Press Release, Administration takes steps to ensure Americans signing up through the marketplace have coverage 
and access to the care they need on January 1, December 12, 2013.  Accessed at http://www.hhs.gov/news/press/2013pres/12/20131212a.html on 
August 7, 2015.  See also Juliet Eilperin, Administration Extends Enrollment Deadline Once Again, Washington Post, Dec. 24, 2013.  Accessed at 
http://www.washingtonpost.com/news/post-politics/wp/2013/12/24/2-million-visited-healthcare-gov-on-monday-as-deadline-looms/ on August 7, 2015.  
89 HHS, ASPE, Health Insurance Marketplace: Summary Enrollment Report for the Initial Annual Open Enrollment Period, May 1, 2014.  Accessed at 
http://aspe.hhs.gov/sites/default/files/pdf/76876/ib_2014Apr_enrollment.pdf on October 9, 2015. 
90 An inconsistency does not mean that an applicant provided incorrect information or is receiving Federal financial assistance inappropriately. 
91 OIG, Marketplaces Faced Early Challenges Resolving Inconsistencies with Applicant Data, OEI-01-14-00180, June 2014. 
92 Without the ability to resolve inconsistencies, the marketplace cannot ensure that an applicant meets each of the eligibility requirements for 
enrollment in a health plan. 
93 CMS reported that although not fully automated, the eligibility system was completed in May 2014 and that contractors then began processing the 
outstanding inconsistencies.  CMS reported that by September 2014, they had resolved the majority of inconsistencies and sent coverage termination 
notices to the 115,000 individuals who failed to submit supporting documentation. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           81  ENDNOTES 
 
94 OIG, Not All of the Federally Facilitated Marketplaces Internal Controls Were Effective in Ensuring That Individuals Were Properly Determined Eligible 
for Qualified Health Plans and Insurance Affordability Programs, A-09-14-01011, August 2015. 
95 CMS concurred with OIGs recommendation that CMS take action to improve the Federal Marketplaces internal controls related to verifying 
applicants eligibility and resolving and expiring inconsistencies.  CMS replied that it has an extensive resolution process in place to resolve data matching 
issues and is continuously improving and refining those processes.  For example, even when a consumer is not legally required to provide a Social Security 
number (SSN), CMS highly recommends to consumers that they provide an SSN as part of the application process for everyone on the application who has 
one, since providing an SSN enables the FFM to use more efficient electronic verification processes.  This should decrease the number of data matching 
issues.   
96 HHS, ASPE, Health Insurance Marketplace: Summary Enrollment Report for the Initial Annual Open Enrollment Period, May 1, 2014.  Accessed at 
http://aspe.hhs.gov/sites/default/files/pdf/76876/ib_2014Apr_enrollment.pdf on October 9, 2015. 
97 OIG, Federal Marketplace:  Inadequacies in Contract Planning and Procurement, OEI-03-14-00230, January 2015. 
98 Ibid.  
99 Ibid.  These contract values do not include option years that CMS later decided not to exercise.   
100 Ibid.   
101 OIG, CMS Did not Identify All Federal Marketplace Contract Costs and Did Not Properly Validate the Amount to Withhold for Defect Resolution on the 
Principal Federal Marketplace Contract, A-03-14-03002, September 2015. 
102 OIG, CMS Could Not Effectively Ensure that Advance Premium Tax Credit Payments Made Under the Affordable Care Act Were Only for Enrollees Who 
Paid their Premiums, A-02-14-02025, December 2015. 
103 45 CFR  155.410(f). 
104 45 CFR  155.335.  See also CMS, Guidance on Annual Redeterminations for Coverage for 2015, June 26, 2014.  Accessed at 
https://www.cms.gov/CCIIO/Resources/Regulations-and-Guidance/Downloads/2014-0626-Guidance-on-annual-redet-option-2015-FINAL.pdf on 
September 21, 2015. 
105 The Hill, Obamacare sign-ups hit 6.4 million, December 23, 2014.  Accessed at http://thehill.com/policy/healthcare/227960-obamacare-counts-64m-
sign-ups on August 25, 2015. 
106 The Washington Post, Nevada will join federal HealthCare.gov exchange, May 21, 2014.  Accessed at 
http://www.washingtonpost.com/blogs/govbeat/wp/2014/05/21/nevada-will-join-federal-healthcare-gov-exchange/ on September 10, 2015.  MSNBC, 
Oregon to ditch troubled health insurance exchange, April 24, 2014.  Accessed at http://www.msnbc.com/msnbc/oregon-nevada-massachusetts-
maryland-struggle-obamacare-exchanges on September 10, 2015.  Kaiser Health News, Red State Idaho Launches Its Own Obamacare Exchange, 
November 12, 2014.  Accessed at http://khn.org/news/red-state-idaho-launches-its-own-obamacare-exchange/ on September 10, 2015. 
107 HHS, ASPE, Health Insurance Marketplace 2015: Average Premiums After Advance Premium Tax Credits through January 30 in 37 States Using the 
HealthCare.gov Platform, February 9, 2015.  Accessed at https://aspe.hhs.gov/report/health-insurance-marketplace-2015-average-premiums-after-
advance-premium-tax-credits-through-january-30-37-states-using-healthcaregov-platform on September 10, 2015. 
108 Kevin Lucas, Justin Giovannelli & Sean Miskell, After a Slow Start, Federal Small Business Health Insurance Marketplace Offers New and Improved 
Functions. Commonwealth Fund, February 19, 2015.  Accessed at www.commonwealthfund.org/publications/blog/2015/feb/federal-shop-marketplace-
offers-new-and-improved-functions on August 13, 2015. 
109 Kaiser Family Foundation, Repayments and Refunds:  Estimating the Effects of 2014 Premium Tax Credit Reconciliation.  Accessed at 
http://kff.org/health-reform/issue-brief/repayments-and-refunds-estimating-the-effects-of-2014-premium-tax-credit-reconciliation/ on August 24, 2015. 
110 U.S. House of Representatives, Committee on Ways and Means, Subcommittee on Health, Written Testimony of John A. Koskinen, Commissioner IRS, 
September 10, 2014.  See also JoAnn Volk, Sabrina Corlette, Sandy Ahn, and Tricia Brooks, The Center on Health Insurance Reforms, Report from the First 
Year of Navigator Technical Assistance Project:  Lessons Learned and Recommendations for the Next Year of Enrollment, October 2014.  Accessed at 
http://www.rwjf.org/content/dam/farm/reports/issue_briefs/2014/rwjf416166 on August 14, 2015. 
111 45 CFR  155.215(b)(2). 
112 Managed Healthcare, Websites improve for second open enrollment cycle, January 21, 2015.  Accessed at 
http://managedhealthcareexecutive.modernmedicine.com/managed-healthcare-executive/news/websites-improve-second-open-enrollment-cycle on 
October 14, 2015.  
113 Associated Press, Surge in Health Law Sign-Ups, December 16, 2014.  Accessed at 
http://bigstory.ap.org/article/9872c1aeec284f7490692c83cc458e97/surge-health-law-sign-ups on November 18, 2015. 
114 HHS, By the Numbers: Open Enrollment for Health Insurance, February 2015.  Accessed at 
http://web.archive.org/web/20150215101950/http://www.hhs.gov/healthcare/facts/factsheets/2015/02/open-enrollment-by-the-numbers.html on 
February 12, 2016. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           82  ENDNOTES 
 
115 CMS, June 30, 2015 Effectuated Enrollment Snapshot, June 30, 2015.  Accessed at https://www.cms.gov/Newsroom/MediaReleaseDatabase/Fact-
sheets/2015-Fact-sheets-items/2015-09-08.html on November 5, 2015.  CNN, Obamacare enrollment unlikely to meet 2015 goal, November 10, 2014.  
Accessed at http://money.cnn.com/2014/11/10/news/economy/obamacare-enrollment-2015/ on November 5, 2015.  Congressional Budget Office, 
Updated Estimates of the Insurance Coverage Provisions of the Affordable Care Act, February 2014.  Accessed at 
https://www.cbo.gov/sites/default/files/cbofiles/attachments/45010-breakout-AppendixB.pdf on November 5, 2015. 
116 HHS, ASPE, Health Plan Choice and Premiums in the 2015 Health Insurance Marketplace, January 8, 2015.  Accessed at 
http://aspe.hhs.gov/sites/default/files/pdf/77176/healthPremium2015.pdf on November 5, 2015. 
117 CMS, Secretary Burwell announces steps to bolster management and accountability ahead of the 2015 open enrollment period, June 20, 2014.  
Accessed at http://www.hhs.gov/news/press/2014pres/06/20140620a.html on September 3, 2015. 
118 CMS, HHS and CMS Announce New Members of the Management Team Ahead of 2015 Open Enrollment, August 26, 2014.  Accessed at 
http://www.hhs.gov/news/press/2014pres/08/20140826b.html on September 3, 2015. 
119 The HHSAR proposed rule was published on March 2, 2015 (80 Fed. Reg. 11265).  Comments to the proposed rule closed on May 1, 2015. 
120 CMS, Active Contributors.  Accessed at https://www.cms.gov/Research-Statistics-Data-and-Systems/CMS-Information-
Technology/SystemLifecycleFramework/downloads/ActiveContributors.pdf on August 10, 2015. 
121 CMS, Office of Enterprise Information, August 4, 2015.  Accessed at https://www.cms.gov/About-CMS/Agency-Information/CMSLeadership/Office-
OEI.html on September 3, 2015.  EC-Council, Emery Csulak Chief Information Security Officer/Senior Privacy Official.  Accessed at 
http://ciso.eccouncil.org/ciso-events/ciso-summit-capital-region/speakers-capital-region/emery-csulak/ on September 3, 2015. 
122 OIG, Health Insurance Marketplaces Generally Protected Personally Identifiable Information But Could Improve Certain Information Security Controls, 
A-18-14-30011, September 2014. 
123 CMS, CMS Creates New Chief Data Officer Post, November 19, 2014.  Accessed at https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-
releases/2014-Press-releases-items/2014-11-19.html on September 3, 2015. 
124 Kaiser Family Foundation, Americans Views on the ACA Hold Steady, with 43% Now Viewing it Favorably and 42% Unfavorably, April 2016.  Accessed 
at http://kff.org/health-costs/poll-finding/kaiser-health-tracking-poll-january-2016/ on February 11, 2016. 
125 Kaiser Family Foundation, Kaiser Health Tracking Poll:  January 2016, January 2016.  Accessed at http://kff.org/health-costs/poll-finding/kaiser-health-
tracking-poll-january-2016/ on February 11, 2016. 
126 The Washington Post, Obama Administration to End Contract with CGI Federal, Company Behind HealthCare.gov, January 10, 2014.  Accessed at 
http://www.washingtonpost.com/politics/obama-administration-to-end-contract-with-cgi-federal-company-behind-
healthcaregov/2014/01/10/001eb05a-719e-11e3-8b3f-b1666705ca3b_story.html on September 9, 2015. 
127 U.S. House of Representatives, Committee on Oversight and Government Reform, Issa: CMS Head Tavenner to Testify on HealthCare.gov Woes, 
September 4, 2014.  Accessed at https://oversight.house.gov/release/issa-cms-head-tavenner-testify-healthcare-gov-woes/ on September 25, 2015.  
HHS, Statement by Marilyn Tavenner, Administrator, Centers for Medicare & Medicaid Services, U.S. Department of Health and Human Services (HHS) on 
Affordable Care Act Implementation before Committee on Oversight and Government Reform United States House of Representatives, September 18, 
2014.  Accessed at http://www.hhs.gov/asl/testify/2014/09/t20140918.html on September 25, 2015.  The Wall Street Journal, Hacker Breached 
HealthCare.gov Insurance Site, September 4, 2014.  Accessed at http://www.wsj.com/articles/hacker-breached-healthcare-gov-insurance-site-
1409861043 on September 25, 2015. 
128 CMS, CMS Blog: Protecting Consumer Privacy on HealthCare.gov, January 24, 2015.  Accessed at 
https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-releases/2015-Press-releases-items/2015-01-24.html on September 25, 2015. 
129 CMS, What Consumers Need to Know about Corrected Form 1095, February 20, 2015.  Accessed at http://blog.cms.gov/2015/02/20/what-consumers-
need-to-know-about-corrected-form-1095-as/ on September 25, 2015. 
130 U.S. House of Representatives, Committee on Oversight and Government Reform, Issa: Obama Administration Must Explain Padded ObamaCare 
Enrollment Figures, November 20, 2014.  Accessed at https://oversight.house.gov/release/issa-obama-administration-must-explain-padded-obamacare-
enrollment-figures/ on September 25, 2015. 
131 OIG, CMSs Internal Controls Did Not Effectively Ensure the Accuracy of Aggregate Financial Assistance Payments Made to Qualified Health Plan Issuers 
Under the Affordable Care Act, A-02-14-02006, June 2015. 
132 OIG, CMS Could Not Effectively Ensure that Advance Premium Tax Credit Payments Made Under the Affordable Care Act Were Only for Enrollees Who 
Paid their Premiums, A-02-14-02025, December 2015. 
133 CMS, Policy-Based Payments:  Approach to 2016 Marketplace Payment Program Integrity, Withholding for Issuers Delayed on Policy-Based Payments 
Implementation, and Payment Adjustment for Issuers Deemed Policy-Based Payments Ready , December 4, 2015.  Accessed at 
https://www.cms.gov/CCIIO/Resources/Regulations-and-Guidance/Downloads/Policy-based-Payment-Guidance.pdf on December 11, 2015.  
Correspondence with CMS Office of Legislation, February 9, 2016. 
134 Kaiser Family Foundation, Marketplace Enrollment as a Share of the Potential Marketplace Population.  Accessed at http://kff.org/health-reform/state-
indicator/marketplace-enrollment-as-a-share-of-the-potential-marketplace-population-2015/ on August 25, 2015. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           83  ENDNOTES 
 
135 OIG, Not All of the Federally Facilitated Marketplaces Internal Controls Were Effective in Ensuring That Individuals Were Properly Determined Eligible 
for Qualified Health Plans and Insurance Affordability Programs, A-09-14-01011, August 2015. 
136 CMS, CMS Issues the Final HHS Notice of Benefit and Payment Parameters for 2016, February 2, 2015.  Accessed at 
https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-releases/2015-Press-releases-items/2015-02-20-4.html on September 4, 2015. 
137 45 CFR  155.410(e).  CMS, Final HHS Notice of Benefit and Payment Parameters for 2016, February 20, 2015.  Accessed at 
https://www.cms.gov/Newsroom/MediaReleaseDatabase/Fact-sheets/2015-Fact-sheets-items/2015-02-20.html on September 9, 2015.  CMS, Important 
Marketplace deadlines.  Accessed at https://www.HealthCare.gov/marketplace-deadlines/2016/ on September 9, 2015. 
138 CMS, Health Insurance Marketplace Open Enrollment Snapshot  Week 13, February 4, 2016.  Accessed at https://www.cms.gov/Newsroom/Media
ReleaseDatabase/Fact-sheets/2016-Fact-sheets-items/2016-02-04.html on February 8, 2016. 
 
OEI-06-14-00350            HealthCare.gov:  Case Study of CMS Management of the Federal Marketplace           84