Healthcare.gov: Ineffective Planning and Oversight Practices Underscore the Need for Improved Contract Management
Published: 2014-07-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In March 2010, the Patient Protection and Affordable Care Act required the establishment of health insurance marketplaces by January 1, 2014. Marketplaces permit individuals to compare and select insurance plans offered by private insurers. For states that elected not to establish a marketplace, CMS was responsible for developing a federal marketplace. In September 2011, CMS contracted for the development of the FFM, which is accessed through Healthcare.gov.
When initial enrollment began on October 1, 2013, many users encountered challenges accessing and using the website. GAO was asked to examine various issues surrounding the launch of the Healthcare.gov website. Several GAO reviews are ongoing.
This report assesses, for selected contracts, (1) CMS acquisition planning activities; (2) CMS oversight of cost, schedule, and system capability changes; and (3) CMS actions to address contractor performance. GAO selected two task orders and one contract that accounted for 40 percent of CMS spending and were central to the website. For each, GAO reviewed contract documents and interviewed CMS program and contract officials as well as contractors.
section_title: What GAO Found
paragraphs: The Centers for Medicare & Medicaid Services (CMS) undertook the development of Healthcare.gov and its related systems without effective planning or oversight practices, despite facing a number of challenges that increased both the level of risk and the need for effective oversight. CMS officials explained that the task of developing a first-of-its-kind federal marketplace was a complex effort with compressed time frames. To be expedient, CMS issued task orders to develop the federally facilitated marketplace (FFM) and federal data services hub (data hub) systems when key technical requirements were unknown, including the number and composition of states to be supported and, importantly, the number of potential enrollees. CMS used cost-reimbursement contracts, which created additional risk because CMS is required to pay the contractor's allowable costs regardless of whether the system is completed. CMS program staff also adopted an incremental information technology development approach that was new to CMS. Further, CMS did not develop a required acquisition strategy to identify risks and document mitigation strategies and did not use available information, such as quality assurance plans, to monitor performance and inform oversight.
CMS incurred significant cost increases, schedule slips, and delayed system functionality for the FFM and data hub systems due primarily to changing requirements that were exacerbated by oversight gaps. From September 2011 to February 2014, FFM obligations increased from $56 million to more than $209 million. Similarly, data hub obligations increased from $30 million to nearly $85 million. Because of unclear guidance and inconsistent oversight, there was confusion about who had the authority to approve contractor requests to expend funds for additional work. New requirements and changing CMS decisions also led to delays and wasted contractor efforts. Moreover, CMS delayed key governance reviews, moving an assessment of FFM readiness from March to September 2013—just weeks before the launch—and did not receive required approvals. As a result, CMS launched Healthcare.gov without verification that it met performance requirements.
Late in the development process, CMS identified major performance issues with the FFM contractor but took only limited steps to hold the contractor accountable. In April and November 2013, CMS provided written concerns to the contractor about product quality and responsiveness to CMS direction. In September 2013, CMS program officials became so concerned about the contractor's performance that they moved operations to the FFM contractor's offices to provide on-site direction. At the time, CMS chose to forego actions, such as withholding the payment of fee, in order to focus on meeting the website launch date. Ultimately, CMS declined to pay about $267,000 in requested fee. This represents about 2 percent of the $12.5 million in fees paid to the FFM contractor. CMS awarded a new contract to another firm for $91 million in January 2014 to continue FFM development. As of June 2014, costs on the contract had increased to over $175 million due to changes such as new requirements and other enhancements, while key FFM capabilities remained unavailable. CMS needs a mitigation plan to address these issues. Unless CMS improves contract management and adheres to a structured governance process, significant risks remain that upcoming open enrollment periods could encounter challenges.
section_title: What GAO Recommends
paragraphs: GAO recommends that CMS take immediate actions to assess increasing contract costs and ensure that acquisition strategies are completed and oversight tools are used as required, among other actions. CMS concurred with four recommendations and partially concurred with one.

section_title: Letter
paragraphs: The Patient Protection and Affordable Care Act (PPACA), enacted in  March 2010, made fundamental changes to the availability and  affordability of health insurance coverage. A central provision of the law  required the establishment of state health insurance exchanges, now  commonly referred to as marketplaces, by January 1, 2014. Marketplaces  permit individuals to compare and select private health insurance plans.  For states that elected not to establish a marketplace, PPACA required  the federal government to establish and operate a federal marketplace.
The Centers for Medicare & Medicaid Services (CMS) within the  Department of Health and Human Services (HHS) was responsible for  designing, developing, and implementing the information technology (IT)  systems needed to support the federal marketplace which users access  via the Healthcare.gov website. CMS largely relied on contractors to  develop, build, and operate the necessary information technology  systems. When initial enrollment began on October 1, 2013, many users  were unable to successfully access and use the Healthcare.gov website  to obtain health insurance information due to problems such as website  failures, errors, and slow response times.
Given the high degree of congressional interest in examining the  development, launch, and other issues associated with accessing the  federal marketplace through the Healthcare.gov website, GAO is  conducting a body of work in order to assist Congress with its oversight  responsibilities. This report examines selected contracts and task orders  central to the development and launch of the Healthcare.gov website by  assessing (1) CMS acquisition planning activities; (2) CMS oversight of  cost, schedule, and system capability changes; and (3) actions taken by  CMS to identify and address contractor performance issues.
To address these objectives, we reviewed the Federal Procurement Data  System-Next Generation, which is the government’s procurement  database, to identify CMS contracts and task orders related to the IT  systems supporting the Healthcare.gov website and amounts obligated  from fiscal year 2010 through March 2014. We performed data reliability  assessments and confirmed that the data were sufficiently reliable for our  purposes. Based on this information as well as interviews with CMS  contracting and program officials, we selected one contract and two task  orders issued under an existing 2007 contract for our review. The  contract and task orders combined accounted for more than 40 percent of  the total CMS reported obligations related to the development of  Healthcare.gov and its supporting systems as of March 2014. Specifically,  we selected the task orders issued to CGI Federal Inc. (CGI Federal) for  the development of the federally facilitated marketplace (FFM) system  and to QSSI, Inc. (QSSI) for the development of the federal data services  hub (data hub) in September 2011—and the contract awarded to  Accenture Federal Services in January 2014 to complete FFM  development and enhance existing functionality.
To assess CMS acquisition planning activities, we reviewed the Federal  Acquisition Regulation (FAR) and relevant HHS/CMS policies and  guidance and evaluated contract file documents. To assess CMS  oversight of cost, schedule, and system capability changes, we reviewed  contract modifications, contract deliverables, contractor monthly status  reports, and other documents. To assess actions taken by CMS to  identify and address contractor performance issues, we identified  monitoring requirements and analyzed contract file documentation. To  support work on all three objectives, we interviewed contracting officials in  CMS’s Office of Acquisition and Grants Management and program  officials in CMS’s Office of Information Services. In addition, we  interviewed the contractors to obtain their perspective on CMS’s oversight  of cost, schedule, and system capabilities. Appendix I provides additional  details about our scope and methodology.
We conducted this performance audit from January 2014 to July 2014, in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Each marketplace created under PPACA is intended to provide a  seamless, single point of access for individuals to enroll in qualified health  plans, apply for income-based financial subsidies established under the  law and, as applicable, obtain an eligibility determination for other health  coverage programs, such as Medicaid or the State Children’s Health  Insurance Program (CHIP). To obtain health insurance offered through  the marketplace, individuals must complete an application and meet  certain eligibility requirements defined by PPACA, such as being a U.S.  citizen or legal immigrant. For those consumers determined eligible, the  marketplaces permit users to compare health plans and enroll in the plan  of their choice. States had various options for marketplace participation,  including (1) establishing their own state-based marketplace, (2) deferring  to CMS to operate the federal marketplace in the state, or (3) participating  in an arrangement called a partnership marketplace in which the state  assists with some federal marketplace operations.
In our June 2013 report on CMS efforts to establish the federal  marketplace, we concluded that certain factors—such as the evolving  scope of marketplace activities required in each state—suggested the  potential for implementation challenges going forward.draft of that report, HHS emphasized the progress it had made since  PPACA became law and expressed its confidence that marketplaces  would be open and functioning in every state on October 1, 2013.
subsections: section_title: Timeline of Key Events
paragraphs: PPACA required the establishment of marketplaces in each state by  January 2014. Based on the expectation that individuals and families  would need time to explore their coverage options and plan issuers would  need time to process plan selections, HHS established October 1, 2013,  as the beginning of the enrollment period for all marketplaces, including  the federal marketplace.legal or regulatory, and organizational events during that development  period, as well as future milestones through the beginning of open  enrollment for 2015.
section_title: Healthcare.gov and Supporting Systems
paragraphs: The Healthcare.gov website is supported by several systems, including  the FFM and the federal data services hub. Additional components  include the Enterprise Identity Management System that confirms the  consumer’s identity when entering the system.
subsections: section_title: Healthcare.gov Website
paragraphs: Healthcare.gov is the Internet address of a federal government-operated  website that serves as the online user interface for the federal  marketplace. The website allows the consumer to create an account,  input required information, view health care plan options and make a plan  selection.
section_title: FFM System
paragraphs: The FFM accepts and processes data entered through the website and  was intended to provide three main functions:
Eligibility and enrollment. This module guides applicants through a  step-by-step process to determine their eligibility for coverage and  financial assistance, after which they are shown applicable coverage  options and have the opportunity to enroll.
Plan management. This module interacts primarily with state  agencies and health plan issuers. The module is intended to provide a  suite of services for activities such as submitting, monitoring, and  renewing qualified health plans.
Financial management. This module facilitates payments to issuers,  including premiums and cost-sharing reductions, and collects data  from state-based marketplaces.
Other FFM functions include services related to system oversight,  communication and outreach strategies, and customer service.
section_title: Federal Data Services Hub
paragraphs: The data hub routes and verifies information among the FFM and external  data sources, including other federal and state sources of information and  issuers.Security number with the Social Security Administration and connects to  the Department of Homeland Security to assess the applicant’s  citizenship or immigration status.
For example, the data hub confirms an applicant’s Social  The data hub’s connection with other federal and state databases  enables exchanges to determine whether an applicant is eligible for or  enrolled in some other type of health coverage, such as the Department  of Defense’s (DOD) TRICARE program or Medicaid—and therefore  ineligible for subsidies to offset the cost of marketplace plans. hub also communicates with issuers by providing enrollment information  and receiving enrollment confirmation in return. See figure 2 for an  overview of Healthcare.gov and selected supporting systems.
These subsidies include premium tax credits to offset qualified health plan premium  costs and cost-sharing reductions to reduce policyholders’ out-of-pocket payments,  including deductibles and co-payments, for covered services.
section_title: Federal Implementation Costs
paragraphs: While CMS was tasked with oversight of marketplace establishment,  several other federal agencies also have implementation responsibilities.  Three agencies—CMS, the Internal Revenue Service (IRS), and the  Department of Veterans Affairs (VA)—reported almost all of the IT-related  obligations supporting the implementation of the Healthcare.gov and its  supporting systems. IT-related obligations include funds committed for  the development or purchase of hardware, software, and system  integration services, among other activities. These obligations totaled  approximately $946 million from fiscal year 2010 through March 2014,  with CMS obligating the majority of this total.
section_title: CMS Contracts and Task Orders for Healthcare.gov and Its Supporting Systems
paragraphs: As of March 2014, CMS reported obligating $840 million for the  development of Healthcare.gov and its supporting systems, over 88  percent of the federal total. According to agency data, these obligations  were spread across 62 contracts and task orders. We focused our review  on two CMS task orders issued under an existing 2007 contract. The task  orders were for the development of two core Healthcare.gov systems— the FFM and the data hub. We also reviewed a letter contract awarded by  CMS in January 2014 to continue FFM development. The two task orders  and the additional contract account for $369 million, or more than 40  percent, of the total CMS reported obligations as of March 2014.
section_title: Acquisition Process
paragraphs: The contract and task orders we examined are subject to the Federal  Acquisition Regulation System, which provides uniform policies and  procedures for acquisition by all executive agencies. The system includes  the HHS acquisition regulation, which implements or supplements the  FAR. HHS’s supplement to the FAR, which contain additional HHS  policies and procedures, is referred to as the Department of Health and  Human Services Acquisition Regulation (HHSAR). The FAR and HHSAR  address issues pertaining to the contracting process and include activities  related to three phases: pre-award, competition and award, and post- award. See figure 3 for an overview of these phases and selected  activities related to each.
To implement and oversee PPACA’s marketplace and private health  insurance requirements, HHS established the Office of Consumer  Information and Insurance Oversight (OCIIO) in April 2010 as part of the  HHS Office of the Secretary. In January 2011, the OCIIO moved to CMS  and became the Center for Consumer Information and Insurance  Oversight (CCIIO). Within CMS, establishment of the federal marketplace  was managed by CCIIO, with responsibilities shared with the Office of  Information Services (OIS), and the Office of Acquisition and Grants  Management (OAGM). HHS’s acquisition process for the data hub and  FFM task orders involved multiple participants, including:
The contracting officer. The contracting officer has the authority to  enter into, administer, and/or terminate contracts and make related  determinations. The contracting officer is responsible for ensuring  performance of all necessary actions for effective contracting,  ensuring compliance with the terms of the contract, and safeguarding  the interests of the United States in its contractual relationships.
The contracting officer’s representative (COR). The COR—also  referred to as the contracting officer’s technical representative—is  designated in writing by the contracting officer to perform specific  technical or administrative functions. Unlike the contracting officer, a  COR has no authority to make any commitments or changes that  affect price, quality, quantity, delivery, or other terms and conditions of  the contract and cannot direct the contractor or its subcontractors to  operate in conflict with the contract terms and conditions.
The government task leader (GTL). The GTL is a representative of  the program office who assists the COR and is responsible for day-to- day technical interaction with the contractor. The GTL is also  responsible for monitoring technical progress, including the  surveillance and assessment of performance, and performing  technical evaluations as required, among other responsibilities.
section_title: Oversight Weaknesses and Lack of Adherence to Planning Requirements Compounded Acquisition Planning Challenges
paragraphs: CMS undertook the development of Healthcare.gov and its related  systems without effective planning or oversight practices, despite facing a  number of challenges that increased both the level of risk and the need  for oversight. According to CMS program and contracting officials, the  task of developing a first-of-its-kind federal marketplace was a complex  effort that was exacerbated by compressed time frames and changing  requirements. CMS contracting officials explained that meeting project  deadlines was a driving factor in a number of acquisition planning  activities, such as the selection of a cost-reimbursement contract, the  decision to proceed with the contract award process before requirements  were stable, and the use of a new IT development approach. These  actions increased contract risks, including the potential for cost increases  and schedule delays, and required enhanced oversight. However, CMS  did not use information available to provide oversight, such as quality  assurance surveillance plans. CMS also missed opportunities to consider  the full range of risks to the acquisition by not developing a written  acquisition strategy, even though the agency was required to do so. As a  result, key systems began development with risks that were not fully  identified and assessed.
subsections: section_title: Acquisition Planning Activities Carried High Levels of Risk for the Government
paragraphs: Meeting project deadlines was a driving factor in a number of acquisition  planning activities. HHS had 15 months between enactment of PPACA  and the agency’s request for proposal to develop requirements for the  FFM and data hub. In a prior report on acquisition planning at several  agencies, including HHS, we found that the time needed to complete  some pre-solicitation planning activities—such as establishing the need  for a contract, developing key acquisition documents such as the  requirements document, the cost estimate, and, if required, the  acquisition plan; and obtaining the necessary review and approvals— could be more than 2 years. The time needed depended on factors that  were present for this acquisition including complexity of the requirements,  political sensitivity, and funding.developing requirements for a complex, first-of-its-kind system in these  compressed time frames and indicated that more time was needed.
CMS program officials noted challenges  The FFM and data hub task orders were issued under an existing 2007  contract for enterprise system development. This approach was  reasonable in these circumstances because, according to contracting  officials, the task orders could be issued more quickly than using a full  and open competitive approach. The 2007 contract had been awarded to  16 vendors who were then eligible to compete for individual task orders.  The 2007 contract was specifically established to improve efficiency when  new IT requirement arose—such as the federal marketplace  development. The 16 eligible contractors had experience with CMS’s IT  architecture and could come up to speed quickly. The solicitation for the  2007 contract sought contractors with experience in software design,  development, testing and maintenance in complex systems environments  to provide a broad range of IT services including planning, design,  development, and technical support, among others. Of the 16 eligible  contractors, four contractors responded with proposals for each system.
CMS used a source selection process that considered both cost and non- cost factors. This type of source selection process is appropriate when it  may be in the best interest of the agency to consider award to other than  the lowest priced offer or the highest technically rated offer.the request for proposals indicated that cost and non-cost factors were  weighted equally. The non-cost factors for technical evaluation included  logical and physical design, project plan, and staffing plan, among others.  In addition, CMS considered contractor past performance, but did not  include that factor in the technical evaluation. CMS determined that the  selected contractors for both task orders offered the most advantageous  combination of technical performance and cost.
subsections: section_title: Requirements for Developing the FFM System Were Not Well Defined When the Task Order Was Issued
paragraphs: The FAR requires that agencies ensure that requirements for services are  clearly defined. In addition, in our August 2011 review of opportunities to  build strong foundations for better services contracts, we found that well- defined requirements are critical to ensuring the government gets what it  needs from service contractors. We also found that program and  contracting officials at the four agencies we reviewed—which included  HHS—noted that defining requirements can be a challenging part of  acquisition planning and is a shared responsibility between program and  contracting officials. Further, our March 2004 report on software- intensive defense acquisitions found that while requirements for a project  can change at any point, officials must aggressively manage  requirements changes to avoid a negative effect on project results, such  as cost increases and schedule delays.
In order to begin work quickly, CMS proceeded with the award process  before FFM contract requirements, which included general technical  requirements for system development, were finalized. For example, at the  time the task order was issued, CMS did not yet know how many states  would opt to develop their own marketplaces and how many would  participate in the federally facilitated marketplace, or the size of their  uninsured populations. CMS also had not completed rulemaking  necessary to establish key marketplace requirements. The statement of  work for the FFM acknowledged a number of these unknown  requirements, for example, stating that requirements for state support  were not fully known and the FFM system “must be sufficiently robust to  provide support of state exchange requirements at any point in the life  cycle.” In addition, the FFM statement of work noted that the  requirements related to a number of FFM services would be finalized after  contract award, including services related to all three main functional  areas—eligibility and enrollment, financial management, and plan  management—as well as system oversight, communication, and  customer service.
The technical requirements for both the FFM and data hub were  developed by CMS staff with contractor support and documented in a  statement of work for each task order. Both statements called for the  contractor to design a “solution that is flexible, adaptable, and modular to  accommodate the implementation of additional functional requirements  and services.” However, according to CMS program officials,  requirements for data hub development were more clearly defined at the  time that task order was issued than FFM requirements. These officials  also stated that, prior to issuing the task order, CMS was able to develop  a prototype for the data hub and a very clear technical framework to guide  the contractor, but due to still-changing requirements, CMS could not  provide the same guidance for FFM development. We have previously  found that unstable requirements can contribute to negative contract  outcomes, including cost overruns and schedule delays.
section_title: CMS Used a Contract Type That Carried Risk for the Government and Required Additional Oversight
paragraphs: FAR §16.301-2(a)(1) & (2). steps to minimize the use of cost reimbursement contracts.CMS’s use of the cost-plus-fixed-fee contract type may have been a  reasonable choice under the circumstances, the related risks increased  the need for oversight.
In our November 2007 report on internal control deficiencies at CMS, we  found that certain contracting practices, such as the frequent use of cost  reimbursement contracts, increased cost risks to CMS because CMS did  not implement sufficient oversight for cost reimbursement contracts at  that time. acknowledged the increased responsibilities and risks associated with  managing a cost reimbursement contract and included a number of  oversight elements in the task orders to support contract oversight and  manage risks. These elements included contract deliverables such as  earned value management reports, monthly financial and project status  reports, and a quality assurance surveillance plan.
See GAO, Centers for Medicare and Medicaid Services: Internal Control Deficiencies  Resulted in Millions of Dollars of Questionable Contract Payments, GAO-08-54  (Washington, D.C.: Nov. 15, 2007). We made nine recommendations to the Administrator  of CMS to improve internal control and accountability in the contracting process and  related payments to contractors. All nine recommendations have been implemented. the supplies or services conform to contract requirements.found that the quality assurance surveillance plans were not used to  inform oversight. For example, contracting and program officials,  including the COR and contracting officer, were not sure if the quality  assurance surveillance plan had been provided as required by the FFM  and data hub task orders. Although a copy was found by CMS staff in  June 2014, officials said they were not aware that the document had been  used to review the quality of the contractor’s work. Instead, CMS program  officials said they relied on their personal judgment and experience to  determine quality.
section_title: CMS Selected a New IT Development Approach to Save Time, but Increased Risks
paragraphs: In 2012, GAO reported on the use of Agile methods in the Federal government. See  GAO, Software Development: Effective Practices and Federal Challenges in Applying  Agile Methods, GAO-12-681 (Washington, D.C.: July 27, 2012). In this report we made  one recommendation to the Federal CIO Council to encourage the sharing of these  practices. and that deviating from traditional procedural guidance to follow Agile  methods was a challenge. We also reported that new tools and training  may be required, as well as updates to procurement strategies.  Therefore, the new approach that CMS selected in order to speed work  also carried its own implementation risks.
section_title: CMS Did Not Fully Adhere to HHS Acquisition Planning Requirements and Missed Opportunities to Capture and Consider Risks Important to the Program’s Success
paragraphs: While a number of CMS’s acquisition planning actions were taken in an  effort to manage acquisition challenges, CMS missed opportunities to  fully identify and mitigate the risks facing the program. HHS acquisition  policy requires the development of a written acquisition strategy for major  IT investments, such as the FFM system. According to HHS policy, an  acquisition strategy documents the factors, approach, and assumptions  that guide the acquisition with the goal of identifying and mitigating risks.HHS provides a specific acquisition strategy template that requires  detailed discussion and documentation of multiple strategy elements,  including market factors and organizational factors, among others.
According to program officials, the acquisition planning process for the  FFM and data hub task orders began in 2010, prior to HHS’s decision to  move its Office of Consumer Information and Insurance Oversight  (OCIIO) to CMS, and continued into early 2011. Program officials stated  that the planning process included discussions of an acquisition strategy.  However, CMS program and contracting staff did not complete the  required acquisition strategy for FFM and data hub development.  According to contracting and program officials, CMS has not been  preparing acquisition strategies for any of its major IT acquisitions, not  just those related to systems supporting Healthcare.gov. This is a  longstanding issue. In November 2009 we found deficiencies in CMS  contract management internal controls practices such as the failure to  follow existing policies and the failure to maintain adequate  documentation in contract files.CMS is planning steps to strengthen the agency’s program and project  management, including training related to the acquisition strategy  requirement.
According to CMS contracting officials,  Contracting officials from OAGM explained that at CMS the majority of  acquisition planning is done by the program office and OAGM began  discussions of the upcoming task orders related to Healthcare.gov and its  supporting systems with program officials in February 2011. In June  2011, OAGM accepted a Request for Contract package—a set of  documents used to request and approve a contract action—from the  program office. The package documents some elements of an acquisition  strategy. Specifically, it indicated the type of contract to be used and the  selected contract approach; however, the documents do not include the  rationale for all decisions and did not address a number of planning  elements required in HHS acquisition strategy, such as organizational  factors, technological factors, and logistics.
In the absence of an acquisition strategy, key risks and plans to manage  them were not captured and considered as required. The acquisition  strategy provides an opportunity to highlight potential risk areas and  identify ways to mitigate those risks. For example, the strategy guidance  requires the consideration of organizational factors that include  management and their capabilities, available staff and their skills, and  risks associated with the organizational structure. Organizational factors  were a potential risk area for these projects because the CMS  organizations responsible for the FFM and data hub experienced  significant changes just prior to and during the planning period.  Specifically, OCIIO was established in 2010 and integrated into CMS in  January 2011, just prior to the beginning of planning discussions with  OAGM. According to CMS contracting and program officials, some of the  246 OCIIO staff transitioned to the new CCIIO and others joined CMS’s  Office of Information Services (OIS) and OAGM. In the context of these  organizational changes and the other considerable project risks, the  acquisition strategy could have been a powerful tool for risk identification  and mitigation. By failing to adhere to this requirement, CMS missed  opportunities to explain the rationales for acquisition planning activities  and to fully capture and consider risks important to the success of the  program.
section_title: Changing Requirements and Oversight Gaps Contributed to Significant Cost Growth, Schedule Delays, and Reduced Capabilities during FFM and Data Hub Development
paragraphs: CMS incurred significant cost increases, schedule slips, and reduced  system functionality in the development of the FFM and data hub  systems—primarily attributable to new and changing requirements  exacerbated by inconsistent contract oversight. From September 2011 to  February 2014, estimated costs for developing the FFM increased from  an initial obligation of $56 million to more than $209 million; similarly, data  hub costs increased from an obligation of $30 million to almost $85  million. New and changing requirements drove cost increases during the  first year of development, while the complexity of the system and rework  resulting from changing CMS decisions added to FFM costs in the second  year. In addition, required design and readiness governance reviews  were either delayed or held without complete information and CMS did  not receive required approvals. Furthermore, inconsistent contractor  oversight within the program office and unclear roles and responsibilities  led CMS program staff to inappropriately authorize contractors to expend  funds.
subsections: section_title: FFM and Data Hub Task Orders Experienced Significant Increases
paragraphs: Obligations for both the FFM and data hub rose significantly during the  two-and-a-half-year development period, with the FFM task order  increasing almost four-fold, from $55.7 million obligated when issued in  late 2011 to more than $209 million obligated by February 2014. Similarly,  the data hub task order almost tripled, increasing from $29.9 million to  $84.5 million during the same period. Figure 4 shows FFM and data hub  obligation growth during this time.
Development cost increases for the FFM and data hub were due to a  combination of factors, including costs associated with adding or  changing requirements. For example, CMS was aware that a number of  key business requirements for the FFM and data hub would not be known  until after the task orders were issued in September 2011, and it  acknowledged some of these uncertainties in the statements of work,  such as noting that the actual number of states participating in the federal  marketplace and the level of support each state required was not  expected to be known until January 2013. We previously found in March  2004 that programs with complex software development experienced cost  increases and schedule delays when they lacked controls over their  requirements, noting that leading software companies found changing  requirements tend to be a major cause of poor software development  outcomes.
Subsequent modifications to the FFM and data hub task orders show the  costs associated with adding requirements beyond those initial  uncertainties. For example, CMS obligated an additional $36 million to the  FFM and $23 million to the data hub in 2012, in large part to address  requirements that were added during the first year of development, such  as increasing infrastructure to support testing and production and adding  a transactional database. Some of these new requirements resulted from  regulations and policies that were established during this period. For  example, in March 2012, federal rulemaking was finalized for key  marketplace functions, resulting in the need to add services to support the  certification of qualified health plans for partnership marketplace states.  Other requirements emerged from stakeholder input, such as a new  requirement to design and implement a separate server to process  insurance issuers’ claims and enrollment data outside of the FFM. CMS  program officials said that this resulted from health plan issuers’ concerns  about storing proprietary data in the FFM. The FFM and data hub task  orders were both updated to include this requirement in 2012, which was  initially expected to cost at least $2.5 million.
subsections: section_title: System Complexities and Rework Further Added to FFM Costs in the Second Year
paragraphs: During the second year of development, from September 2012 to  September 2013, the number of task order modifications and dollars  obligated for the development of the FFM and data hub continued to  increase. New requirements still accounted for a portion of the costs, but  the second-year increases also reflected the previously unknown  complexities of the original requirements and associated rework,  particularly for the FFM. For example, according to the FFM contractor,  one of the largest unanticipated costs came from CMS’ directions to  purchase approximately $60 million in software and hardware that was  originally expected to be provided by another Healthcare.gov contractor.  Most of these costs were added through task order modifications in 2013.
In April 2013, CMS added almost $28 million to the FFM task order to  cover work that that was needed because of the increasingly complex  requirements, such as additional requirements to verify income for  eligibility determination purposes. The FFM contractor said some of these  costs resulted from CMS’s decisions to start product development before  regulations and requirements were finalized, and then to change the FFM  design as the project was ongoing, which delayed and disrupted the  contractor’s work and required them to perform rework. In addition, CMS  decisions that appeared to be final were reopened, requiring work that  had been completed by the contractor to be modified to account for the  new direction. This included changes to various templates used in the  plan management module and the application used by insurance issuers,  as well as on-going changes to the user interface in the eligibility and  enrollment module. According to the FFM contractor, CMS changed the  design of the user interface to match another part of the system after  months of work had been completed, resulting in additional costs and  delays. In November 2012, the contractor estimated that the additional  work in the plan management module alone could cost at least $4.9  million.
By contrast, CMS program officials explained that the data hub generally  had more stable requirements than the FFM, in part due to its functions  being less technically challenging and because CMS had had more time  to develop the requirements. While the obligations for the data hub also  increased at the same rate as the FFM in the first year of development,  they did so to a lesser degree during the second year. According to the  data hub contractor, these increases were due to CMS-requested  changes in how the work was performed, which required additional  services, as well as hardware and software purchases.
section_title: CMS Experienced Schedule Delays, Conducted Incomplete Governance Oversight Reviews, and Delayed Some Capabilities for the FFM and Data Hub
paragraphs: In addition to increased costs, the FFM and data hub experienced  schedule delays, which contributed to CMS holding incomplete  governance oversight reviews and eventually reduced the capabilities it  expected the FFM contractor to produce by the October 1, 2013,  deadline.
subsections: section_title: CMS Delayed Scheduled Governance Reviews, Reducing Time Available for FFM and Data Hub Testing and Implementation Reviews
paragraphs: CMS initially established a tight schedule for reviewing the FFM and data  hub development in order to meet the October 1, 2013, deadline for  establishing enrollment through the website. Each task order lists the key  governance reviews that the systems were required to meet as they  progressed through development.
The FFM and data hub task orders initially required the contractors to be  prepared to participate in most of the CMS governance reviews— including a project baseline and final detailed design reviews—within the  first 9 months of the awards. This would allow CMS to hold the final  review needed to implement the systems—operational readiness—at  least 6 months before the Healthcare.gov launch planned for October 1,  2013. In April 2013, CMS extended the requirements analysis and design  phase. According the CMS program officials, requirements were still  changing and more time was needed to finalize the FFM design. As a  result, CMS compressed time frames for conducting reviews for the  testing and implementation phases. Under the revised schedule, the  contractor had until the end of September 2013—immediately prior to the  date of the planned launch—to complete the operational readiness  review, leaving little time for any unexpected problems to be addressed  despite the significant challenges the project faced. Figure 5 shows the  schedule of planned and revised development milestone reviews in the  FFM and data hub task orders.
The four reviews shown in figure 5—architecture, project baseline, final  detailed design, and operational readiness— are among those required  under the exchange life cycle framework, the governance model CMS  specifically designed to meet the need to quickly develop the FFM and  data hub using the Agile development approach. The life cycle  framework requires technical reviews at key junctures in the development  process, such as a final detailed design review to ensure that the design  meets requirements before it is developed and tested. To accommodate  different development approaches, the life cycle framework allows  program offices leeway regarding how some reviews are scheduled and  conducted, permitting more informal technical consultations when holding  a formal review would cause delays. However, the framework requires  that the four governance or milestone reviews be approved by a CMS  governance board.
section_title: Some Governance Reviews Were Not Fully Conducted or Approved
paragraphs: Despite the revised FFM schedule, it is not clear that CMS held all of the  governance reviews for the FFM and data hub or received the approvals  required by the life cycle framework. The framework was developed to  accommodate multiple development approaches, including Agile. A  senior CMS program official said that although the framework was used  as a foundation for their work, it was not always followed throughout the  development process because it did not align with the modified Agile  approach CMS had adopted. CMS program officials explained that they  held multiple reviews within individual development sprints—the short  increments in which requirements are developed and software is  designed, developed, and tested to produce a building block for the final  system. However, CMS program officials indicated that they were focused  on responding to continually changing requirements which led to them  participating in some governance reviews without key information being  available or steps completed. Significantly, CMS held a partial operational  readiness review for the FFM in September 2013, but development and  testing were not fully completed and continued past this date. As a result,  CMS launched the FFM system without the required verification that it  met performance requirements.
Furthermore, the life cycle framework states that CMS must obtain  governance-board approval before the systems proceed to the next  phase of development, but we did not see evidence that any approvals  were provided. CMS records show that CMS held some governance  reviews, such as design readiness reviews. However, the governance  board’s findings identified outstanding issues that needed to be  addressed in subsequent reviews and they were not approved to move to  the next stage of development.
section_title: CMS Postponed Some FFM Capabilities to Meet Deadlines
paragraphs: By March 2013, CMS recognized the need to extend the task orders’  periods of performance in order to allow more time for development. CMS  contract documents from that time estimated that only 65 percent of the  FFM and 75 percent of the data hub would be ready by September 2013,  when development was scheduled to be completed. Recognizing that  neither the FFM nor the data hub would function as originally intended by  the beginning of the initial enrollment period, CMS made trade-offs in an  attempt to provide necessary system functions by the October 1, 2013,  deadline. Specifically, CMS prioritized the elements of the system needed  for the launch, such as the FFM eligibility and enrollment module, and  postponed the financial module, which would not be needed until post- enrollment. CMS also delayed elements such as the Small Business  Health Options Program marketplace, initially until November 2013, and  then until 2015. See figure 6 for the modules’ completion status as of the  end of the task order in February 2014.
In September 2013, CMS extended the amount of time allotted for  development under the FFM and data hub task orders, which accounted  for the largest modifications. The additional obligations—$58 million for  the FFM and $31 million for the data hub—included some new elements,  such as costs associated with increasing FFM capacity needed to support  anticipated internet traffic, but our review of the revised statements of  work show that the additional funding was primarily for the time needed to  complete development work rather than new requirements.
After the FFM was launched on October 1, 2013, CMS took a number of  steps to respond to system performance issues through modifications to  the FFM task order. These efforts included adding more than $708,000 to  the FFM task order to hire industry experts to assess the existing system  and address system performance issues. CMS also greatly expanded the  capacity needed to support internet users, obligating $1.5 million to  increase capacity from 50 terabytes to 400 terabytes for the remainder of  the development period. While CMS program officials said that the  website’s performance improved, only one of the three key components  specified in the FFM task order was completed by the end of the task  order’s development period. (See figure 6.) According to program  officials, the plan management module was complete, but only some of  the elements of the eligibility and enrollment module were provided and  the financial management remained unfinished.
section_title: Unclear Contract Oversight Responsibilities Exacerbated FFM and Data Hub Cost Growth
paragraphs: We identified approximately 40 instances during FFM development in  which CMS program staff inappropriately authorized contractors to  expend funds totaling over $30 million because those staff did not adhere  to established contract oversight roles and responsibilities. Moreover,  CMS contract and program staff inconsistently used and reviewed  contract deliverables on performance to inform oversight.
subsections: section_title: CMS Staff Inappropriately Authorized Contractors to Expend Funds
paragraphs: The FFM task order was modified in April 2013 to add almost $28 million  to cover cost increases that had been inappropriately authorized by CMS  program officials in 2012. This issue also affected the data hub task  order, which had an estimated $2.4 million cost increase over the same  period. In November 2012, the FFM contractor informed CMS of a  potential funding shortfall due to work and hardware that CMS program  officials had directed the contractor to provide. The FAR provides that the  contracting officer is the only person authorized to change the terms and  conditions of the contract. Further, other government personnel shall not  direct the contractor to perform work that should be the subject of a  contract modification. The federal standards for internal control also  state that transactions and significant events need to be authorized and  executed by people acting within the scope of their authority, to ensure  that only valid transactions to commit resources are initiated.
CMS documents show that the cost growth was the result of at least 40  instances in which work was authorized by various CMS program  officials, including the government task leader (GTL)—who is responsible  for day-to-day technical interaction with the contractor—and other staff  with project oversight responsibilities, who did not have the authority to  approve the work. This was done without the knowledge of the  contracting officer or the contracting officer’s representative. This  inappropriately authorized work included adding features to the FFM and  data hub, changing designs in the eligibility and enrollment module, and  approving the purchase of a software license. CMS later determined that  the work was both necessary and within the general scope of the task  order but the cost of the activities went beyond the estimated cost amount  established in the order and thus required a modification.
section_title: Inappropriate Authorizations Due to Unclear Oversight Responsibilities
paragraphs: A senior CMS program official described a three-pronged approach to  contract oversight that involved various CMS offices, including the COR  and GTL in the program offices, and the contracting officer in OAGM. The  COR and GTL were assigned overlapping responsibilities for monitoring  the contractor’s technical performance, but CMS’s guidance to clarify their  roles did not fully address the need to ensure that directions given to  contractors were appropriate. CMS program officials said the guidance  was issued in 2006, several years before the FFM and data hub task  orders were issued. The guidance generally noted that CORs are  responsible for financial and contractual issues while GTLs have day-to- day technical interactions with the contractors. However, the guidance did  not clarify the limitations on COR’s and GTL’s authorities, such as not  providing contractors with technical direction to perform work outside the  scope of the contract.
CMS program officials also described difficulties clarifying oversight  responsibilities in organizations that were new to CMS, which contributed  to the inappropriately authorized work. Program responsibilities were  shared between CCIIO, which was primarily responsible for developing  business requirements, and the information technology staff in OIS,  where the GTL and COR were located. CCIIO was relatively new to CMS,  having been incorporated shortly before the FFM and data hub task  orders were issued. OIS program officials explained that CCIIO was not  as experienced with CMS’s organization and did not strictly follow their  processes, including for oversight. CMS documents show that there were  concerns about inappropriate authorizations prior to the cost growth  identified in late 2012, as officials in the OIS acquisition group had  repeatedly cautioned other OIS and CCIIO staff about inappropriately  directing contractors.
Furthermore, CMS program officials said that CCIIO staff did not always  understand the cost and schedule ramifications associated with the  changes they requested. As the FFM in particular was in the phase of  development in which complexities were emerging and multiple changes  were needed, there were a series of individual directions that, in sum,  exceeded the expected cost of the contract. As a result of the  unauthorized directions to contractors, the CMS contracting officer had to  react to ad hoc decisions made by multiple program staff that affected  contract requirements and costs rather than directing such changes by  executing a contract modification as required by the FAR.
In April 2013, shortly after the inappropriate authorizations and related  cost increases for the FFM and data hub task orders were identified, a  senior contracting official at CMS sent instructions on providing technical  directions to contractors to the program offices that had been involved in  the authorizations and to CMS directors in general. Specifically, the  program offices were reminded to avoid technical direction to  contractors—particularly when there is an immediate need for critical  functions—which might constitute unauthorized commitments by the  government. This instruction has not been incorporated into existing  guidance on the roles and responsibilities of the CORs and GTLs. CMS  contracting and program officials also reported additional steps to bolster  contract oversight such as reminding the FFM contractor not to undertake  actions that result in additional costs outside of the statement of work  without specific direction from the contracting officer.
section_title: CMS Provided Inconsistent Oversight of Contract Performance
paragraphs: It was not always clear which CMS officials were responsible for  reviewing and accepting contractor deliverables, including items such as  the required monthly status and financial reports and the quality  assurance surveillance plan that aid the government in assessing the  costs and quality of the contractor’s work. According to contracting  officials, reviewing such deliverables helped to provide the additional  oversight that cost-reimbursable task orders require per the FAR to  reduce risks of cost growth. However, particularly in the first year of FFM  development, contract documentation shows repeated questions about  who was responsible for reviewing the deliverables and difficulties finding  the documents. Both task orders were ultimately modified to require that  deliverables be provided to the contracting officer, who had previously  just been copied on transmittal letters, in addition to the program office.
In September 2012, the COR oversight function transferred to the  acquisition group within CMS’s OIS and a new COR was assigned to  manage both the FFM and data hub task orders. A CMS program official  explained that the acquisition group typically fulfills the COR role for CMS  contracts and that it had been unusual for those functions to be provided  by another office. Upon assuming oversight responsibilities, the new COR  could not locate a complete set of FFM and data hub deliverables and the  original COR was unable to provide them. Instead, the new COR had to  request all monthly status and financial reports directly from the  contractors. When the new COR began reviewing the reports in the fall of  2012, he said he noticed that the FFM contractor had not been projecting  the burn rate, a key measure that shows how quickly money is being  spent. The COR asked the contractor to provide the figures in November  2012, at which point the cost growth was identified, even though the  contract had been modified in August 2012 to add almost $36 million to  the task order. We found that the burn rate was not included in earlier  reports, but its absence had gone unnoticed due to ineffective contract  oversight. In November 2007, we had found internal control deficiencies  at CMS related to the inadequate review of contractor costs.
section_title: CMS Identified Significant Contractor Performance Issues for the FFM Task Order but Took Limited Action
paragraphs: CMS took limited action to address significant FFM contractor  performance issues as the October 1, 2013, deadline for establishing  enrollment through the website neared, and ultimately hired a new  contractor to continue FFM development. Late in the development  process, CMS became increasingly concerned with CGI Federal’s  performance. In April and November 2013, CMS provided written  concerns to CGI Federal regarding its responsiveness to CMS’s direction  and FFM product quality issues. In addition, in August 2013, CMS was  prepared to take action to address the contractor’s performance issues  that could have resulted in withholding of fee; however, CMS ultimately  decided to work with CGI Federal to meet the deadline. CMS contracting  and program officials stated that the contract limited them to only  withholding fee as a result of rework. Ultimately, CMS declined to pay  only about $267,000 of requested fee. This represented about 2 percent  of the $12.5 million in fee paid to CGI Federal. Rather than pursue the  correction of performance issues with CGI Federal, in January 2014 CMS  awarded a new one-year contract to Accenture Federal Services for $91  million to continue FFM development. This work also has experienced  cost increases due to new requirements and other enhancements, with  costs increasing to over $175 million as of June 2014.
subsections: section_title: CMS Deemed Early Contractor Performance Satisfactory and Took Limited Action to Address Significant Contractor Performance Issues as the Deadline Neared
paragraphs: CMS generally found CGI Federal and QSSI’s performance to be  satisfactory in September 2012, at the end of the first year of  development. CMS noted some concerns related to FFM contractor  performance, such as issues completing development and testing on  time; however, CMS attributed these issues to the complexity of the FFM   Further, according to  and CMS’s changing requirements and policies.program officials, during the first year of FFM development, few defined  products were to be delivered as requirements and the system’s design  were being finalized. For example, as previously identified in this report,  under the revised FFM development schedule the final detailed design  review for the FFM—a key development milestone review to ensure that  the design meets requirements before it is developed and tested, was  delayed from June 2012 to March 2013. Therefore, CMS had limited  insight into the quality of CGI Federal’s deliverables during the first year  as development and testing of certain FFM functionality had not yet been  completed. CMS found QSSI’s performance satisfactory in September  2012. CMS program officials told us that they did not identify significant  contractor performance issues during data hub development, and that the  data hub generally worked as intended when Healthcare.gov was  launched on October 1, 2013.
subsections: section_title: CMS Identified Significant FFM Contractor Performance Issues as the Deadline Approached, but CMS Opted Against Taking Remedial Contractual Actions at That Time
paragraphs: During the second year of development, which began in September 2012,  CMS identified significant FFM contractor performance issues as the  October 1 deadline approached (see figure 7). In April 2013, CMS  identified concerns with CGI Federal’s performance, including not  following CMS’s production deployment processes and failing to meet  established deadlines, as well as continued communication and  responsiveness issues. To address these issues, the contracting officer’s  representative (COR) sent an email to CGI Federal outlining CMS’s  concerns and requesting that CGI Federal provide a plan for correcting  the issues moving forward. CMS accepted CGI Federal’s mitigation plan.  The plan included changes, according to CGI Federal officials, to  accommodate CMS’ communication practices, which CGI Federal  believed to be the root cause of some of the CMS-identified issues. CMS  contracting officials said that they were satisfied with CGI Federal’s  overall mitigation approach, which seemed to address the performance  issues that CMS had identified at that time.
According to CMS program officials, they grew increasingly concerned  with CGI Federal’s performance late in the development process in June  and July 2013 as the scheduled launch date approached. Specifically,  CMS program officials identified concerns with FFM technical and code  quality during early testing of the enrollment process. The initial task order  schedule had called for the FFM’s development and test phase to be  complete by this point, but these efforts were delayed in the revised  schedule. CMS program officials explained that they identified issues  such as inconsistent error handling, timeouts, and pages going blank.  Overall, more than 100 defects were identified, which resulted in delays  while CGI Federal worked to correct them. According to CGI Federal  officials, the code reflected the instability of requirements at that time.  However, once requirements were more stable, after October 2013, the  contractor was able to quickly make improvements to the FFM’s  performance.
In August 2013, CMS contracting and program officials decided to take  formal action to address their concerns with CGI Federal’s performance  by drafting a letter to the contractor. Specifically, CMS identified concerns  with the contractor’s code quality, testing, failure to provide a key  deliverable, and scheduled releases not including all agreed upon  functionality. The letter further stated that CMS would take aggressive  action, such as withholding fee in accordance with the FAR, if CGI  Federal did not improve or if additional concerns arose. However, the  contracting officer withdrew the letter one day after it was sent to CGI  Federal, after being informed that the CMS Chief Operating Officer  preferred a different approach. CMS contracting and program officials told  us that, rather than pursue the correction of performance issues, the  agency determined that it would be better to collaborate with CGI Federal  in completing the work needed to meet the October 1, 2013, launch. CMS  contracting officials told us that the agency did not subsequently take any  remedial actions to address the issues outlined in the August 2013 letter.
By early September 2013, CMS program officials told us that they  became so concerned about the contractor’s performance that CMS  program staff moved their operations to CGI Federal’s location in  Herndon, Virginia to provide on-site direction leading up to the FFM  launch. CMS had identified issues such as deep-rooted problems with  critical software defects during testing and demonstration of the product  and CGI Federal’s inability to perform quality assurance adequately  including full testing of software. According to CMS program officials,  CMS staff members worked on-site with CGI Federal for several weeks to  get as much functionality available by October 1, 2013, as possible,  deploying fixes and new software builds daily.
section_title: CMS Took Some Actions to Hold the FFM Contractor Accountable after the Healthcare.gov Launch
paragraphs: After the Healthcare.gov launch on October 1, 2013, CMS contracting  officials began preparing a new letter detailing their concerns regarding  contractor performance which was sent to CGI Federal in November  2013. In its letter, CMS stated that CGI Federal had not met certain  requirements of the task order statement of work, such as FFM  infrastructure requirements including capacity and infrastructure  environments, integration, change management, and communication  issues—some of which had been previously expressed in writing to CGI  Federal. In addition, CMS stated that some of these issues contributed to  problems that Healthcare.gov experienced after the October 1, 2013  launch. CMS’s letter also requested that CGI Federal provide a plan to  address these issues. CGI Federal responded in writing, stating that it  disagreed with CMS’s assertion that CGI Federal had not met the  requirements in the FFM statement of work. In its letter, CGI Federal  stated that delays in CMS’s establishment and finalization of  requirements influenced the time available for development and testing of  the FFM. CGI Federal further stated that disruptions to its performance as  a result of delays in finalizing requirements were compounded by the  scheduled launch date, which resulted in CMS reprioritizing tasks and  compressing time frames to complete those tasks. CGI Federal officials  said they did not provide a formal plan for addressing CMS’s concerns  because they regarded them as unfounded, but agreed to work with CMS  to avoid future issues and optimize the FFM’s performance.
In addition, after the October 1, 2013, launch, CMS contracting officials  told us that they provided additional FFM oversight by participating in  daily calls with CGI Federal on the stability of the FFM and the status of  CGI Federal’s work activities. Contracting officials told us that the  increased oversight of FFM development helped to fix things more  quickly. Further, the COR increasingly issued technical direction letters to  clarify tasks included in the FFM statement of work and focus CGI  Federal’s development efforts.  technical direction letters to CGI Federal in October 2013, directing CGI  Federal to follow the critical path for overall performance improvement of  the FFM, purchase software licenses, and collaborate with other  stakeholders, among other things. According to program officials, written  technical direction letters issued by the COR had more authority than  technical direction provided by the GTL.
subsections: section_title: CMS Declined to Pay FFM Contractor Fee for Rework
paragraphs: Technical direction letters provide supplementary guidance to contractors regarding  tasks contained in their statements of work or change requests. pays the contractor’s allowable costs, plus an additional fee that was  negotiated at the time of award. This means that despite issues with CGI  Federal’s performance, including CGI Federal’s inability to deliver all  functionality included in the FFM statement of work, CMS was required to  pay CGI Federal for allowable costs under the FFM task order. CGI  Federal’s task order provides that, if the services performed do not  conform with contract requirements, the government may require the  contractor to perform the services again for no additional fee. If the work  cannot be corrected by re-performance, the government may, by contract  or otherwise, perform the services and reduce any contractor’s fee by an  amount that is equitable under the circumstances, or the government may  terminate the contract for default.
Even though CMS was obligated to pay CGI Federal’s costs for the work  it had performed for the FFM, CMS contracting and program officials said  they could withhold only the portion of the contractor’s fee that it  calculated was associated with rework to resolve FFM defects. Ultimately,  CMS declined to pay about $267,000 of the fixed fee requested by CGI  Federal. This is approximately 2 percent of the $12.5 million in fixed fee  that CMS paid to CGI Federal. Officials from CGI Federal said that they  disagreed with the action and that the CMS decisions were not final and  they could reclaim the fee by supplying additional information. CMS  contracting and program officials told us that it was difficult to distinguish  rework from other work. For example, program officials explained that it  was difficult to isolate work that was a result of defects versus other work  that CGI Federal was performing, and then calculate the corresponding  portion of fee to withhold based on hours spent correcting defects.
section_title: Contractor’s Total Fee Increased during Development
paragraphs: Through each contract modification, as CMS increased the cost of  development, it also negotiated additional fixed fee for the FFM and data  hub contractors. Under the original award of $55.7 million, CGI Federal  would have received over $3.4 million in fee for work performed during  the development period. As of February 2014, when CMS had obligated  over $209 million dollars for the FFM effort, CMS negotiated and CGI  Federal was eligible to receive more than $13.2 million in fee.2014, CMS had paid CGI Federal $12.5 million in fee. Likewise, CMS  negotiated additional fixed fee for the data hub task order, QSSI’s eligible  fee rose from over $716,000 under the original $29.9 million award to  more than $1.3 million for work performed through February 2014.
section_title: Costs Continue to Increase with New FFM Contractor
paragraphs: Rather than pursue the correction of performance issues and continuing  FFM development with CGI Federal, CMS determined that its best  chance of delivering the system and protecting the government’s financial  interests would be to award a new contract to another vendor. In January  2014, CMS awarded a one-year sole source contract (cost-plus-award- fee) with an estimated value of $91 million to Accenture Federal Services  to transition support of the FFM and continue the FFM development that  CGI Federal was unable to deliver. CMS’s justification and approval  document for the new award states that the one-year contract action is an  interim, transitory solution to meet CMS’s immediate and urgent need for  specific FFM functions and modules—including the financial management   This work has also experienced cost increases. Figure 8 shows  module.increases in obligations for the Accenture Federal Services contract since  award in January 2014.
The financial management module of the FFM includes the services  necessary to spread risk among issuers and to accomplish financial  interactions with issuers. Specifically, this module tracks eligibility and  enrollment transactions and subsidy payments to insurance plans,  integrates with CMS’s existing financial management system, provides  financial accounting and outlook for the entire program, and supports the  reconciliation calculation and validation with IRS.
According to the CMS justification and approval document, CMS  estimated that it would cost $91 million over a one-year period for  Accenture Federal Services to complete the financial management  module and other FFM enhancements. As of June 5, 2014, the one-year  contract had been modified six times since contract award and CMS had  obligated more than $175 million as a result of new requirements,  changes to existing requirements, and new enhancements. For example,  CMS modified the contract to incorporate additional work requirements  and functionality related to the Small Business Health Options Program  marketplace, state-based marketplace transitions, and hardware  acquisition.
CMS had yet to fully define requirements for certain FFM functionality,  including the financial management module, when the new contract to  continue FFM development was awarded in January 2014. Accenture  Federal Services representatives told us that while they had a general  understanding of requirements at the time of award, their initial focus  during the period January through April 2014 was on transitioning work  from the incumbent contractor and clarifying CMS’ requirements.  Accenture Federal Services representatives attributed contract increases  during this period to their increased understanding of requirements, as  well as clarifying additional activities requested under the original  contract. Further, although the justification and approval document  stressed that delivery of the financial management module was needed  by mid-March 2014, contracting and program officials explained that time  frames for developing the module were extended post-award, and as of  June 2014, the financial management module was still under  development. Financial management module functionality is currently  scheduled to be implemented in increments from June through December  2014.
section_title: Conclusions
paragraphs: CMS program and contracting staff made a series of planning decisions  and trade-offs that were aimed at saving time, but which carried  significant risks. While optimum use of acquisition planning and oversight  was needed to define requirements, develop solutions, and test them  before launching Healthcare.gov and its supporting systems, the efforts  by CMS were plagued by undefined requirements, the absence a  required acquisition strategy, confusion in contract administration  responsibilities, and ineffective use of oversight tools. In addition, while  potentially expedient, CMS did not adhere to the governance model  designed for the FFM and data hub task orders, resulting in an ineffectual  governance process in which scheduled design and readiness reviews  were either diminished in importance, delayed, or skipped entirely. By  combining that governance model with a new IT development approach  the agency had not tried before, CMS added even more uncertainty and  potential risk to their process. The result was that problems were not  discovered until late, and only after costs had grown significantly.
As FFM contractor performance issues were discovered late in  development, CMS increasingly faced a choice of whether to stop  progress and pursue holding the contractor accountable for poor  performance or devote all its efforts to making the October deadline. CMS  chose to proceed with pursuing the deadline. After October 1, 2013, CMS  decided to replace the contractor, but in doing so had to expend  additional funds to complete essential FFM functions. Ultimately, more  money was spent to get less capability.
Meanwhile, CMS faces continued challenges to define requirements and  control costs to complete development of the financial management  module in the FFM. Unless CMS takes action to improve acquisition  oversight, adhere to a structured governance process, and enhance other  aspects of contract management, significant risks remain that upcoming  open enrollment periods could encounter challenges going forward.
section_title: Recommendations for Executive Action
paragraphs: In order to improve the management of ongoing efforts to develop the  federal marketplace, we recommend that the Secretary for Health and  Human Services direct the Administrator of the Centers for Medicare &  Medicaid Services to take the following five actions:
Take immediate steps to assess the causes of continued FFM cost  growth and delayed system functionality and develop a mitigation plan  designed to ensure timely and successful system performance.
Ensure that quality assurance surveillance plans and other oversight  documents are collected and used to monitor contractor performance.
Formalize existing guidance on the roles and responsibilities of  contracting officer representatives and other personnel assigned  contract oversight duties, such as government task leaders, and  specifically indicate the limits of those responsibilities in terms of  providing direction to contractors.
Provide direction to program and contracting staff about the  requirement to create acquisition strategies and develop a process to  ensure that acquisition strategies are completed when required and  address factors such as requirements, contract type, and acquisition  risks.
Ensure that information technology projects adhere to requirements  for governance board approvals before proceeding with development.
section_title: Agency Comments, Third-Party Views, and Our Evaluation
paragraphs: We provided a draft of this product to the Department of Health and  Human Services and the Centers for Medicare & Medicaid Services for  review and comment.
In its written comments, which are reprinted in appendix III, HHS  concurred with four of our five recommendations and described the  actions CMS is taking to improve its contracting and oversight practices.  HHS partially concurred with our recommendation that CMS assess the  causes of continued FFM cost growth. The agency says that CMS  already has assessed the reasons for cost growth under the CGI Federal  task order and that any increase in costs since the contract with  Accenture Federal Services for continued development of the FFM was  finalized is attributable to additional requirements, not cost overruns. We  recognize that much of the increase in costs under the Accenture Federal  Services contract is due to new requirements or enhancements.  Nevertheless, based on our review of the contract modifications, not all  the increase in costs from $91 million to more than $175 million, when  measured from the initial projection, is attributable to new requirements.  For example, as CMS stated in its comments, after additional analysis  CMS determined a $30 million cost increase was needed to complete the  contract’s original scope of work. We continue to believe that a further  assessment is needed to ensure that costs as well as requirements are  under control and that the development of the FFM is on track to support  the scheduled 2015 enrollment process.
All three contractors, as well as HHS, provided additional technical  comments, which we incorporated in the report where appropriate.
We are sending copies of this report to the Secretary of Health and  Human Services and the Administrator of the Centers for Medicare &  Medicaid Services. In addition, the report is available at no charge on the  GAO website at http://www.gao.gov/.
If you or your staff have any questions about this report, please contact  William T. Woods at (202) 512-4841 or woodsw@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. GAO staff who made key  contributions to this report are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines selected contracts and task orders central to the  development and launch of the Healthcare.gov website by assessing (1)  Centers for Medicare & Medicaid Services (CMS) acquisition planning  activities; (2) CMS oversight of cost, schedule, and system capability  changes; and (3) actions taken by CMS to identify and address contractor  performance issues.
To address these objectives, we used various information sources to  identify CMS contracts and task orders related to the information  technology (IT) systems supporting the Healthcare.gov website.  Specifically, we reviewed data in the Federal Procurement Data System- Next Generation, which is the government’s procurement database, to  identify CMS contracts and task orders related to the IT systems  supporting the Healthcare.gov website and amounts obligated for fiscal  years 2010 through March 2014. In addition, we reviewed CMS provided  data on the 62 contracts and task orders related to the IT systems  supporting the Healthcare.gov website and amounts obligated as of  March 2014. To select contracts and task orders to include in our review,  we analyzed Federal Procurement Data System-Next Generation and  CMS data to identify contracts and task orders that represent large  portions of spending for Healthcare.gov and its supporting systems. We  then selected one contract and two task orders issued under an existing  2007 contract and interviewed contracting officials in CMS’s Office of  Acquisition and Grants Management and program officials in CMS’s  Office of Information Services to confirm that these contracts are central  to development of Healthcare.gov and its supporting systems. The  contract and task orders combined accounted for more than 40 percent of  the total CMS reported obligations related to the development of  Healthcare.gov and its supporting systems as of March 2014. Specifically,  we selected the task orders issued to CGI Federal Inc. (CGI Federal) for  the development of the federally facilitated marketplace (FFM) system  and to QSSI, Inc. QSSI for the development of the federal data services  hub (data hub) in September 2011—and the contract awarded to  Accenture Federal Services in January 2014 to continue FFM  development and enhance existing functionality.
To describe federal implementation costs for Healthcare.gov and its  supporting systems, we interviewed program officials and obtained  relevant documentation to identify eight agencies that reported IT-related  obligations or used existing contracts and task orders or operating  budgets to support the development and launch of the Healthcare.gov  website. These eight agencies include the Centers for Medicare &  Medicaid Services (CMS), Internal Revenue Service (IRS), Social  Security Administration, Veterans Administration (VA), Peace Corps,  Office of Personnel Management, Department of Defense (DOD), and  Department of Homeland Security. We then obtained and analyzed  various types of agency-provided data to identify overall IT-related costs  for Healthcare.gov and its supporting systems. Three agencies, including  CMS, IRS, and VA reported almost all of the IT-related obligations  supporting the implementation of Healthcare.gov and its supporting  systems as of March 2014. We performed data reliability checks on  contract obligation data provided by these three agencies, such as  checking the data for obvious errors and comparing the total amount of  funding obligated for each contract and task order as reported by each  agency to data on contract obligations in Federal Procurement Data  System-Next Generation or USASpending.gov.were sufficiently reliable for the purpose of this report.
We found that these data  To assess CMS acquisition planning activities, we reviewed Federal  Acquisition Regulation (FAR) and relevant Department of Health and  Human Services (HHS)/CMS policies and guidance. We also evaluated  contract file documents for three selected contracts and task orders,  including acquisition planning documentation, request for proposal,  statements of work, cost estimates, and technical evaluation reports to  determine the extent to which CMS’s acquisition planning efforts met FAR  and HHS/CMS requirements. In assessing CMS’s acquisition planning  efforts, we looked for instances where CMS took steps to mitigate  acquisition program risks during the acquisition planning phase, including  choice of contract type and source selection methodology. In addition, we  interviewed CMS contracting and program officials to gain a better  understanding of the acquisition planning process for select contracts and  task orders including the rationale for choosing the selected contract type  and the analysis conducted to support the source selection process. We  also reviewed prior GAO reports on CMS contract management to assess  the extent to which CMS’s acquisition planning activities addressed  issues previously identified by GAO.
To assess CMS oversight of cost, schedule, and system capability  changes, we analyzed contract file documents for one selected contracts  and two task orders. As part of our assessment of the selected contracts  and task orders, we reviewed contract modifications, contractor monthly  status and financial reports, statements of work, contractor deliverables,  schedule documentation, and contracting officer’s representative files,  and meeting minutes to determine if there were any changes and whether  system development proceeded as scheduled. We performed a data  reliability check of cost data for selected contracts and task orders by  comparing contract modification documentation to contract obligation data  in Federal Procurement Data System-Next Generation. To evaluate the  extent to which CMS adhered to its governance process, we compared  the governance model the agency intended would guide the design,  development, and implementation of Healthcare.gov and its supporting  systems, to the development process the agency actually used for the  FFM and data hub. We also obtained and analyzed documentation from  governance reviews to identify the date and content of the reviews to  determine if key milestone reviews were held in accordance to the  development schedule. In addition, we reviewed FAR and federal  standards for internal control for contract oversight to evaluate the extent  to which CMS’s approach to contract oversight for the selected contracts  and task orders met FAR and federal internal control standards. We  interviewed CMS contracting and program officials to gain a better  understanding of FFM and data hub cost, schedule, and system  capabilities, and to obtain information on the organization and staffing of  offices and personnel responsible for performance monitoring for selected  contracts and task orders. We also interviewed contractors to obtain their  perspective on CMS’s oversight of cost, schedule, and system  capabilities. Further, as part of our assessment of CMS’s development  approach for the FFM and data hub, we reviewed prior GAO work  regarding information technology and development.
To assess actions taken by CMS to identify and address contractor  performance issues, we reviewed relevant FAR and HHS guidance for  contract monitoring and inspection of services to identify steps required  for selected contracts and task orders and recourse options for  unsatisfactory performance. In addition, we obtained and analyzed  contract file documentation including contracting officer’s representative  files, contractor deliverables, contractor monthly status and financial  reports, contractor performance evaluations, and meeting minutes to  determine the extent to which performance was reported and what steps,  if any, were taken to address any issues. To determine contractor fee not  paid during development, we obtained and analyzed CMS contractor  invoice logs and contract payment notifications. We also interviewed CMS  contracting and program officials to obtain additional information  regarding contractor performance and actions taken by CMS, if any, to  address contractor performance issues.
We conducted this performance audit from January 2014 to July 2014, in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Cumulative Cost Increases for the Task Orders for Developing the Federally Facilitated Marketplace System and Federal Data Services Hub Task Orders
subsections: section_title: Task order issued/ modified Federally Facilitated Marketplace System (FFM) Issuance
paragraphs: $55,744,082  FFM task order issued to CGI Federal  $91,515,772  Obligates an additional $35.8 million, primarily to provide for new  and increased system requirements resulting from program office  decisions and finalized regulations.  $91,515,772  No cost modification for administrative purposes, including  identifying a new contracting officer’s representative.  $27,688,008   $119,203,779  Obligates an additional $27.7 million needed to avert a potential  cost overrun. The funding supports an increased level of effort to  add system functionality not included in the statement of work and  increased infrastructure needs.   $474,058   $119,677,837  Obligates approximately $474,000 for additional infrastructure  requirements, specifically requirements for the content delivery  network that delivers web services.  $58,143,472   $177,821,309  Modified to extend the period of performance for FFM development  until February 28, 2014, and obligate an additional $58.1 million,  primarily to support the extension.  $18,215,807   $196,037,116  Obligates an additional $18.2 million to purchase a software  license.  196,037,116  Modified to issue a change order directing the contractor to  develop and implement an identity management software solution.  $1,479,309   $197,516,425  Obligates $1.5 million to increase capacity of the content delivery  network from 50 terabytes to 400 terabytes.  $6,981,666   $204,498,091  Obligates $7.0 million to definitize the change order issued under  Modification 7. It also funds software licenses and the industry  experts hired to improve system performance.  0  $204,498,091  Modified to issue a change order directing the contractor to begin  transitioning services to a new contractor.  $5,133,242   $209,631,333  Obligates $4.8 million to definitize the change order issued under  Modification 10 and fund post-transition consulting services  through April 30, 2014.
section_title: Data Hub Issuance
paragraphs: $29,881,693  Data hub task order issued to QSSI  ($4,180,786) $25,700,907  Modified to cancel a stop work order that was issued due to a GAO  bid protest and direct the contractor to continue performance of the  task order. Obligations are reduced by $4.2 million in accordance  with the contractor’s revised task order proposal (submitted as part  of the bid protest process).
section_title: Task order issued/ modified Modification 2
paragraphs: $48,717,984  Obligates an additional $23.0 million, primarily to provide for new  and increased system requirements resulting from program office  decisions and finalized regulations.  $48,717,984  No cost modification for administrative purposes, including  identifying a new contracting officer’s representative.  $53,709,598  Obligates $5.0 million to fund an electronic data interchange tool  and related labor to support enrollment services.  $84,527,128  Modified to extend the period of performance for data hub  development until February 28, 2014, and obligate an additional  $30.8 million, primarily to support the extension.  $84,527,128  No cost modification to transfer funds among contract line items  and revise personnel.  $99,657,839  Modified to exercise option year 1: Operations and Maintenance.
section_title: Appendix III: Comments from the Department of Health and Human Services
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, W. William Russell, Assistant  Director; Jennifer Dougherty; Elizabeth Gregory-Hosler; Andrea Yohe;  Susan Ditto; Julia Kennon; John Krump; Ken Patton; Roxanna Sun; and  Kevin Walsh made key contributions to this report .