State Health Insurance Marketplaces: CMS Should Improve Oversight of State Information Technology Projects
Published: 2015-09-16T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Patient Protection and Affordable Care Act required the establishment of health insurance exchanges—or marketplaces—to allow consumers and small employers to compare, select, and purchase health insurance plans. States can elect to establish a state-based marketplace, or cede this authority to CMS to establish a federally facilitated marketplace. To assist states in establishing their marketplaces and supporting IT systems, federal funding was made available, including grants and Medicaid matching funds. CMS has responsibilities for overseeing states' use of these funds and the establishment of their marketplaces.
The objectives of this study were to (1) determine how states have used federal funds for IT projects to support their marketplaces and the status of the marketplaces, (2) determine CMS's and states' roles in overseeing these projects, and (3) describe IT challenges states have encountered and lessons learned. To do this, GAO surveyed the 50 states and the District of Columbia, reviewed relevant documentation from the states and CMS, and interviewed CMS officials.
section_title: What GAO Found
paragraphs: States reported to the Department of Health and Human Services' (HHS) Centers for Medicare & Medicaid Services (CMS) that they spent about $1.45 billion in federal marketplace grant funding on information technology (IT) projects supporting health insurance marketplaces, as of March 2015. The majority of this spending was for state-based marketplaces (i.e., marketplaces established and operated by the states). These marketplaces reported spending nearly 89 percent of the funds on IT contracts, and CMS has ongoing efforts to track states' IT spending in more detailed categories. States also reported spending, as of December 2014, $2.78 billion in combined federal and state funds designated for Medicaid eligibility and enrollment systems—a portion of which was used for marketplace IT projects. However, the specific amount spent on marketplace-related projects was uncertain, as only a selected number of states reported to GAO that they tracked or estimated this information. Regarding the status of states' marketplace IT projects, 14 states with state-based marketplaces had developed and were operating IT systems to support their marketplaces, but, as of February 2015, not all system functions were complete. In addition, as of November 2014, 7 of 37 states using the federal marketplace system could not transfer health insurance applications between their state Medicaid systems and a key component of the federal marketplace or had not completed testing or certification of these functions. According to CMS officials, states operating their own IT systems and states using the federal marketplace system were continuing to improve the development and operation of their marketplaces in the enrollment period that began in November 2014.
CMS tasked various offices with responsibilities for overseeing states' marketplace IT projects. However, the agency did not always clearly document, define, or communicate its oversight roles and responsibilities to states as called for by best practices for project management. According to some states, this resulted in instances of poor communication with CMS, which adversely affected states' deadlines, increased uncertainty, and required additional work. CMS also did not involve all relevant senior executives in decisions to approve federal funding for states' IT marketplace projects; such involvement, according to leading practices for investment management, can increase accountability for decision making. Further, while CMS established a process that required the testing of state marketplace systems to determine whether they were ready to be made operational, these systems were not always fully tested, increasing the risk that they would not operate as intended. For their part, states oversaw their IT projects through state agencies or quasi-governmental entities, depending on marketplace type, as well as using other oversight mechanisms.
States reported a number of challenges in establishing the systems supporting their marketplaces. These fell into several categories, including project management and oversight, system design and development, resource allocation and distribution, and marketplace implementation and operation. States also identified lessons learned from dealing with such challenges, including the need for strong project management and clear requirements development. CMS has taken various actions to respond to state challenges, identify lessons learned, and share best practices with states; continuing these efforts will be important as states work to complete their marketplace systems.
section_title: What GAO Recommends
paragraphs: GAO is recommending that CMS define and communicate its oversight roles and responsibilities, ensure senior executives are involved in funding decisions for state IT projects, and ensure that states complete testing of their systems before they are put into operation. HHS concurred with GAO's recommendations.

section_title: Letter
paragraphs: The Patient Protection and Affordable Care Act (PPACA), signed into  law on March 23, 2010, includes provisions to reform aspects of the  private health insurance market and expand the availability and  affordability of health care coverage. The act required the establishment  of health insurance exchanges, now commonly referred to as  “marketplaces,” in each state and the District of Columbia by January 1,  2014. These marketplaces are required to allow consumers, such as  individuals and small employers, to compare, select, and purchase health  insurance offered by participating private issuers of qualified health  plans.
The Department of Health and Human Services’ (HHS) Centers for  Medicare & Medicaid Services (CMS) is responsible for overseeing the  establishment of the marketplaces, including providing funding and  oversight for states’ marketplace development efforts and creating a  federally facilitated marketplace that can be used by states that do not  choose to establish and operate their own. For their part, states are  responsible for undertaking various efforts, including information  technology (IT) projects needed to support the development of their own  marketplaces or connections to the federal marketplace.
As with the federal marketplace, states’ marketplaces began enrolling  individuals in health insurance plans on October 1, 2013. However,  individuals attempting to access the systems supporting the marketplaces  encountered various challenges. In light of these challenges, you asked  us to review the states’ and CMS’s actions related to the IT projects  supporting states’ health insurance marketplaces. Our specific objectives  were to (1) determine how states have used federal funds for IT projects  to establish, support, and connect to health insurance marketplaces,  including amounts spent, and the overall status of their development and  operation; (2) determine CMS’s and states’ roles in overseeing these  state IT projects; and (3) describe IT challenges that states have  encountered in developing and operating their marketplaces and  connected systems, and lessons learned from their efforts.
To address the objectives, we administered a survey to all 50 states and  the District of Columbia to collect pertinent information about the IT  projects supporting their health insurance marketplaces. We pre-tested  the survey with marketplace and Medicaid officials from 7 states to  ensure that the questions were clear, comprehensive, and unbiased, and  to minimize the burden the survey placed on respondents. We developed  two versions of this survey: one for states that established their own  marketplaces and one for states that used the federally facilitated  marketplace. Based on CMS’s classification of states for the first  enrollment period, 17 states received the state-based version of the  survey, and 34 states received the federally facilitated version. The  survey was administered between September 30, 2014, and November  19, 2014, and focused on IT projects that supported health insurance  marketplaces for individuals. We received responses from 46 states and  the District of Columbia.
To determine how states have used federal funds for IT projects to  establish, support, and connect to health insurance marketplaces, and the  overall status of their development and operation, we reviewed guidance  that CMS provided to the states regarding federal funding for and the  development of marketplaces and Medicaid eligibility and enrollment  systems, such as the marketplace grant funding opportunity  announcement. We also reviewed sections of GAO’s IT investment  management framework relevant to managing project costs. We then  analyzed the states’ survey responses regarding their project costs and  development, as well as any supporting documentation that they provided  concerning applicable federal marketplace grants and Medicaid funding.  In addition, we reviewed CMS data on spending of marketplace grant  funds and Medicaid funding for IT. Specifically, we reviewed funding and  status documentation submitted by the states to CMS, including state IT  spending and status summaries, and asked CMS officials responsible for  reviewing the states’ federal marketplace grant and Medicaid matching  funding a series of questions concerning its accuracy and reliability. We  determined that the funding data provided in the responses were  sufficiently reliable for our purposes and noted any limitations of the state- reported spending data in the report.
To determine CMS’s and states’ roles in overseeing the development of  marketplace IT solutions, we analyzed the survey responses, HHS/CMS  guidance provided to states, and CMS’s policies and procedures and  other documentation describing its roles and responsibilities as applicable  to states’ marketplace development efforts. We compared CMS’s policies  and procedures to best practices included in GAO’s IT investment  management framework and to the Project Management Institute’s A  Guide to the Project Management Body of Knowledge (PMBOK®  Guide)  to identify whether CMS had established roles and  responsibilities that were consistent with industry practices. We also  assessed the manner in which CMS communicated guidance and  information on roles and responsibilities to the states.
Further, we reviewed CMS’s funding oversight process and compared it  to the sections of GAO’s IT investment management framework that are  relevant to the management of project cost to determine if the agency  followed best practices for overseeing the marketplace investments. We  also reviewed CMS’s Enterprise Life Cycle guidance for systems  development reviews, and reports documenting states’ operational  readiness reviews to assess the extent to which CMS followed its  processes. We also reviewed the survey responses and supporting  documentation to determine states’ marketplace oversight roles and how  the states viewed CMS’s oversight and guidance in regard to their  marketplace-related projects. Lastly, we interviewed CMS officials  responsible for overseeing implementation of the state marketplaces to  obtain their perspectives on their marketplace roles.
To describe IT challenges encountered in developing and operating the  marketplaces and connected systems and lessons learned from these  efforts, we analyzed the survey responses related to challenges and  lessons learned. Specifically, in administering the survey, we asked the  states to rate their experiences with each of various challenges  presented, based on the type of marketplace they used (i.e., one  established by the state or the federally facilitated marketplace). The  challenges to be considered by states that developed their own  marketplaces were divided into five areas in the survey (project  management and oversight, marketplace IT solution design, marketplace  IT solution development, resource allocation and distribution, and  marketplace implementation and operation). The challenges to be  considered by states that used the federally facilitated marketplace were  divided into two areas (project management and oversight and system  design and development) based on the IT work each marketplace  performs.
We asked states to rate their experiences with each of these challenge  areas using a 5-point scale with the following response options: very  great challenge, great challenge, moderate challenge, somewhat of a  challenge, or little or no challenge. In analyzing the states’ ratings of the  challenges, we used combined counts of “very great” and “great”  responses to identify the greatest challenges for each area. We then  discussed the top two greatest challenges in this report. If a challenge  area applied to both a state-based marketplace and a state with a  federally facilitated marketplace, we selected the greatest challenges  from each marketplace type.
In addition, we asked states to identify lessons learned as they applied to  the categories of challenges. We then analyzed states’ responses to  determine the number of lessons learned reported by each state. Further,  we obtained input from CMS officials responsible for overseeing states’  marketplace implementation regarding their perspectives on the states’  challenges and lessons learned. A more detailed discussion of our  objectives, scope, and methodology is provided in appendix I.
We conducted this performance audit from April 2014 to September 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: PPACA directed each state to establish and operate a health insurance  marketplace by January 1, 2014. In cases where states elected not to  establish and operate a marketplace, the law directed the federal  government to establish and operate a health insurance marketplace on  their behalf. These marketplaces were expected to provide a seamless,  single point-of-access for individuals to enroll in private health insurance  plans and apply for income-based financial assistance established under  the law.
PPACA and HHS regulations and guidance require every state to have  marketplace capabilities that enable them to carry out four key functions,  among others:
Eligibility and enrollment. The marketplace must enable individuals  to assess and determine their eligibility for enrollment in healthcare  coverage. In addition, the marketplace must provide individuals the  ability to obtain an eligibility determination for other federal healthcare  coverage programs, such as Medicaid and the Children’s Health  Insurance Program (CHIP). Once eligibility is determined,  individuals must be able to apply for and enroll in applicable coverage  options.
Plan management. The marketplace is to provide a suite of services  for state agencies and health plan issuers to facilitate activities such  as submitting, monitoring, and renewing qualified health plans.
Financial management. The marketplace is to facilitate payments of  premiums to health plan issuers and also provide additional services  such as payment calculation for risk adjustment analysis and cost- sharing reductions for individual enrollments.
Consumer assistance. The marketplace must be designed to  provide support to consumers in completing an application, obtaining  eligibility determinations, comparing coverage options, and enrolling in  healthcare coverage.
To provide these capabilities, PPACA further required the states, as well  as HHS (who delegated this role to CMS) to establish supporting  automated systems and capabilities. Toward this end, states and CMS  undertook projects to design, develop, implement, and operate health  insurance marketplace systems.
States electing to establish their own marketplaces (hereafter referred to  as a state-based marketplace) were required, in accordance with CMS  guidance and regulations, to develop their own IT solutions, including a  web portal for individual consumers to interact with and select healthcare  coverage, as well as supporting systems that perform functions such as  real-time eligibility queries, transferring application information to state  Medicaid/CHIP agencies, sending taxpayer information to the Internal  Revenue Service, and exchanging enrollment information with issuers of  qualified health plans.
In addition, state-based marketplace IT solutions were required to  interface with CMS systems designed to exchange information with  external partners, including other federal agencies and states, and  facilitate the electronic payment of insurance premiums to plan issuers.  As an alternative to their web portals, these states were also required to  set up call centers through which consumers could apply for coverage.
A state that operates its own marketplace can request that CMS perform  eligibility and enrollment functions using federal IT systems. We refer to  this as a state-based marketplace using the federal marketplace IT  solution. This type of marketplace evolved when certain states  encountered IT-related challenges during the development of their state  marketplace solutions.
Further, if a state elected not to establish its own marketplace, CMS  assumed some or all aspects of the marketplace operations for that state  using two additional marketplace types:
Federally facilitated marketplace: CMS is responsible for all aspects of  establishing and operating the marketplace including the four key  functions.
Federally facilitated partnership: CMS is responsible for establishing  and operating the eligibility enrollment and financial management  functions, while the state assists with plan management and/or  consumer assistance.
In these cases, states rely to varying degrees on the systems developed  by CMS to support a federally facilitated marketplace. These include  Healthcare.gov—the federal website that serves as the user interface for  individuals to obtain information about health coverage, set up a user  account, select a health plan, and apply for healthcare coverage—and  several supporting systems. The supporting systems include a system for  verifying an applicant’s identity and establishing a login account; a  transactional database to facilitate eligibility and enrollment, plan  management, financial services, and other functions; and a data services  hub that serves as a single portal for exchanging information with external  partners. For example, federal agencies such as the Social Security  Administration (SSA), Department of Homeland Security (DHS), and  Internal Revenue Service (IRS) provide or verify information used in  making determinations of a person’s eligibility for coverage and financial  assistance.
For plan year 2015, 14 states had a state-based marketplace, 3 had a  state-based marketplace using the federal marketplace IT solution, 27  had a federally facilitated marketplace, and 7 had a federally facilitated  partnership (see fig. 1).
Depending on the type of marketplace established in his or her state, an  individual user would apply for health coverage through either their state’s  web portal or through Healthcare.gov. The key functions required to enroll  that individual would then be carried out by a combination of state and  federal systems specific to the type of marketplace.
A general depiction of both the state and federal marketplace IT solutions  is provided in figure 2.
subsections: section_title: Federal Funding Available to States for Establishing Marketplaces
paragraphs: States had access to two sources of federal funding to establish their  marketplaces: federal marketplace grants and Medicaid matching funds.  CMS allows states to use both Medicaid matching funds and marketplace  grants to pay for shared system services and functions that states needed  to establish for marketplace operations, such as developing a rules  engine system and establishing interfaces to the federal data services  hub. Various offices within CMS were tasked with overseeing grant  reviews, Medicaid advanced planning document reviews, and IT gate  reviews to ensure that states followed a standardized funding process for  their marketplace-related IT projects. These offices included the Center  for Consumer Information and Insurance Oversight (CCIIO), Center for  Medicaid and Chip Services (CMCS), and the Office of Technology  Solutions (OTS).
subsections: section_title: Marketplace Grants
paragraphs: PPACA authorized HHS to award federal exchange grants (now referred  to as marketplace grants) for planning and establishing marketplaces.  The act did not specify an exact amount of marketplace grant funding, but  appropriated to HHS, out of any moneys in the Treasury not otherwise  appropriated, an amount necessary to make marketplace grant awards.  The act directed HHS to determine the total amount of funding that it  would make available to each state for each fiscal year and authorized  the department to award marketplace grants to states through December  2014.
On the basis of this authority, HHS established four separate programs  for awarding marketplace grants to states.
Planning Grants: Provided states with resources to conduct the initial  research and planning needed to build a marketplace and determine  how it would be operated and governed. The grants were awarded to  states in 2010 and 2011 and provided 1 year of funding. A state could  receive only one planning grant.
Early Innovator Grants: Provided funding to a state or group of  states that were identified as early leaders in building their  marketplaces, to assist in designing and implementing the IT  infrastructure needed to operate the marketplaces. All marketplace IT  components, including software and data models, developed with  these grants could be adopted and modified by other states to fit their  specific needs. The grants were awarded in February 2011 and the  grant funds were available for 2 years. A state could only receive one  of these grants.
Establishment Grants (Level 1): Provided funding for a 1-year  project period to states pursuing any marketplace type. This funding  was intended to help states undertake additional marketplace  establishment activities, such as changes in response to legislative or  regulatory requirements, developing IT systems, and consulting with  key stakeholders. The grants were awarded between May 2011 and  December 2014. Once awarded, the funds were available for 1 year,  and a state could apply for multiple grants.
Establishment Grants (Level 2): Provided funding for a multi-year  project period to states that have legal authority to implement a  marketplace and are further along in marketplace development and  are pursuing a state-based marketplace. This funding was designed  to help the states carry out all marketplace activities, including  consumer and stakeholder engagement and support, eligibility and  enrollment, plan management, and technology development. The  grants were awarded between May 2011 and December 2014. Once  awarded, the grant funds remain available for up to 3 years. A state  could receive only one grant.
States establishing state-based marketplaces were expected to carry out  activities in a number of areas to receive these marketplace grants.  These activity areas included stakeholder consultation, program  integration, IT systems development, financial management, oversight  and program Integrity, health Insurance market reform, and business  operations of the marketplace.
Once grants were awarded, funding was disbursed using the Payment  Management System, which is an HHS-administered system that  provides federal agencies and grant recipients the tools to manage grant  payments. Grantees submitted progress reports documenting financial  expenditures and program progress through an online data collection  system on a monthly and semi-annual basis.
As of December 31, 2014, CMS had awarded approximately $5.51 billion  in federal marketplace grants to states. Of these grant funds awarded,  CMS had authorized states to spend approximately $2.16 billion on IT to  support state-based marketplaces and federally facilitated partnerships as  of March 2015. According to CMS, funds authorized for IT contracts  could be designated as restricted and required prior approval from the  various CMS offices mentioned previously before the funds could be  spent. For states that opted to use the federally facilitated marketplace, IT  funds were not provided after it was determined that these states were  not establishing a state-based marketplace.
section_title: Medicaid Matching Funds
paragraphs: With the enactment of PPACA, changes to Medicaid eligibility and  enrollment systems were needed in order for the Medicaid program to  operate seamlessly with the marketplaces, as well as to implement new  Medicaid eligibility policies. Specifically, in all states, the Medicaid  eligibility and enrollment system had to be replaced or modernized to  meet the more streamlined enrollment process requirements of PPACA  and its implementing policies, which included real-time transfer of  applications between the state Medicaid agencies and the marketplace  and immediate Medicaid eligibility determinations, regardless of the type  of marketplace a state elected to use.
Under federal law, states are eligible to receive funding, in the form of an  enhanced federal matching rate of 90 percent (referred to as 90/10  funding), for the design, development, or installation of their Medicaid  claims processing and information retrieval systems. Because states’  Medicaid eligibility and enrollment systems had to be replaced or  modernized to meet the PPACA requirements, CMS expanded the  availability of federal Medicaid funds at the enhanced matching rate of 90  percent to help states pay for required changes, including their  interfaces to establish connections to the federal marketplace IT solution  through the federal data services hub or the state marketplace IT solution.  This enhanced federal matching rate is available to cover costs incurred  by the states related to changes to their Medicaid eligibility systems from  April 19, 2011, to December 31, 2015. All states are eligible to obtain the  90/10 funds for IT-related changes they make to their Medicaid eligibility  and enrollment systems.
In addition, a state may receive funding in the form of a 75 percent federal  matching rate for the maintenance and any ongoing costs of operating its  upgraded Medicaid eligibility and enrollment system. The funding is  generally available when the upgraded system becomes operational, and  it does not expire.
In updating their Medicaid eligibility and enrollment systems, states could  use federal funds for full system replacements or for more limited  modifications, with the scope of a state’s changes depending on a  number of factors, including the age of the system and the extent of  integration among state programs.
Federal regulations require the approval of advanced planning documents  in order for states to be able to draw down the 90/10 and 75/25 matching  funds. To access Medicaid matching funding, states must first submit  these planning documents to CMS. In its role as the agency that oversees  the Medicaid program and provides guidance and technical assistance to  states related to Medicaid eligibility and enrollment system changes, CMS  is to review these documents to ensure that certain technical and  operational criteria are met before states are eligible for the funding. To  receive approval, states must develop IT systems that meet technical  standards and conditions. These standards and conditions require states  to develop systems that are flexible, align with the Medicaid Information  Technology Architecture principles, and promote data exchanges and the  reuse of Medicaid technologies across systems and states.
Figure 3 provides a timeline of the health insurance marketplaces’ major  activities previously mentioned, including dates when federal funding  became available and enrollment time frames.
section_title: States Faced Development and Operations Difficulties during the First Marketplace Enrollment Period
paragraphs: During the first enrollment period, states faced difficulties developing and  operating their marketplace IT solutions. For state-based marketplaces,  various sources reported that technical issues varied widely,  contributing to websites that froze midway through the process of  applying for coverage, system crashes, and systems taken offline for  days at a time, ultimately causing applicants to face long waits for  eligibility determinations. One state reported technical problems serious  enough to prevent any online enrollment; thus, thousands of individuals  had to enroll manually using paper applications.
The problems experienced in state-based marketplaces for the first  enrollment period were different in each state, but they included poor system performance and delays in addressing information partially completed software functionality, enrollment errors causing long wait times and applications to get stuck difficulties getting individuals’ identities verified through the systems,  the inability to easily make changes to individuals’ insurance coverage  in response to events such as births or income changes.
States that relied on the federally facilitated marketplace and federally  facilitated partnerships also encountered problems in the development  and operation of their IT solutions during the first enrollment period. For  example, in these states, consumers attempting to enroll in health plans  through Healthcare.gov and its supporting systems were met with  confusing error messages, slow load times for forms and pages, and in  some cases, website outages. We previously reported that  Healthcare.gov and its supporting systems were hindered by inadequate  system capacity, numerous errors in software code, and limited system  functionality—all of which impeded the systems’ performance and their  availability for consumers’ use.
Regarding state Medicaid systems, states with a federally facilitated  marketplace, federally facilitated partnership, or state-based marketplace  using the federal marketplace IT solution reported challenges in  implementing the requirement to transfer or send and receive  applications. For example, none of these types of states were able to  transfer applications via the marketplace by the start of the first  enrollment period on October 1, 2013.
section_title: Prior GAO Reports Highlighted Concerns and Made Recommendations Related to Improving Health Insurance Marketplaces
paragraphs: Over the past 2 years, we have issued various reports highlighting  challenges that CMS and the states have faced in implementing and  operating health insurance marketplaces. For example, in an April 2013  report, we described the actions of seven states that were in various  stages of developing an information technology infrastructure to establish  marketplaces, including redesigning, upgrading, or replacing their  outdated Medicaid and CHIP eligibility and enrollment systems. Six of  the seven states were also building the IT infrastructure needed to  integrate systems and allow consumers to navigate among health  programs, but identified challenges with the complexity and magnitude of  the IT projects, time constraints, and guidance for developing their  systems.
In September 2014, we reported that while CMS had taken steps to  protect the security and privacy of data processed and maintained by the  systems that support Healthcare.gov, weaknesses remained in both the  processes used for managing information security and privacy as well as  the technical implementation of IT security controls. Specifically, we  noted that Healthcare.gov and the related systems had been deployed  despite incomplete security plans and privacy documentation, incomplete  security tests, and the lack of an alternate processing site to avoid major  service disruptions. Accordingly, we recommended that CMS implement  22 information security controls. We also recommended that the agency  improve its system security plans, privacy documentation, security tests,  and alternate processing site for the systems that support Healthcare.gov.  HHS concurred with all 22 of the recommendations to improve the  effectiveness of its information security control and fully or partially  concurred with our remaining information security program-related  recommendations. The department stated that it intends to take steps to  address the weaknesses, including updating its security plans, developing  required computer matching agreements, and developing a backup site  for Healthcare.gov.
In December 2014, we reported that all states using the federal  marketplace IT solution had faced challenges transferring applications to  and from that system. We pointed out that none of the states using the  federal marketplace IT solution in the first enrollment period were able to  implement application transfers, which required the establishment of two  IT connections: one connection to transfer applications found ineligible for  Medicaid coverage from the state Medicaid agency to the federal  marketplace IT solution, and another connection to transfer applications  found ineligible for coverage from the federally facilitated marketplace to  the state Medicaid agency.
Most recently, in March 2015, we reported that several problems with the  initial development and deployment of Healthcare.gov and its supporting  systems had led to consumers encountering widespread performance  issues when trying to create accounts and enroll in health plans. We  noted, for example, that CMS had not adequately conducted capacity  planning, adequately corrected software coding errors, or implemented all  planned functionality. In addition, the agency did not consistently apply  recognized best practices for system development, which contributed to  the problems with the initial launch of Healthcare.gov and its supporting  systems. In this regard, weaknesses existed in the application of  requirements, testing, and oversight practices. Further, we noted that  HHS had not provided adequate oversight of the Healthcare.gov initiative  through its Office of the Chief Information Officer.
We made recommendations aimed at improving requirements  management, system testing processes, and oversight of development  activities for systems supporting Healthcare.gov. HHS concurred with all  of our recommendations and subsequently took or planned steps to  address the weaknesses, including instituting a process to ensure  functional and technical requirements are approved, developing and  implementing a unified standard set of approved system testing  documents and policies, and providing oversight for Healthcare.gov and  its supporting systems through the department-wide investment review  board.
section_title: States Spent Federal Grant and Medicaid Funds to Establish Marketplace IT Systems, although Not All Marketplace IT Functions Are Fully Operational
paragraphs: States reported to CMS that they spent federal marketplace grant funds,  as well as Medicaid matching funds, on various IT projects to establish,  support, and connect to health insurance marketplaces. Specifically,  states reported spending about $1.45 billion in federal marketplace grant  funds from September 2010 through March 2015. The states also  reported spending federal funds designated for Medicaid eligibility and  enrollment systems on marketplace-related IT projects, although the  actual amount spent was uncertain, as only a selected number of states  reported on our survey that they tracked or estimated this information. In  this regard, from April 2011 through December 2014, states reported  spending $2.78 billion in combined federal and state Medicaid funds, a  portion of which was spent to support the marketplaces.
States that chose to establish state-based marketplaces were responsible  for the majority of the federal marketplace grant spending. These states’  efforts typically included developing web portals and supporting data  processing systems to carry out key marketplace-related functions, and  establishing electronic connections in order to exchange information with  various states, federal partners, and issuers.
Fourteen states with state-based marketplaces had developed and were  operating IT systems to support their marketplaces; however, not all  system functions were complete as of February 2015. In addition,  according to a CMS status report, as of November 2014, 7 of 37 states  using the federal marketplace IT solution could not transfer applications  for health insurance coverage between their state Medicaid systems and  the federal data services hub or had not completed testing or certification  of these functions. According to CMS officials, states operating IT  systems and states using the federal marketplace IT solution were  continuing to improve the development and operation of their  marketplaces in the second enrollment period.
subsections: section_title: States Spent Most Federal Marketplace Grant Funds to Develop Systems Supporting State-Based Marketplaces and Used Medicaid Funds to Connect to Marketplaces
paragraphs: States reported to CMS spending approximately $1.45 billion in federal  grant funds on IT projects to establish, support, and connect to health  insurance marketplaces from September 2010 to March 2015. States  that established state-based marketplaces, including state-based  marketplaces using the federal marketplace IT solution, reported having  spent approximately $1.37 billion of these funds. In addition, states with a  federally facilitated marketplace reported spending approximately $47  million, while those with a federally facilitated partnership reported  spending approximately $32 million.
Table 1 provides a summary of the states’ reported use of marketplace  grant funds for their IT projects as of March 2015.
In addition to the $1.45 billion of reported IT spending, approximately  $703 million of authorized grant funding for IT projects had not been  spent as of mid-March 2015. For additional details on the amount of  marketplace grant funding awarded and spent, see appendix II.
States with state-based marketplaces were authorized by CMS to spend  $2.02 billion for IT until December 2015, and this authorized amount per  state ranged from approximately $55 million to $325 million as of March  2015. As shown in figure 4, the reported spending of grant funds among  the 17 states that were approved to establish state-based marketplaces,  (i.e., the 14 state-based marketplaces and the 3 state-based  marketplaces using the federal marketplace IT solution), ranged from  approximately $29 million (in Minnesota) to approximately $254 million (in  California), as of March 2015.
Regarding states with a federally facilitated marketplace or federally  facilitated partnership, 19 of these states were authorized by CMS to  spend $378 million for IT, and this authorized amount per state ranged  from approximately $158,000 to $81 million as of March 2015. These  states reported marketplace grant IT spending that ranged from  approximately $30,000 (in Alabama) to approximately $20 million (in  Iowa), as of March 2015 (see fig. 5). The 15 other states that used these  two types of marketplaces were not authorized to spend grant funds for IT  projects. In June 2015, CCIIO officials told us that, with the exception of  Arkansas, Mississippi, and Utah, states with a federally facilitated  marketplace or federally facilitated partnership are no longer authorized to  spend marketplace grant funding for information technology because they  are no longer investing in the long-term creation of a modern eligibility  system to be shared between a state-based marketplace and the state  Medicaid program.
CMS required the states to report their grant spending for marketplace IT  projects in five broad budget categories: contracts, consultants,  personnel, equipment, and supplies. In this regard, the 17 states that  established state-based marketplaces, including state-based  marketplaces that used the federal marketplace IT solution, reported  spending the following approximate amounts in these categories, as of  March 2015: $1.13 billion on contracts, $76.18 million on consultants, $39.00 million on state personnel, $21.06 million on equipment, and $720,000 on supplies.
The largest part of these reported expenditures—nearly 89 percent—was  on contracts for services such as systems integration, project  management, and independent validation and verification.
In addition to costs in these five categories, CMS also asked the states to  report the amount of early innovator IT marketplace grant funding that  they had spent. In response, these states reported that they had spent  approximately $112.4 million of such funding.
The 34 states with a federally facilitated marketplace or federally  facilitated partnership reported spending, as of March 2015,  approximately $69.68 million on contracts, $2.19 million on consultants, $1.66 million on state personnel, $5.68 million on equipment, and $.03 million on supplies.
These states also reported spending $.06 million of early innovator IT  marketplace grant funding.
Table 2 shows marketplace grant spending for IT, by category, as of  March 2015.
During the course of our work, in October 2014, CMS began collecting  data on IT contract costs in new categories aimed to gather a greater  level of detail across states with state-based marketplaces. These new  reporting categories are system integration, project management,  independent verification and validation, middleware software, rules  engine software, and “other.”
As of May 2015, 11 state-based marketplaces had reported costs in some  of these new detailed cost categories. However, CMS’s documentation  indicated that not all states reported using all the new categories. For  example, not all states reported costs in the rules engine and middleware  software categories because those costs were included in the system  integration category or marked in the “other” category. Specifically, only  five states reported costs for developing rules engine software or  middleware software. According to CCIIO officials, CMS is following up  with states on missing amounts. Following through on these efforts to  collect more detailed information on states’ IT contract costs would  increase CMS’s insight into states’ IT spending.
subsections: section_title: States Spent an Undetermined Portion of Their Medicaid Funds on Marketplace IT Projects
paragraphs: States also spent Medicaid funds for marketplace-related IT projects,  such as modifying Medicaid eligibility and enrollment systems to interface  with the marketplaces. Specifically, states spent some portion of  approximately $2.78 billion in combined federal and state Medicaid  funding from April 2011 through December 2014 for marketplace-related  IT projects. Of this amount, $2.42 billion was from 90/10 funding and  $364 million was from 75/25 funding. An undetermined portion of this  spending was used to develop and maintain eligibility and enrollment  systems connections to the marketplaces.
States that established state-based marketplaces, including state-based  marketplaces using the federal marketplace IT solution, reported having  spent approximately $757 million of the 90/10 Medicaid funds for  Medicaid eligibility and enrollment systems. Further, states with a  federally facilitated marketplace reported spending approximately $1.32  billion of these funds, and those with federally facilitated partnerships  reported spending approximately $340 million. The amounts spent  included expenditures for marketplace-related IT projects.
Of the $364 million in 75/25 Medicaid funds, states that established state- based marketplaces, including state-based marketplaces using the  federal marketplace IT solution, reported having spent approximately $56  million. Those with a federally facilitated marketplace reported spending  approximately $285 million, and those with a federally facilitated  partnership reported spending approximately $23 million.
Table 3 provides a summary of states’ Medicaid 90/10 and 75/25  expenditures for Medicaid eligibility and enrollment systems by  marketplace type, as of December 2014.
While CMS required states to report the ratio of Medicaid funds to grant  funds in allocating their planned spending for marketplace-related IT  projects, the agency did not require states to track the actual amount of  Medicaid funds spent specifically on these IT projects. Thus, the total  portion of Medicaid funds spent for those purposes is unknown.
However, as part of our survey, 26 states were able to track or estimate  the portion of marketplace-related IT spending for Medicaid 90/10 funds,  and 17 states were able to track or estimate the portion of marketplace- related IT spending for Medicaid 75/25 funds. The states that tracked or  estimated their use of Medicaid funds reported spending approximately  $750 million of these funds—both 90/10 and 75/25 funds—for  marketplace-related IT projects through June 2014. The remaining  states in our survey did not track the amount or could not provide the  actual or estimated amount of Medicaid funds spent.
Based on the survey responses, states may have tracked or estimated  these amounts using a variety of approaches, thus state-reported data  may not be consistent across states. Table 4 shows the approximate  state-reported amounts of combined federal and state 90/10 and 75/25  Medicaid funding used for marketplace-related IT projects by marketplace  type.
Generally, the states used federal funds (both marketplace grant and  Medicaid matching funds) for various IT projects, including the  establishment and operation of their marketplaces and their connection to  the federal marketplace. Accordingly, the nature and extent of their efforts  varied depending on which marketplace type they chose to establish.
The 17 states that were approved to establish state-based marketplaces,  (i.e., the 14 state-based marketplaces and the 3 state-based  marketplaces using the federal marketplace IT solution) undertook  various IT projects to establish their marketplaces. These states generally  used the funds to develop their IT solutions, including the web portal for  individual consumer interaction (to set up user accounts, select health  plans, and apply for health coverage); systems to perform the key  marketplace functions (eligibility and enrollment, plan management,  financial management, and consumer assistance); functionality for  determining Medicaid and CHIP eligibility using new income standards;  functionality for sharing marketplace enrollment data with qualified health  plan issuers; and interfaces with federal systems through the federal data  services hub (needed to conduct eligibility verifications). In documents  provided to supplement the survey responses, states also reported using  their funds to cover numerous other expenses for state personnel,  systems integrator contracted services, interface development and  maintenance, independent verification and validation services, project  management, technical support, and software licenses.
Among the 34 states with a federally facilitated marketplace or federally  facilitated partnership, IT projects typically involved system development  to connect the states’ existing Medicaid systems to CMS’s federal data  services hub. In addition, 17 of these states reported on our survey that  they conducted projects to explore the option of developing IT systems to  support a state-based marketplace (even though they ultimately chose to  participate in the federally facilitated marketplace). For example, one  state reported to CMS that it used grant funds to develop technical  requirements and an architectural design, along with a request for  proposals to obtain a systems integrator for the implementation of a  marketplace. Another state using the federally facilitated marketplace was  awarded marketplace grant funds to support technology projects in  anticipation of becoming a state-based marketplace. According to CMS  officials, states that initially planned for, but did not pursue, a state-based  marketplace were required to return the funds to CMS or to re-budget the  funds for non-IT costs. In addition, two federally facilitated partnership  states used marketplace grant funds to develop new integrated Medicaid  eligibility and enrollment systems needed to support new requirements,  such as determining income eligibility for Medicaid using new income  standards.
section_title: States Are Continuing to Improve the Development and Operations of Their Marketplace Systems, but Not All IT Functions Are Complete
paragraphs: As of February 2015, the 14 states with state-based marketplaces had  developed and were operating systems to support their marketplaces;  however, not all IT functions were complete.
In particular, CMS reported that these 14 states’ marketplace systems  were performing some, but not all, key functions, including those related  to eligibility and enrollment, financial management, hub services, and IRS  reporting:
With regard to eligibility and enrollment functions, CMS status reports  indicated that eight state-based marketplace systems were fully  operational and operating without interruptions in service. The other  six state-based marketplace systems were partially operational,  meaning that these functions were operational but did not work as  intended and may have required manual processes to supplement  automated functionality. States with partially operational functions  used business process workarounds to complete eligibility and  enrollment functions, such as manually entering and verifying  individuals for healthcare coverage. For example, in one state,  applications to the state-based marketplace were sent by Medicaid as  portable document format (PDF) files and processed by data entry  specialists. In another state, data transferred from the marketplace to  Medicaid was automated, but other information was manually entered.
With regard to financial management functions such as collecting  premium payments, remitting payments to issuers, and payment  calculation for reinsurance, 4 state-based marketplace systems were  fully operational without interruptions in service and 8 state-based  marketplace systems were partially operational and may have  required manual workarounds. (These functions were not applicable  for 2 state-based marketplace systems that decided to rely on issuers  to conduct premium billing and processing functions.)
Although all states developing state IT solutions had received  approval from CMS to connect to the federal data hub, only 1 state- based marketplace state had fully completed development of hub  services functions such as verifying an individual’s identity and  citizenship and retrieving tax information for evaluating taxpayer  eligibility for insurance affordability programs. Thirteen state-based  marketplace states had partially completed hub services functions,  meaning that they had not yet implemented all hub services because  the testing or development had not been completed or independent  verification and validation attestation had not yet been received.
With regard to submissions to IRS regarding information such as  premium tax credits, 1 state had fully completed performance testing  of these functions, 10 states had partially completed performance  testing, and 2 states had not completed any performance testing of  these functions. Additionally, these functions were not applicable for  1 state, which used the federal IT system in the previous enrollment  period and was not responsible for IRS reporting.
The operational status of the state-based marketplace IT systems by  functional category, as of February 2015 is summarized in table 5.
Further, between the first and second enrollment periods, 6 of the 17  states with state-based marketplaces and state-based marketplaces  using the federal marketplace IT solution changed their IT solution. In  response to our survey, these states cited a variety of reasons for doing  so, such as significant flaws in the system, unsuccessful system roll out,  and non-working technology.
The primary IT development and operations changes, as reported by the  6 states to CMS, were the following:
Two states with state-based marketplaces, Oregon and Nevada,  stopped development on their marketplace IT solutions and decided  instead to use the federal marketplace IT solution (i.e., Healthcare.gov  and related systems) for eligibility and enrollment functions.
New Mexico had delays in developing and operating its marketplace  and used the federal marketplace IT solution as its platform for  eligibility and enrollment for the first enrollment period. For the second  open enrollment, the state continued to use the federal marketplace IT  solution for the eligibility and enrollment functionality and  subsequently decided to continue using the federal marketplace IT  solution indefinitely.
Maryland changed its IT solution to one that had been successfully  implemented in Connecticut for the second enrollment period.
Massachusetts replaced its existing system and implemented a  commercial-off-the-shelf technology solution for the second  enrollment period.
Idaho, which previously used the federal marketplace IT solution,  developed and operated its own marketplace IT solution for the  second enrollment period.
According to CMS documentation regarding marketplaces using the  federal marketplace IT solution, as of November 2014, 7 of 37 states  using the federal marketplace IT solution could not transfer applications  for health insurance coverage between their state Medicaid systems and  the federal data services hub or had not completed testing or certification  of these functions. Specifically, 3 of the states could not transfer—send  and receive—applications for health insurance coverage between the  state Medicaid and federal marketplace IT solution. The other 4 states  had not completed testing and certification of those functions. CMS  officials stated that the agency was continuing to work with the 7 states  that had not fully implemented these functions to ensure implementation  as soon as possible.
In addition, as of April 2015, the transfer of applications between state  Medicaid systems and the federal marketplace IT solution were not taking  place in real time, and according to a CMCS official, achieving this  capability is a goal for 2015 or 2016. For example, in one state, it took  about 15 minutes to send applications between state Medicaid systems  and the federal marketplace IT solution in either direction. In another  example, a state held on to applications received and sent them at the  end of the day. According to CMCS officials, states using the federal  marketplace IT solution continue to focus on completing their eligibility  system modernization, resolving defects, and making improvements to  systems so that business processes require less manual intervention.
section_title: CMS and States Established a Framework for Oversight, but CMS Oversight Was Not Always Effectively Executed
paragraphs: To address the requirements of PPACA and its implementing policies,  CMS engaged in various activities to oversee the states’ marketplace IT  projects. In particular, the agency assigned oversight roles and  responsibilities, put in place various reporting systems, and established a  series of reviews that were to help ensure that states’ systems were  adequately tested and functioning as intended. Nonetheless, even with  these steps, CMS did not clearly document, define, and communicate its  oversight roles and responsibilities to state officials, and it did not  consistently involve senior executives in the review and approval of  federal funding for states’ IT marketplace projects. In addition, CMS’s  reviews of the states’ progress were not always effective in ensuring that  systems and capabilities being developed to support the states’  marketplaces were fully tested before they became operational.
States that established and operated their own (state-based)  marketplaces generally used quasi-governmental entities to oversee their  marketplace IT projects; they also relied on various oversight  mechanisms, including executive steering committees, management  change control boards, and technical review boards. Meanwhile, states  with a federally facilitated marketplace or federally facilitated partnership  oversaw their IT projects through existing state agencies.
subsections: section_title: CMS Identified Oversight Roles and Responsibilities, but These Were Not Always Clearly Documented, Defined, or Communicated
paragraphs: To oversee states’ efforts in undertaking IT projects to support the  establishment and operation of their marketplaces, CMS identified  numerous internal offices and groups to which it had assigned roles and  responsibilities. As previously mentioned, three key offices—CCIIO, OTS,  and CMCS—were responsible for overseeing states’ efforts in  establishing the marketplaces. These three offices were to conduct  oversight activities, such as being involved in joint grant reviews,  Medicaid advanced planning document reviews, and IT gate reviews, to  ensure that states followed a standardized funding process.
Their primary roles and duties included the following:
CCIIO led the marketplace implementation, and within that office,  State Officers were assigned to be accountable for day-to-day  communications with the state marketplace officials. CCIIO officials  were also involved in grant funding decisions.
OTS was responsible for systems integration and software  development efforts to ensure that the functions of the marketplaces  were carried out. A primary participant within OTS was the IT project  manager, who was the individual responsible for monitoring, among  other things, state-based marketplaces’ IT development activities and  support for states that transitioned from one marketplace type to  another. OTS officials also provided technical reviews to State  Officers to inform grant funding decisions.
CMCS was the office responsible for coordinating and approving  Medicaid matching fund requests and implementation activities  related to the state health insurance marketplaces. The office carried  out these responsibilities in conjunction with CCIIO. CMCS officials  identified the enrollment and eligibility specialists as the primary  contacts within their office.
In addition, CMS established a group called the Cross Component  Committee to address marketplace-related issues across states. The  committee, which included members from OTS, CCIIO, and CMCS, was  tasked with overseeing the states’ progress to ensure that all marketplace  requirements were aligned with CMS policy. Major policy issues identified  through the committee were raised to business unit directors within the  agency.
CMS also informed us of other offices and groups within the agency that  had roles and responsibilities for overseeing states’ marketplace IT  projects. Based on written and oral descriptions of the various offices and  groups, as provided by CCIIO, CMCS, and OTS officials, we compiled the  information in table 6 to summarize CMS’s identified roles and  responsibilities for overseeing state marketplace IT projects.
In addition to establishing marketplace roles and responsibilities, CMS  identified various reporting systems that were to be used to assist federal  officials in overseeing state marketplace IT project funding and progress.  For example, the agency relied on state marketplace information that it  compiled in multiple computer systems to make funding decisions and  provide technical assistance to state officials. CMS also maintained or  utilized other systems that allowed states to apply for marketplace grant  funding online and to transfer funds to states to establish and operate  their marketplace. Additional systems allowed states to report to CMS  on their grant IT expenditures; upload documentation related to their  marketplace IT projects, such as project plans and testing and  requirements documents; and share best practices with each other.
Project management best practices emphasize the importance of clearly  documenting, defining, and communicating project roles and  responsibilities during the organizational planning process. During this  process, to make the most effective use of the people involved with a  project, best practices cite the importance of identifying, documenting,  and clearly assigning project roles, responsibilities, and reporting  relationships. Effective communication means that the information is  provided in the right format, at the right time, to the right audience, and  with the right impact. Adequate communications planning avoids  problems such as delays in message delivery, insufficient communication  to stakeholders, and misunderstanding or misinterpretation of the  message communicated.
According to best practices identified in the Project Management  Institute’s Guide to the Project Management Body of Knowledge, a key  document needed to ensure that communication is carried out effectively  is a communications management plan. The communications  management plan describes how project communications will be planned,  structured, monitored, and controlled in a comprehensive document,  including stakeholder communication requirements; the method of  updating and refining the communications management plan as the  project progresses and develops; and charts the information flow in the  project. Among other things, it should include persons or groups who are  responsible for communicating and receiving the information, the process  and associated time frames for escalating issues that cannot be resolved  at lower levels, and workflows that show the order of information  authorization. In addition, according the Project Management Institute’s  Guide to the Project Management Body of Knowledge, a communications  management plan is a comprehensive document that contains the entire  scope of the project and is updated regularly to reflect the current  communication and stakeholders.
However, while CMS established roles and responsibilities to help  oversee marketplace activities, the agency did not always clearly  document, define, and communicate marketplace IT project roles and  responsibilities to the states. Despite the complexity inherent in  overseeing marketplace IT project efforts across 50 states and the District  of Columbia, CMS did not have a comprehensive communication plan  that clearly documented and defined its state marketplace oversight  structure and all the associated roles and responsibilities of key  organizations and officials that were involved in state marketplace  oversight. Instead, the agency’s definition and communication of roles  and responsibilities were dispersed among various websites, operating  procedures, and other documents, such as those we used in developing  table 6. For example, roles for officials such as the CMS Administrator  and Principal Deputy Administrator were located on the agency’s website,  while other roles and responsibilities, such as those of the CCIIO State  Officers, were described in one of the agency’s standard operating  procedures. Additionally, CMS officials within CCIIO and CMCS stated  that some roles and responsibilities are embedded in memorandums of  agreement.
Further, while the agency had documented selected stakeholder  responsibilities in a matrix that CCIIO, OTS, and CMCS officials said  applied to state marketplace IT projects, this document only identified  responsibilities specifically associated with CMS’s development of the  Healthcare.gov web portal supporting the federally facilitated marketplace  and did not include all the personnel associated with oversight of the state  marketplaces. Specifically, it did not identify all stakeholders that would  be included in a more comprehensive communications plan developed for  the management of state marketplace IT projects, including the CCIIO  State Officers, the Marketplace Chief Executive Officer, and relevant state  officials.
The agency also provided a standard operating procedure for  marketplace communications and technical assistance that contained  selected CMS roles and instructions for providing technical assistance to  states. However, the procedure was identified as a draft document from  January 2013, and was limited to addressing technical assistance, which  did not represent the full range of stakeholder and IT oversight activities.  For example, the document did not identify all groups that are to receive  pertinent information, a process identifying time frames and the  management chain for escalating the communication of information, or  workflows for issuing and disseminating guidance to states.
Further, officials within CCIIO, CMCS, and OTS did not recognize certain  organizations as having a role in marketplace IT activities, even though  they should have done so. For example, while the officials told us that the  Office of Communications does not have a role in states’ marketplace IT  oversight, this office is identified as a member in the charters of key  committees and boards responsible for state marketplace IT project  oversight, including the Cross Component Committee, Marketplace  Oversight Board, and Health Reform Operations Board.
In discussing this matter, CCIIO and CMCS officials acknowledged that  they had not created a comprehensive communication plan containing all  relevant oversight roles and responsibilities. According to these officials,  certain roles and responsibilities were not defined and documented  because they were considered to be general public knowledge for which  no detailed documentation was necessary. They added that, in the  absence of a specific document or process, states were informed of who  their points of contact were by e-mail or weekly calls. Further, these  officials noted that all communications to the states were routed through  the CCIIO State Officers, thus replacing the need for a comprehensive  communications management plan.
As previously described, CMS provided oversight and technical  assistance to states in establishing their marketplaces. In responding to  our survey, states with a state-based marketplace, including those using  the federal marketplace IT solution, provided generally positive ratings of  the clarity, completeness, and timeliness of CMS’s communication, while  federally facilitated states, including federally facilitated partnerships,  provided a higher rate of dissatisfaction. Similarly, state-based  marketplace states provided generally positive ratings of the clarity,  completeness, and timeliness of CMS’s guidance, while federally  facilitated states provided a higher rate of dissatisfaction.
While states with all marketplace types reported in our survey being  generally satisfied with the level of CMS oversight and assistance,  several states identified instances of delayed or insufficient  communications with CMS. Specifically, of the 36 states that responded  to our survey question regarding CMS’s overall oversight and assistance,  25 states rated it as just right, 4 rated it as more than enough, and 7 rated  it as less than enough. Further, of the 17 states that provided comments,  5 spoke positively about CMS’s support and 1 spoke positively about the  completeness and timeliness of CMS guidance.
The remaining 11 states provided both mixed and negative comments  regarding the completeness and timeliness of CMS guidance that  included roles and responsibilities. For example, these states noted that  they generally had experienced some type of delay in message delivery  from CMS, insufficient communication with the stakeholders, and  misunderstandings or misinterpretations of the messages communicated.  For example, these states generally reported that they lacked complete  and timely policy and business guidance from CMS, which impacted their  IT development deadlines, created rework, and necessitated moving  forward to develop solutions without knowing if the agency would approve  or disapprove of their marketplace solutions.
Overall, responses to our survey questions indicate that CMS may not  have always provided the level of consistent and comprehensive  communication of roles and responsibilities that is necessary to support  states in effectively establishing and operating their marketplace systems.  Having a comprehensive communications management plan that  identifies and conveys the roles and responsibilities of key organizations  and officials could be a valuable resource as states move forward on any  further marketplace IT efforts.
section_title: Federal Funding Decisions for State Marketplace IT Projects Did Not Always Include Senior-Executive- Level Oversight
paragraphs: To oversee its own IT projects, such as the development of  Healthcare.gov and related systems, CMS created a process called the  eXpedited Lifecycle Process. This process required reviews and  approvals by senior-level CMS executives, generally the Director or  Deputy Director of the agency’s IT unit—OTS—and business units,  including CCIIO, CMCS, and OAGM. According to the agency’s guidance,  these senior-level executives should be individuals who have the  authority to speak for, vote for, and otherwise make commitments on  behalf of their business units. This approach is consistent with best  practices in GAO’s IT investment management framework, which  emphasizes the importance of having senior executive-level decision  makers, such as the heads of IT and business units, involved in  investment decisions. Such involvement by senior executives provides  accountability for investment decisions and helps ensure that these  decisions are consistent and reflect the goals of the agency.
Similar to the eXpedited Lifecycle Process, CMS created its  Establishment Review process, which states were required to comply with  (as part of their cooperative agreements with CMS) in order to receive  marketplace grant funding. The Establishment Review process is a  structured grant monitoring approach that consists of multiple technical  reviews for assessing the state’s progress and associated IT project  documentation. States must obtain CMS approval to access restricted IT  grant funds by passing technical review gates associated with the  planning, design, and implementation of their projects.
However, unlike the eXpedited Lifecycle Process that CMS uses to  manage its own investments at the federal level, the Establishment  Review process did not include representation from all relevant senior  executives in the agency to review and approve the planned marketplace  IT projects prior to releasing federal funding to the states. Specifically,  CMS’s standard operating procedure for State Officers identified the IT  and business units involved in the Establishment Review process, which  included CCIIO, CMCS, OTS, and OAGM, among others. However, with  the exception of the Director of CCIIO, it did not clearly require  involvement by the heads of the other IT and business units involved in  this process. For example:
CMS did not demonstrate that senior-level executives from all relevant  business and IT units were involved in the initial approval of grant  awards. According to the operating procedure and officials from these  business and IT units, the agency’s Objective Review Committee was  tasked with reviewing state applications for federal marketplace  grants. This committee consisted of subject matter experts from both  inside and outside the federal government who scored applications  during a review in which the State Officer participated to answer  questions. The State Officer then prepared federal marketplace grant  funding recommendations to OAGM and the Deputy Director of the  State Exchange Group within CCIIO, who made the final decision on  grant awards. However, it was unclear who these subject matter  experts were or whether there were executives at the appropriate  level involved with these decisions.
CMS did not provide evidence that senior executives from all relevant  business and IT units were involved in approving the release of  restricted IT funds from marketplace grants as states progressed with  their projects. According to CMS’s standard operating procedure and  officials in CCIIO and OAGM, decisions to release restricted state IT  funding were made by the Deputy Director of the State Exchange  Group within CCIIO and OAGM grant management officers, who were  responsible for reviewing and providing guidance on grant services for  state marketplaces. These decisions were based on input from CCIIO  State Officers, who served as primary points of contact to assigned  states, and IT project managers in OTS, who were responsible for  monitoring state-based marketplaces’ IT development activities.  However, these officials did not hold executive-level positions.
CMS did not provide evidence of executive-level involvement in the  approval of Medicaid funds for marketplace IT projects. CMCS  officials stated that they followed CMS’s Establishment Review  process in order for states to receive Medicaid matching funds and  that the approval of these funds was a coordinated effort between  CCIIO and CMCS. However, they did not identify the specific officials  responsible for approving these funds or provide evidence to show the  approval process included senior executives from CMCS, CCIIO, and  other relevant business units.
CCIIO, CMCS, and OTS officials told us that they believed their  Establishment Review process included the appropriate officials to review  and approve state requests for federal funding. These officials added that  they used their existing organizational structure to oversee decisions  regarding marketplace grants and Medicaid funds.
However, without the involvement of senior executives from all relevant IT  units, such as OTS and business units such as CCIIO and CMCS to  review and approve all federal funds invested in the state marketplace IT  projects, CMS has less assurance that decisions are being coordinated  among officials with a perspective across their respective business units  and the agency as a whole. By ensuring such executive involvement,  CMS would increase accountability for decisions to fund states’ IT  projects and better ensure these decisions are well informed and make  efficient use of federal funds.
section_title: CMS Reviews of State Marketplace IT Projects Did Not Fully Ensure State Systems Were Ready for Operation
paragraphs: As part of its marketplace oversight, CMS established a process to review  states’ progress on related IT projects. This framework, called the  Enterprise Life Cycle, requires states to provide CMS specific artifacts  supporting their projects, such as the concept of operations, system test  documents, and project plans, among others. The framework focuses on  incremental reviews of the projects at distinct stages, or “gates.” For each  review, states are expected to show CMS an acceptable level of progress  and maturity in their projects’ development before proceeding to the next  project phase. Table 7 describes the various Enterprise Life Cycle gate  reviews.
These reviews were important because they were intended to  demonstrate that the state marketplaces were ready to go live. In  particular, during the operational readiness reviews, states establishing  state-based marketplaces were required to demonstrate that they had  met requirements, such as concluding all system testing, before the IT  projects could proceed from development to operations. The Enterprise  Life Cycle guidance defines this review as the agency’s determination  that the state marketplace is ready to go into production. Based on these  operational readiness reviews, CMS was to either approve the state’s  system for operation or grant a conditional approval to proceed if the  system was substantially compliant with the requirements of the review.
However, the operational readiness reviews did not always meet the  agency’s stated goal to ensure that states’ marketplace systems were  ready for production. For the first enrollment period, CMS conducted  operational readiness reviews of 15 state-based marketplaces in August  and September 2013. However, CMS conditionally passed all of those  states without fully ensuring that they had conducted all required system  testing and demonstrated that their systems were ready for production as  called for in its Enterprise Life Cycle guidance. For example, CMS  documentation from these operational readiness reviews showed the  following:
Maryland demonstrated several eligibility and enrollment functions.
However, the state had only completed approximately half of the  planned user acceptance testing and had over 100 outstanding high- priority defects. In addition, almost 500 total defects had yet to be  resolved.
Nevada also demonstrated several eligibility and enrollment functions.
However, the state had not submitted test reports for all end-to-end  system testing, and user acceptance testing was in progress. The  report identified 42 critical or major defects that needed to be  addressed.
Massachusetts demonstrated several eligibility and enrollment  functions. However, the state had not completed testing and reported  1,170 open defects.
Nonetheless, all state-based marketplace systems were conditionally  approved and went live on October 1, 2013. Consumers in many states  subsequently experienced widespread problems when using these IT  solutions to apply for health insurance coverage during the first  enrollment period, and in four states these problems were so severe that  the states switched to a different solution.
According to CMS officials, these four states implemented new  marketplace IT solutions or used the federal marketplace IT solution in  the second open enrollment period and successfully conducted  enrollment even if some states had to create manual workarounds.  However, according to CMS documentation, as of November 2014, eight  states continued to have outstanding follow-up items from their  operational readiness reviews that had not been addressed. In May 2015,  officials in CCIIO, CMCS, and OTS stated they were actively working with  these states to complete their outstanding open items.
CCIIO officials further noted that the Enterprise Life Cycle gate reviews  were not intended to be “pass or fail,” but to set the appropriate level of  expectations for the status and progress of marketplace development and  implementation and to identify areas where states may require  assistance. In addition, CCIIO officials stated that, if all the milestones  were not met during the gate review, they planned to conduct more  frequent follow-up to improve the state’s position. They also said that  although the IT component did not work for certain states, the agency  granted conditional approvals because the states were able to build  workarounds and put manual processes in place to allow individuals to  submit applications and enroll in health coverage. Officials in OTS added  that, although they made suggestions for improvements, states could  choose whether or not to implement CMS’s recommendations.
However, when CMS granted states conditional approval to go live, they  did not ensure states’ systems had been fully tested, which is part of the  structured and disciplined approach to oversight that is outlined in the  agency’s Enterprise Life Cycle. By not ensuing that systems were  completely tested, the agency lacked assurance that the states’  marketplace IT systems would performed as intended which, in some  cases, resulted in applicants facing long waits for eligibility  determinations, websites freezing midway through the process of  applying for coverage, and systems being taken offline for days at a time,  forcing applicants to enroll manually.
section_title: States’ Oversight Roles Varied Depending on Marketplace Type
paragraphs: The extent and manner of oversight that states exercised over  marketplace IT projects depended in large part on the type of  marketplace they chose to establish. For state-based marketplaces, state  officials were responsible for overseeing various IT activities associated  with the development and operations of their marketplaces. Specifically,  states were required to oversee the planning involved with becoming a  state-based marketplace. Thus, among other things, state officials were  responsible for ensuring that key functionality requirements in areas such  as eligibility and enrollment, plan management, consumer assistance,  and financial management, were included in the development of the  marketplace.
Additionally, these states were responsible for overseeing contractors,  who carried out various marketplace IT project-related activities, such as  system integration, platform builds, project management, independent  verification and validation, and security assessments. State officials were  to follow CMS policy and guidance when establishing the marketplaces,  including preparing project artifact deliverables, such as the marketplace  concepts of operation, system test documents, and project plans. They  also were to comply with financial and performance reporting  requirements of CMS’s Enterprise Life Cycle process.
To oversee their marketplaces, 13 of 17 states with state-based  marketplaces, including those using the federal marketplace IT solution,  reported on our survey that they established “quasi-governmental”  entities, which were created by state legislation to oversee marketplace  activities and interface with CMS to fulfill the state’s marketplace  responsibilities. These entities are governed by a board made up of  representatives from consumer groups and health insurance issuers,  since CMS policy requires a balance of consumer and business interests  on the board. The board is responsible for governance of the  marketplace, making key marketplace decisions, and holding regularly  scheduled meetings.
By contrast, 4 of these 17 states reported on our survey that they chose  to operate their marketplace through an existing state agency, such as a  state department of health or Medicaid agency. If a state-based  marketplace was housed within an existing state agency, then that  marketplace was typically led by directors or an advisory board, and the  leadership team typically reported to the governor’s office.
States with state-based marketplaces, including those using the federal  marketplace IT solution, reported on our survey that they also established  various committees and boards to assist state officials in overseeing the  marketplace’s IT funding and progress. These oversight committees and  boards included steering committees, management change control  boards, and technical review boards, among others.
Steering committees: All states with state-based marketplaces had  established this type of committee. A steering committee is to provide  leadership, direction, and support for IT projects. For example, one  state’s steering committee was reported to be made up of senior  leadership from various agencies within the state and was responsible  for ensuring that marketplace IT goals aligned with various state  agencies’ goals. In addition, the committee served as a forum for  project strategy development and operations, policy, and technology  recommendations to its board of directors.
Management change control boards: Thirteen of the 17 states with  state-based marketplaces established this type of board. A  management change control board is to oversee a project’s scope  and requirements. For example, one state reported that its  management change control board was chaired by its project director  and oversaw not only changes to the scope and requirements, but  also its marketplace project schedule, costs, and deliverables.
Technical review boards: Nine of the 17 states with state-based  marketplaces established this type of board. A technical review board  provides technical findings and recommendations to project  stakeholders. For example, one state reported that its technology  committee provided leadership and helped to analyze the impact of  the marketplace on existing IT standards and informed other teams  and stakeholders about policy changes that could impact the project.
In addition, 7 of the 17 states with state-based marketplaces, including  those using the federal marketplace IT solution, reported on our survey  that they used additional oversight mechanisms beyond these three.  Specifically, one state reported that its marketplace and state  administration established an integrated project management office to  assist with coordination of Medicaid and tax credit applications and  eligibility functions. Another state reported using a cross-agency group  made up of agencies involved in marketplace eligibility functions from  both IT and policy perspectives.
Further, all states relying on the federally facilitated marketplace and  federally facilitated partnerships that responded to our survey indicated  that they used existing state agencies to oversee implementation of their  marketplace IT projects. Existing state agencies included state  departments of health or Medicaid agencies, which coordinated directly  with CMS. In addition, these states’ officials oversaw the contractors who  were responsible for various marketplace-related activities, such as  building interfaces to connect the state systems to the federal data  services hub for transferring information between the federally facilitated  marketplace and state Medicaid programs.
section_title: States Encountered Challenges and Identified Lessons Learned and Best Practices in Managing, Overseeing, Developing, and Operating Marketplace IT Systems
paragraphs: States encountered various challenges in their efforts to design, develop,  and implement marketplace IT systems. States with state-based  marketplaces reported experiencing challenges in each of five areas  identified in our survey: project management and oversight, marketplace  IT solution design, marketplace IT solution development, resource  allocation and distribution, and marketplace implementation and  operation. In addition, states with a federally facilitated marketplace  reported facing challenges in two areas identified in the survey: project  management and oversight and system design and development.
While states operating both state-based and federally facilitated  marketplace IT solutions reported in the survey that they faced similar  issues, various challenges were more common for states developing their  own IT solution because the scope of their efforts was larger than that of  states with a federally facilitated marketplace. For example, those with  state-based marketplaces generally reported experiencing issues with  marketplace eligibility and enrollment functions; while for states with a  federally facilitated marketplace, those functions were performed by CMS.
To varying extents, states identified lessons learned and best practices  from their experiences with and efforts to address the challenges. CMS  was aware of state challenges and took various actions to provide  technical assistance. It also has taken steps to facilitate the sharing of the  lessons learned and related best practices, which will continue to be  important as states work to complete the remaining functions for their  marketplace systems.
subsections: section_title: Project Management and Oversight Challenges Include Compressed Time Frames and Project Governance, Oversight, and Decision Making
paragraphs: Compressed time frames was rated as the greatest challenge by  officials of both states with a state-based marketplace and states with a  federally facilitated marketplace. Specifically, 13 of 17 states with state- based marketplaces and 20 of 30 states with a federally facilitated  marketplace considered compressed time frames a great or very great  challenge, and it was also reported as a factor driving other challenges.  State officials noted that their IT project schedules were constrained by  the need to deliver functionality in time for the first enrollment period  beginning on October 1, 2013. For example, one state-based  marketplace official reported that compressed time frames affected the  state’s development and testing time, which impacted all phases of  testing (system, integration, performance, and user acceptance).
Project governance, oversight, and decision making was also rated as  one of the greatest challenges in the project management and oversight  area by officials of both states with a state-based marketplace and states  with a federally facilitated marketplace. Specifically, 10 of 17 states with  state-based marketplaces and 8 of 30 states with a federally facilitated  marketplace rated project governance, oversight, and decision making as  a great or very great challenge.
Based on our analysis of narrative survey responses, 14 states with state- based marketplaces and 15 states with a federally facilitated marketplace  also identified lessons learned or best practices in the area of project  management and oversight. For example, regarding compressed time  frames, a best practice identified by 1 state was to double the amount of  lead time normally expected when planning for implementation of  complex IT projects. Another state reported a lesson learned regarding  compressed time frames, which was related to IT systems design and  development. This state learned that taking a two-phased approach  whereby the state modified its legacy Medicaid eligibility system first, and  then proceeded with a full-scale system upgrade, helped meet deadlines  while avoiding significant problems that had arisen in other states.
States also reported lessons learned and best practices related to project  governance, oversight, and decision making. For example, one state  reported reshaping its project management team and, thus, making  progress for the second open enrollment season. A second state realized  too late that it needed more governance and a dedicated program  management office. This state’s officials also said that it was important to  recognize that the marketplace is an IT project as well as an insurance  project, and that it was critical to have a proper mix of both sides to  ensure success.
section_title: State-Based Marketplace IT Solution Design and Development Challenges Include Interfacing with Insurers and Developing Website Eligibility Functions
paragraphs: Developing interfaces and interoperability with insurers was rated as one  of the greatest challenges by 9 of 17 states with state-based  marketplaces. For example, 1 state reported challenges with a system  that was supposed to allow users to pay for and enroll in insurance plans;  however, that basic feature was not appropriately developed by launch or  for months afterward. The state hired a contractor to reconcile enrollment  and premium tax credit issues between its insurance carriers and its IT  solution, but all issues were not resolved, and the state was still working  through this process when officials responded to our survey.
Developing state marketplace website eligibility functions for both state  Medicaid and Qualified Health Plans was also rated as one of the  greatest challenges by 9 of the 17 states. For example, one state official  reported that their applicants could not have their eligibility determined for  Qualified Health Plans, Medicaid, and premium tax credits without the  assistance of specially trained customer service representatives or  community partners and agents. Another state’s original IT solution was  not working appropriately, so officials approached CMS, who offered to let  the state use the Healthcare.gov platform for eligibility and enrollment. A  third state cited numerous multi-stage workarounds to circumvent defects  in eligibility and enrollment functionality. This included, for example, 100  percent manual validation of all enrollment files.
Although 8 states with state-based marketplaces identified lessons  learned or best practices in the marketplace IT solution design and  development area, with one exception, states did not specifically identify  lessons learned related to developing interfaces and interoperability with  insurers or developing state marketplace website eligibility functions. One  state reported that it learned that projects like this should begin with  simple rules on eligibility, and then add complexity. Further, this state  decided to maintain Medicaid and CHIP enrollees in its legacy system  using a close approximation of eligibility rules to ensure that there was no  disruption in coverage with the launch of a new system. New applications  for Medicaid and CHIP were determined in the new system while  renewals for current enrollees were determined in the legacy system. This  was to enable more time for adequate testing and further development of  Medicaid and CHIP rules in the new system.
section_title: System Design and Development Challenges for States Using the Federally Facilitated Marketplace Include Systems Integration Testing and Changes to Requirements
paragraphs: Conducting systems integration testing was rated as one of the greatest  challenges by 12 of 30 states with a federally facilitated marketplace. For  example, 1 state reported that limited development and testing time  affected all phases of testing including system, integration, performance,  and user acceptance testing. Another state reported that the interface  between the state and the federally facilitated marketplace was delayed  due to implementation delays in the federal marketplace IT solution.  These delays resulted in last-minute changes to the federal systems, both  known (but communicated late) and unknown. Each federal system  change required the state to also change, and such changes and delays  resulted in the state missing deadlines. Other states specifically cited a  lack of end-to-end testing between the federal IT systems and states, as  well as integrating and testing with the federal marketplace and the  federal data services hub, as challenges.
Changes to requirements was rated as one of the greatest challenges by  19 of 30 states with a federally facilitated marketplace. For example, one  state official said that “the aggressive time frame made an impact to the  design. Systems always evolve, but the aggressive schedule forced  design trade-offs along the way.” A second state reported that the  compressed time frame caused CMS to continually define requirements  throughout implementation and into operations, resulting in the  reprogramming of multiple design changes. Lastly, another state official  commented on multiple challenges related to changes in requirements.  This state official said that changes and delays due to clarification of CMS  requirements in areas such as use of the federal data services hub and  identity proofing caused significant rework and some critical functionality  to be deferred, which, because of the aggressive time frame, impacted  operations.
A second state official emphasized developing a comprehensive set of  requirements. The state invested time to develop a comprehensive set of  requirements for all known areas of the system and included broad  requirements referencing CMS guidance documents when detail from  CMS was insufficient. The state then required vendors to explicitly identify  which requirements would be met with delivered functionality, and which  requirements would need to be augmented with customizations or  additional software applications. This kept most of the systems  development in scope and resulted in less than a 10 percent increase in  the negotiated fixed price due to change orders. A third state identified a  best practice regarding guidance and policy—which drive requirements— noting that they should be finalized before states are tasked with  implementing system changes and testing.
Our analysis of narrative survey responses showed that 14 states with a  federally facilitated marketplace reported lessons learned or best  practices related to IT systems design and development, including those  associated with changes to requirements or the development of  requirements. For example, one state official said that there were many  changes leading all the way up to open enrollment. Only after this  occurred did officials recognize that they needed to lock down the scope  of work and disallow “nice-to-haves” to focus on critical functionality.
section_title: Resource Allocation and Distribution Challenges Include Adequate Staff and Funding
paragraphs: Adequate number of staff was rated as one of the greatest challenges by  9 of 17 states with state-based marketplaces. In one case, a state official  reported that the state had only approved the hiring of approximately one- third of the staff it requested and, as of October 2014, had never hired a  certified project manager to oversee their state’s marketplace-related IT  projects. Similarly, staffing limitations forced another state to ask its staff  to work overtime, in some cases more than 60 hours a week for months  on end, in order to complete the work required prior to open enrollment,  resulting in burnout and the loss of key staff soon after the start of the first  open enrollment period.
Adequate funding to sustain a state’s marketplace system was rated as  one of the greatest challenges by 6 of 17 states with state-based  marketplaces. For example, one state official reported that, in order to  meet open enrollment deadlines and reduce schedule risks, the state  decided to use a commercial off-the-shelf product instead of open-source  products, which led to an increase in life-cycle costs.
Our analysis of narrative survey responses found that five states reported  lessons learned or best practices related to resource allocation and  distribution. For one state, the most significant lesson learned was the  amount of testing resources required for all associated types of IT testing.
Due to this, the state has identified a need for additional business analyst  positions and subject matter expert knowledge.
section_title: Marketplace Implementation and Operation Challenges Include Call Center Operations and System Performance
paragraphs: Call center operations was rated as one of the greatest challenges by 9 of  the 17 states with state-based marketplaces. For example, one state  official reported that due to challenges with system performance, their call  center experienced high-traffic volume, and this affected the average time  to handle a call, abandonment rates of calls, and operations. Another  state reported that insufficient time for staff training led to inefficiencies in  call center operations.
System performance was rated as one of the greatest challenges by 7 of  the 17 states with state-based marketplaces. For example, 1 state cited  significant challenges in implementation and operation because its  software did not work as advertised. Also, as mentioned above, system  performance problems affected call center operations. This was  compounded in part because of the surge in users attempting to use the  online marketplace that occurred in the period immediately after going  live.
Our analysis of narrative survey responses showed that two states with  state-based marketplaces identified best practices or lessons learned  related to the operation and implementation of marketplace-related IT  systems. For example, one state cited the importance of contingency  planning that enabled state deployment of additional system capacity  when volume exceeded expectations. Another state reported that the  inability to develop and refine marketplace technology resulted in  significant operational costs, which could have been avoided with a less  aggressive time frame.
Figure 6 summarizes the challenges in each of the five areas rated by  states with state-based marketplaces. Figure 7 depicts the challenges  that states with a federally facilitated marketplace rated in each of their  two respective areas.
section_title: CMS Responded to Challenges and Facilitated the Sharing of Lessons Learned and Best Practices
paragraphs: CMS was aware of states’ challenges and responded to them by  engaging in various outreach to and communication efforts with the  states. According to CCIIO officials, once an issue or challenge was  identified, CMS responded in a number of ways. Specifically, according to  these officials, the agency provided technical assistance that included  discussions with CMS subject matter experts to ensure that appropriate  information and resources were available to address challenges. For  example, the officials said they conducted site visits with state  marketplace officials during which they discussed management and other  issues and made recommendations for improvement, as needed. Other  state challenges that CMS officials indicated they were aware of included  issues with compressed schedules, state governance, legislative  requirements, vendor management, personnel and resources, and call- center operations.
Additionally, CMS made efforts to both directly share and facilitate the  sharing of identified lessons learned and best practices among the states.  CCIIO officials reported that lessons learned and best practices were  shared through various methods such as discussion forums, including bi- weekly forum meetings with senior state officials, conference calls, and  weekly newsletters distributed to grantees, and through various reporting  and document sharing systems maintained by CMS.
In taking steps to respond to state challenges, identify lessons learned,  and share best practices with states, CMS performs an essential role of  advising state officials and others involved with health insurance  marketplace IT projects. It will be important for CMS to continue doing so  as states work to complete the remaining functions for their marketplace  systems.
section_title: Conclusions
paragraphs: States spent approximately $1.45 billion in federal marketplace grant  funds to help establish IT systems supporting their health insurance  marketplaces, as well as a portion of Medicaid funds. As of the second  enrollment period, states had largely established these systems, although  some of their functions remain to be implemented.
While CMS was tasked with overseeing states’ development of their  marketplace IT systems, limitations in CMS’s efforts resulted in oversight  that was not always effectively executed. Specifically, because roles and  responsibilities were not always clearly defined, documented or  communicated, as recommended by leading practices for project  management, a number of states faced hurdles in communicating with  stakeholders and receiving timely CMS guidance. In addition, although  called for by leading practices in investment management, relevant senior  executives in the agency were not always involved in overseeing  decisions to fund states’ marketplace IT projects, resulting in less  accountability for such decisions. Further, because CMS’s reviews of  state IT projects did not ensure state systems were fully tested as called  for in CMS’s guidance, systems were put into place that, in some cases,  did not perform as intended. States also had a key oversight role, which  varied depending on the type of marketplace.
Finally, states reported a number of challenges and lessons learned in  establishing their marketplaces, with state-based marketplaces  encountering some unique challenges. CMS has taken various actions to  facilitate the sharing of these challenges and lessons learned, as well as  best practices among the states, and it will be important for CMS to  continue these efforts as states work to complete the remaining functions  for their marketplace systems.
section_title: Recommendations for Executive Action
paragraphs: To improve the oversight of states’ marketplace IT projects, we  recommend that the Secretary of Health and Human Services direct the  Administrator of the Centers for Medicare & Medicaid Services to take the  following three actions:  clearly document, define, and communicate to all state marketplace  officials and stakeholders the roles and responsibilities of those CMS  officials involved in overseeing state marketplaces in a  comprehensive communication management plan; ensure that all CMS senior executives from IT and business units who  are involved in the establishment of state marketplace IT projects  review and approve funding decisions for these projects; and ensure that states have completed all testing of marketplace system  functions prior to releasing them into operation.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report, signed by HHS’s  Assistant Secretary for Legislation. In the comments (reprinted in  appendix III), the department stated that it concurred with all three of our  recommendations. The department added that it had taken various  actions that were focused on improving its oversight and accountability for  states’ marketplace efforts.
While the actions discussed are important, the department did not always  identify specific activities being taken or planned that would address the  full extent of the recommendations. Specifically, with respect to our  recommendation that CMS clearly document, define, and communicate  its roles and responsibilities for overseeing state marketplaces in a  comprehensive communication management plan, the department noted  that a State Officer is assigned to each state to serve as the primary point  of contact and that CMS’s roles and responsibilities are communicated  through this official. The department also stated that these roles and  responsibilities are documented in several resources, including standard  operating procedures and weekly newsletters to state officials. However,  the department did not indicate that CMS would develop a  communications management plan to provide a comprehensive and  consistent means of identifying and conveying the roles and  responsibilities of key CMS organizations to all states and the District of  Columbia. As we noted in our report, CMS’s standard operating  procedures and other documents did not identify all the relevant  stakeholders or activities involved in its oversight process. Thus, we  maintain that a comprehensive communications management plan would  be a valuable resource as states move forward on any further  marketplace IT efforts.
With respect to our recommendation that CMS include senior executives  from all relevant IT and business units in funding decisions for state  marketplace IT projects, HHS stated that the department already includes  senior executives in its funding decisions for these projects. However, as  noted in our report, CMS did not provide evidence that key senior  executives from CCIIO, CMCS, and OTS were involved in various funding  decisions associated with the states’ IT projects. For example, CMS did  not demonstrate that senior-level executives from all relevant business  and IT units were involved in the initial approval of grant awards or the  release of restricted IT funds from marketplace grants as states  progressed with their projects. In addition, CMS did not provide evidence  of senior executive involvement in the approval of Medicaid funds for  marketplace IT projects. By ensuring such executive involvement, CMS  would increase accountability for decisions to fund states’ IT projects and  ensure that these decisions are well informed in order to make efficient  use of federal funds.
With respect to our recommendation to ensure that states have  completed all testing of marketplace system functions prior to releasing  them into operation, HHS noted that it will continue to follow its guidelines  to determine if state marketplace system functions are ready for release.
The department added that it will continue to work closely with state- based marketplaces to improve their systems and verify that system  requirements are met. We agree that following its review guidance as  defined is important. In particular, as noted in our recommendation, CMS  should ensure that states’ systems are fully tested before approving them  for release into production, rather than relying on workarounds and  manual processes.
HHS also provided technical comments, which we incorporated in the  report as appropriate. Among these comments, the CMS liaison in the  Office of Legislation sent an e-mail on September 10, 2015, stating that  the amount of total marketplace grant spending for the District of  Columbia that CMS provided to us based on its March 2015 report was  incorrect. Accordingly, we revised our analysis and relevant areas of our  report to reflect the new amount provided by the agency.
We also provided relevant excerpts of this report to each of the 50 states  and the District of Columbia and received responses, via e-mail or in  writing, from officials in 15 states. Officials from 5 of these states (Alaska,  Arizona, Maine, Nevada, and Rhode Island) said they had no comments.
Among the remaining 10 states, 6 states (Alabama, Idaho, Indiana,  Minnesota, Washington, and Wisconsin) commented on our discussion of  their marketplace grant data. According to these states, the data we  reported on marketplace grant funding were not always consistent with  their own data. However, the grant funding discussed in our report  reflects state-reported data that CMS provided and represents a  consistent source and time frame of data for all states as of March 2015;  thus, we did not revise our discussion of the reported data in the report.  However, we did revise the report to clarify that the state-reported data  that CMS provided could lag behind actual state marketplace grant data  for a specific date.
In addition, officials from 6 of the 10 states commented on the status of  their systems development and operation.
In e-mail comments, the Grant Compliance Officer of Covered  California provided details on specific functionality Covered California  was still implementing. For example, its small business marketplace  was using manual workarounds for its automated payment  functionality until the system is completed. Regarding the hub  services and IRS reporting submission functions, the official said that  California will continue to enhance and improve efficiencies of the hub  services for the health insurance renewal process, and will complete  performance testing of IRS reporting submissions.
In written comments, the Executive Director of the District of Columbia  Health Benefit Exchange Authority did not agree with some of the  characterizations in our report. Specifically, the Executive Director  concurred with our characterization of the status of the financial  management functions as fully complete and IRS reporting functions  as partially complete, but did not agree that the District of Columbia’s  eligibility and enrollment and hub services functions were only partially  complete. Regarding the eligibility and enrollment functions, the  Executive Director said that our characterization was misleading and  unsupported because these functions were only partially operational  for one specific function and that the marketplace received permission  from CMS to implement an alternate method for implementing another  specific function; thus, the overall eligibility and enrollment function  should have been considered fully operational.
Our characterization of eligibility and enrollment functions as partially  operational was based on CMS’s February 2015 operational status  report which consisted of a larger list of functions than the Executive  Director cited and states were expected to automate all these  functions. While we recognize that the District of Columbia was able  to enroll applicants through its system, CMS’s report indicated that  these specific functions, which support important provisions of  PPACA, were not complete or fully automated. Regarding hub  services, the Executive Director said that the District of Columbia  requested and received permission from CMS not to deploy a specific  function for plan year 2015 but has begun testing this function for plan  year 2016. Since the District of Columbia was still testing this hub  service, it had not fully developed, tested, and implemented this  functionality required by CMS. The District of Columbia Health Benefit  Exchange Authority’s comments are reprinted in appendix IV.
In e-mail comments, the Executive Director of the Office of the  Kentucky Health Benefit Exchange requested that we clarify the  partial rating for IRS required submissions because the Executive  Director believed that the state had been fully compliant with these  requirements. However, according to CMS’s February 2015  operational status report, Kentucky had not completed the most  recent annual submission of IRS data which is used to ensure that  individuals received the correct amount of premium tax credit.
In written comments, the Interim Chief Executive Officer of MNsure,  the Minnesota marketplace, generally agreed with the operational  status ratings for the functional categories. But the official also noted  that while the functions may be rated as partially operational, our  report did not recognize that MNsure delivered the required services  and in some cases used manual workarounds to temporarily meet the  functional requirements. We recognized that states implemented  workarounds to deliver services, but our report focuses on the status  of fully automated functionality delivered by states’ IT projects. For  example, regarding eligibility and enrollment functions, although  MNsure sent automated notices for most consumers, due to system  limitations it was unable to issue automated notices to some  consumers renewing coverage and therefore created manual notices  for these consumers.
In addition, regarding financial management functions, the Interim  Chief Executive Officer said MNsure was billing small business  customers using a manual process in February 2015, but has since  incorporated automation into the process. Further, the official noted  that MNsure opted to have certain financial management functions  performed by CMS. While MNsure made progress in this area, we are  reporting the status according to CMS’s February 2015 operational  status report, which is a consistent source and time frame of data for  all states, and these financial management functions were  categorized as not operational in the report. Regarding hub services,  the Interim Chief Executive Officer generally agreed with the status  and stated that MNsure will continue to plan for testing of these  functions. Regarding IRS reporting, the official generally agreed with  the status and stated that the delays for submitting files to IRS were  due to additional quality assurance work. The MNsure Minnesota  marketplace’s comments are reprinted in appendix V.
In e-mail comments, the Deputy Director of the New York State  Department of Health disagreed that financial management, hub  services and IRS reporting file submissions functions were partially  operational as of February 2015, and believed that the ratings should  reflect fully operational or fully complete. In addition, the Deputy  Director stated that the state should not receive partial ratings  because it opted to have CMS perform certain financial management  functions, determined alternate methods for completing certain hub  services functions, and was waiting for solutions from CMS regarding  IRS reporting file submissions. Although New York opted to have  certain financial management functions performed by CMS, the  agency’s February 2015 operational status report categorized these  functions as not operational. Further, while CMS may have allowed  certain alternate methods or workarounds for hub services functions,  CMS’s operational status report indicated that these specific functions  were not complete or fully automated. Even though New York may  have been waiting for a solution from CMS to complete its IRS  reporting file submissions, CMS’s report noted that this function was  not fully complete.
In written comments, the Chief Executive Officer of the Washington  Health Benefit Exchange concurred with our characterization of the  status of eligibility and enrollment functions and IRS reporting file  submissions but did not agree that its financial management and hub  services functions were only partially operational. The Chief Executive  Officer stated that our report lacked the necessary details for him to  review in order to respond to these characterizations. We later  provided details from CMS’s February 2015 operational status report  that we evaluated to determine the status of the state’s marketplace.  Subsequently, the official stated that certain financial management  functions were incomplete because the state opted to have these  functions performed by CMS. Nonetheless, CMS’s February 2015  operational status report categorized these functions as not  operational. For hub services, the official noted that the Washington  Healthplanfinder successfully used multiple services offered by the  federal hub to verify Social Security numbers, citizenship, lawful  presence, income, and other eligibility factors and that the  marketplace has tested these services. However, CMS’s February  2015 operational status report noted that it had only partially  completed certain hub services for verifying eligibility. The  Washington Health Benefit Exchange’s comments are reprinted in  appendix VI.
Other technical comments provided via e-mail by marketplace and  Medicaid officials within these states were considered and incorporated  into our final report as appropriate  We are sending copies of this report to the Secretary of Health and  Human Services and other interested parties. In addition, the report is  available at no charge on the GAO website at http://www.gao.gov.
Should you or your staffs have questions on matters discussed in this  report, please contact me at (202) 512-6304. I can also be reached by e- mail at melvinv@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made major contributions to this report are listed in  appendix VII.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) determine how states have used federal funds  for IT projects to establish, support, and connect to health insurance  marketplaces, including amounts spent, and the overall status of their  development and operation; (2) determine CMS’s and states’ roles in  overseeing these state IT projects; and (3) describe IT challenges that  states have encountered in developing and operating their marketplaces  and connected systems, and lessons learned from their efforts.
To address the three objectives, we designed and administered a web- based survey to collect information about the state health insurance  marketplace IT projects in the 50 states and the District of Columbia. We  developed two versions of this survey: one for states with state-based  marketplaces, including those using the federal marketplace IT solution,  and one for states with a federally facilitated marketplace or federally  facilitated partnership. Seventeen states received the state-based  version of the survey, and 34 states received the federally facilitated  version. Generally, the survey asked state program officials about  federal and state funding for developing and operating state  marketplace-related IT projects,  state marketplace and project types,
CMS’s and state’s marketplace oversight roles and tools, and  challenges and lessons learned with state marketplace IT  development and operations.
Out of the original population of state health marketplaces in the 50 states  and the District of Columbia, 46 states and the District of Columbia  submitted survey responses; however, not all respondents provided  answers to every question. We did not independently verify the data the  states provided in each case, but we did, in selected cases, compare  them to equivalent CMS data. We also relied on CMS-provided data,  rather than survey data, in most cases because we received more up-to- date and complete information from CMS. The survey was administered  between September 30, 2014, and November 19, 2014. The status of  state marketplace types is as of the end of the second enrollment  period—which ended on February 15, 2015.
Several weeks before the survey period began, we notified recipients that  they would be receiving it and confirmed that they were the appropriate  state contacts. We also followed up with non-respondents several times  before the survey period ended.
In developing the surveys, we took steps to ensure the accuracy and  reliability of responses. We pre-tested the survey with marketplace and  Medicaid officials from seven states to ensure that the questions were  clear, comprehensive, and unbiased, and to minimize the burden the  questionnaire placed on respondents.
To determine how states have used federal funds to establish, support,  and connect to health insurance marketplaces and the overall status of  their development and operation, we reviewed CMS guidance regarding  federal funding and development for marketplaces such as the  marketplace grant funding opportunity announcement, instructions for  marketplace reporting, guidance for marketplace and Medicaid IT  systems, and blueprint guidance for approval of state marketplace types.  We also reviewed best practices for IT investment management and  managing program costs. We then reviewed CMS funding and status  documentation, including notices of grant awards and state IT spending  and status summaries. We also analyzed state survey responses on  costs and development status, including state documentation on federal  grant and Medicaid costs.
To assess the reliability of CMS’s data on state-reported IT spending to  establish, support, and connect to marketplaces, we assessed the  reliability of the systems used to collect the information. We asked  officials responsible for entering and reviewing the grants information a  series of questions about the accuracy and reliability of the data. Among  the sources of data used for our study, we reviewed a spreadsheet  compiled by CMS Center for Consumer Information and Insurance  Oversight officials that contained state-reported grant funding data and  marketplace IT project status information drawn from three separate  information systems: CMS’s On-Line Data Collection System, Grant  Solutions, and the Payment Management System. The spreadsheet  was a consistent source of information that reflected the same cost  factors for all states as of March 2015. Specifically, the spreadsheet  tracked, among other things, the type and total amount of grant funding  provided and available to each state, as well as the time period for  expending those funds. We also reviewed the data to determine if there  were any outliers and other obvious errors in the data. For any anomalies  in the data, we followed up with CMS to either understand or correct  those anomalies.  We determined that the data were sufficiently reliable  for our purposes and noted any limitations in our report. While our report  discusses state-reported IT spending based on CMS data, we did not  verify the accuracy of the data states reported to CMS.
We also reviewed our recent report on Medicaid funding for eligibility IT  system changes, which addressed state-reported Medicaid expenditure  data from CMS-64—a form that states complete quarterly to obtain  federal reimbursement for services provided or administrative costs  incurred. We updated our review of states’ reported expenditures,  beginning with the quarter ending June 30, 2011, the first quarter for  which 90/10 funds were available to states, through the quarter ending  December 31, 2014. To determine the reliability of the CMS-64 data, we  reviewed related documentation and our prior records of interviews with  CMS officials describing how these data are collected and processed; we  also examined other research that has used these data to report state  expenditures. We determined that the data we used in this report were  sufficiently reliable and noted any limitations in our report.
In addition, we reviewed and analyzed CMS documentation of states’  marketplace status and operation progress and challenges to summarize  the status of marketplaces. We reviewed states’ survey responses  regarding changes in and the status of developing and operating their  marketplace IT solutions. We also reviewed CMS state marketplace  operational status reports as of February 2015 and the CMS State  Exchange Resource Tracking System as of April 2015. We did not  independently verify the accuracy of CMS’s data on states’ operational  status. We also obtained input from CMS regarding funding and status of  marketplaces through interviews with knowledgeable officials.
To determine CMS’s and states’ roles in overseeing these state IT  projects, we analyzed applicable federal laws and regulations, CMS  marketplace policies and guidance, documentation on applicable CMS  marketplace roles and responsibilities and state marketplace governance  structures, state survey responses regarding their governance structures,  and state survey responses and ratings regarding the effectiveness of  CMS guidance, oversight, and related systems.
We also compared CMS’s policies and procedures to best practices  included in GAO’s IT investment management framework and to the  Project Management Institute’s A Guide to the Project Management Body  of Knowledge (PMBOK® Guide) to determine whether CMS had roles  and responsibilities clearly documented and communicated in its policies  and procedures. Further, we reviewed CMS’s funding oversight  processes and compared them to relevant sections of GAO’s IT  investment management framework to determine if CMS followed best  practices for overseeing IT investments. We used our survey results to  describe how the states viewed CMS’s oversight and guidance in regard  to the marketplace-related IT projects.
We also reviewed CMS’s Enterprise Life Cycle guidance for systems  development reviews and reports from states’ operational readiness  reviews from August and September 2013 to assess the extent to which  CMS followed its process. In addition, we reviewed state survey  responses and other state-provided documents to determine states’  marketplace oversight roles. Further, we interviewed CMS officials  responsible for the oversight and implementation of the state  marketplaces to obtain their perspective on their marketplace roles.
To describe IT challenges encountered in developing and operating the  marketplace and connected systems as well as lessons learned from  these efforts, we analyzed state survey responses related to challenges,  lessons learned, and best practices identified by state officials and  documentation such as CMS meeting presentations. For the state  surveys, we identified a variety of marketplace-related IT challenges  based on our analysis of CMS and state documentation and interviews,  and grouped these challenges according to several broad areas. State- based marketplace challenges were divided into five areas in the survey  (project management and oversight, marketplace IT solution design,  marketplace IT solution development, resource allocation and distribution,  and marketplace implementation and operation), while federally facilitated  challenges were divided into two areas (project management and  oversight and system design and development) based on the IT work  each marketplace performs. For the purposes of our report, we  consolidated the marketplace IT solution design and marketplace IT  solution development challenge areas for the state-based marketplaces.
In both the state-based and federally facilitated versions of our survey, we  asked states to rate their experience with each of these identified  challenges using a 5-point scale with the following response options: very  great challenge, great challenge, moderate challenge, somewhat of a  challenge, or little or no challenge. In our report, we combined the very  great and great state ratings. We then analyzed states’ ratings of  challenges and using counts of the “very great” and “great” responses, we  selected the greatest (i.e., the top two) challenges from each area for  discussion in this report. If a challenge area applied to both states using a  state-based marketplace and states with a federally facilitated  marketplace, the greatest challenges from each marketplace type were  selected.
Further, we asked each state to identify whether they had identified best  practices or lessons learned within each challenge area of our survey,  and to include specific examples of those best practices and lessons. We  reviewed all written survey responses regarding states’ lessons learned to  ensure these lessons were appropriately categorized into each identified  challenge area. Based on our qualitative analysis of the states’ survey  responses, we identified the number of states that provided lessons  learned and then provided examples of the best practices or lessons  learned that related to the greatest challenges in each area, if there were  any. We also interviewed CMS and state officials responsible for the  oversight and implementation of the state marketplaces to determine  what the agency did to identify and share states’ challenges, best  practices, and lessons learned.
We conducted this performance audit from April 2014 to September 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Health Insurance Marketplace Grant Funding and State-Reported Expenditures
paragraphs: To help states establish a marketplace, the Patient Protection and  Affordable Care Act (PPACA) authorized the Department of Health and  Human Services (HHS) to award federal exchange (now referred to as  marketplace) grants for planning and implementation activities, as well as  for the first year of a marketplace’s operation. States were required to  report marketplace grant spending, including IT spending, to HHS’s  Centers for Medicare & Medicaid Services (CMS).
The following table shows the amount of marketplace grants awarded;  the amount of grants spent or drawn down; the amount authorized for IT;  and the amount spent for IT as of March 2015, for the four different  marketplace types—state-based, state-based using the federal  marketplace IT solution, federally facilitated, and federally facilitated  partnership marketplaces.
section_title: Appendix III: Comments from the Department of Health and Human Services
section_title: Appendix IV: Comments from the District of Columbia Health Benefit Exchange Authority
section_title: Appendix V: Comments from MNsure
section_title: Appendix VI: Comments from the Washington Health Benefit Exchange
section_title: Appendix VII: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Tammi Kalugdan (assistant  director), Christie Motley (assistant director), Christopher Businsky, Debra  Conner, Sandra George, David Hong, Kendrick Johnson, Lee  McCracken, Monica Perez-Nelson, Jerome Sandau, Brandon Sanders,  Andrew Stavisky, Karin Wallestad, Merry Woo, and Elizabeth Wood made  key contributions to this report.