PPACA directed each state to establish and operate a health insurance marketplace by January 1, 2014. In cases where states elected not to establish and operate a marketplace, the law directed the federal government to establish and operate a health insurance marketplace on their behalf. These marketplaces were expected to provide a seamless, single point-of-access for individuals to enroll in private health insurance plans and apply for income-based financial assistance established under the law. PPACA and HHS regulations and guidance require every state to have marketplace capabilities that enable them to carry out four key functions, among others: Eligibility and enrollment. The marketplace must enable individuals to assess and determine their eligibility for enrollment in healthcare coverage. In addition, the marketplace must provide individuals the ability to obtain an eligibility determination for other federal healthcare coverage programs, such as Medicaid and the Children’s Health Insurance Program (CHIP). Once eligibility is determined, individuals must be able to apply for and enroll in applicable coverage options. Plan management. The marketplace is to provide a suite of services for state agencies and health plan issuers to facilitate activities such as submitting, monitoring, and renewing qualified health plans. Financial management. The marketplace is to facilitate payments of premiums to health plan issuers and also provide additional services such as payment calculation for risk adjustment analysis and cost- sharing reductions for individual enrollments. Consumer assistance. The marketplace must be designed to provide support to consumers in completing an application, obtaining eligibility determinations, comparing coverage options, and enrolling in healthcare coverage. To provide these capabilities, PPACA further required the states, as well as HHS (who delegated this role to CMS) to establish supporting automated systems and capabilities. Toward this end, states and CMS undertook projects to design, develop, implement, and operate health insurance marketplace systems. States electing to establish their own marketplaces (hereafter referred to as a state-based marketplace) were required, in accordance with CMS guidance and regulations, to develop their own IT solutions, including a web portal for individual consumers to interact with and select healthcare coverage, as well as supporting systems that perform functions such as real-time eligibility queries, transferring application information to state Medicaid/CHIP agencies, sending taxpayer information to the Internal Revenue Service, and exchanging enrollment information with issuers of qualified health plans. In addition, state-based marketplace IT solutions were required to interface with CMS systems designed to exchange information with external partners, including other federal agencies and states, and facilitate the electronic payment of insurance premiums to plan issuers. As an alternative to their web portals, these states were also required to set up call centers through which consumers could apply for coverage. A state that operates its own marketplace can request that CMS perform eligibility and enrollment functions using federal IT systems. We refer to this as a state-based marketplace using the federal marketplace IT solution. This type of marketplace evolved when certain states encountered IT-related challenges during the development of their state marketplace solutions. Further, if a state elected not to establish its own marketplace, CMS assumed some or all aspects of the marketplace operations for that state using two additional marketplace types: Federally facilitated marketplace: CMS is responsible for all aspects of establishing and operating the marketplace including the four key functions. Federally facilitated partnership: CMS is responsible for establishing and operating the eligibility enrollment and financial management functions, while the state assists with plan management and/or consumer assistance. In these cases, states rely to varying degrees on the systems developed by CMS to support a federally facilitated marketplace. These include Healthcare.gov—the federal website that serves as the user interface for individuals to obtain information about health coverage, set up a user account, select a health plan, and apply for healthcare coverage—and several supporting systems. The supporting systems include a system for verifying an applicant’s identity and establishing a login account; a transactional database to facilitate eligibility and enrollment, plan management, financial services, and other functions; and a data services hub that serves as a single portal for exchanging information with external partners. For example, federal agencies such as the Social Security Administration (SSA), Department of Homeland Security (DHS), and Internal Revenue Service (IRS) provide or verify information used in making determinations of a person’s eligibility for coverage and financial assistance. For plan year 2015, 14 states had a state-based marketplace, 3 had a state-based marketplace using the federal marketplace IT solution, 27 had a federally facilitated marketplace, and 7 had a federally facilitated partnership (see fig. 1). Depending on the type of marketplace established in his or her state, an individual user would apply for health coverage through either their state’s web portal or through Healthcare.gov. The key functions required to enroll that individual would then be carried out by a combination of state and federal systems specific to the type of marketplace. A general depiction of both the state and federal marketplace IT solutions is provided in figure 2. States had access to two sources of federal funding to establish their marketplaces: federal marketplace grants and Medicaid matching funds. CMS allows states to use both Medicaid matching funds and marketplace grants to pay for shared system services and functions that states needed to establish for marketplace operations, such as developing a rules engine system and establishing interfaces to the federal data services hub. Various offices within CMS were tasked with overseeing grant reviews, Medicaid advanced planning document reviews, and IT gate reviews to ensure that states followed a standardized funding process for their marketplace-related IT projects. These offices included the Center for Consumer Information and Insurance Oversight (CCIIO), Center for Medicaid and Chip Services (CMCS), and the Office of Technology Solutions (OTS). PPACA authorized HHS to award federal exchange grants (now referred to as marketplace grants) for planning and establishing marketplaces. The act did not specify an exact amount of marketplace grant funding, but appropriated to HHS, out of any moneys in the Treasury not otherwise appropriated, an amount necessary to make marketplace grant awards. The act directed HHS to determine the total amount of funding that it would make available to each state for each fiscal year and authorized the department to award marketplace grants to states through December 2014. On the basis of this authority, HHS established four separate programs for awarding marketplace grants to states. Planning Grants: Provided states with resources to conduct the initial research and planning needed to build a marketplace and determine how it would be operated and governed. The grants were awarded to states in 2010 and 2011 and provided 1 year of funding. A state could receive only one planning grant. Early Innovator Grants: Provided funding to a state or group of states that were identified as early leaders in building their marketplaces, to assist in designing and implementing the IT infrastructure needed to operate the marketplaces. All marketplace IT components, including software and data models, developed with these grants could be adopted and modified by other states to fit their specific needs. The grants were awarded in February 2011 and the grant funds were available for 2 years. A state could only receive one of these grants. Establishment Grants (Level 1): Provided funding for a 1-year project period to states pursuing any marketplace type. This funding was intended to help states undertake additional marketplace establishment activities, such as changes in response to legislative or regulatory requirements, developing IT systems, and consulting with key stakeholders. The grants were awarded between May 2011 and December 2014. Once awarded, the funds were available for 1 year, and a state could apply for multiple grants. Establishment Grants (Level 2): Provided funding for a multi-year project period to states that have legal authority to implement a marketplace and are further along in marketplace development and are pursuing a state-based marketplace. This funding was designed to help the states carry out all marketplace activities, including consumer and stakeholder engagement and support, eligibility and enrollment, plan management, and technology development. The grants were awarded between May 2011 and December 2014. Once awarded, the grant funds remain available for up to 3 years. A state could receive only one grant. States establishing state-based marketplaces were expected to carry out activities in a number of areas to receive these marketplace grants. These activity areas included stakeholder consultation, program integration, IT systems development, financial management, oversight and program Integrity, health Insurance market reform, and business operations of the marketplace. Once grants were awarded, funding was disbursed using the Payment Management System, which is an HHS-administered system that provides federal agencies and grant recipients the tools to manage grant payments. Grantees submitted progress reports documenting financial expenditures and program progress through an online data collection system on a monthly and semi-annual basis. As of December 31, 2014, CMS had awarded approximately $5.51 billion in federal marketplace grants to states. Of these grant funds awarded, CMS had authorized states to spend approximately $2.16 billion on IT to support state-based marketplaces and federally facilitated partnerships as of March 2015. According to CMS, funds authorized for IT contracts could be designated as restricted and required prior approval from the various CMS offices mentioned previously before the funds could be spent. For states that opted to use the federally facilitated marketplace, IT funds were not provided after it was determined that these states were not establishing a state-based marketplace. With the enactment of PPACA, changes to Medicaid eligibility and enrollment systems were needed in order for the Medicaid program to operate seamlessly with the marketplaces, as well as to implement new Medicaid eligibility policies. Specifically, in all states, the Medicaid eligibility and enrollment system had to be replaced or modernized to meet the more streamlined enrollment process requirements of PPACA and its implementing policies, which included real-time transfer of applications between the state Medicaid agencies and the marketplace and immediate Medicaid eligibility determinations, regardless of the type of marketplace a state elected to use. Under federal law, states are eligible to receive funding, in the form of an enhanced federal matching rate of 90 percent (referred to as 90/10 funding), for the design, development, or installation of their Medicaid claims processing and information retrieval systems. Because states’ Medicaid eligibility and enrollment systems had to be replaced or modernized to meet the PPACA requirements, CMS expanded the availability of federal Medicaid funds at the enhanced matching rate of 90 percent to help states pay for required changes, including their interfaces to establish connections to the federal marketplace IT solution through the federal data services hub or the state marketplace IT solution. This enhanced federal matching rate is available to cover costs incurred by the states related to changes to their Medicaid eligibility systems from April 19, 2011, to December 31, 2015. All states are eligible to obtain the 90/10 funds for IT-related changes they make to their Medicaid eligibility and enrollment systems. In addition, a state may receive funding in the form of a 75 percent federal matching rate for the maintenance and any ongoing costs of operating its upgraded Medicaid eligibility and enrollment system. The funding is generally available when the upgraded system becomes operational, and it does not expire. In updating their Medicaid eligibility and enrollment systems, states could use federal funds for full system replacements or for more limited modifications, with the scope of a state’s changes depending on a number of factors, including the age of the system and the extent of integration among state programs. Federal regulations require the approval of advanced planning documents in order for states to be able to draw down the 90/10 and 75/25 matching funds. To access Medicaid matching funding, states must first submit these planning documents to CMS. In its role as the agency that oversees the Medicaid program and provides guidance and technical assistance to states related to Medicaid eligibility and enrollment system changes, CMS is to review these documents to ensure that certain technical and operational criteria are met before states are eligible for the funding. To receive approval, states must develop IT systems that meet technical standards and conditions. These standards and conditions require states to develop systems that are flexible, align with the Medicaid Information Technology Architecture principles, and promote data exchanges and the reuse of Medicaid technologies across systems and states. Figure 3 provides a timeline of the health insurance marketplaces’ major activities previously mentioned, including dates when federal funding became available and enrollment time frames. During the first enrollment period, states faced difficulties developing and operating their marketplace IT solutions. For state-based marketplaces, various sources reported that technical issues varied widely, contributing to websites that froze midway through the process of applying for coverage, system crashes, and systems taken offline for days at a time, ultimately causing applicants to face long waits for eligibility determinations. One state reported technical problems serious enough to prevent any online enrollment; thus, thousands of individuals had to enroll manually using paper applications. The problems experienced in state-based marketplaces for the first enrollment period were different in each state, but they included poor system performance and delays in addressing information partially completed software functionality, enrollment errors causing long wait times and applications to get stuck difficulties getting individuals’ identities verified through the systems, the inability to easily make changes to individuals’ insurance coverage in response to events such as births or income changes. States that relied on the federally facilitated marketplace and federally facilitated partnerships also encountered problems in the development and operation of their IT solutions during the first enrollment period. For example, in these states, consumers attempting to enroll in health plans through Healthcare.gov and its supporting systems were met with confusing error messages, slow load times for forms and pages, and in some cases, website outages. We previously reported that Healthcare.gov and its supporting systems were hindered by inadequate system capacity, numerous errors in software code, and limited system functionality—all of which impeded the systems’ performance and their availability for consumers’ use. Regarding state Medicaid systems, states with a federally facilitated marketplace, federally facilitated partnership, or state-based marketplace using the federal marketplace IT solution reported challenges in implementing the requirement to transfer or send and receive applications. For example, none of these types of states were able to transfer applications via the marketplace by the start of the first enrollment period on October 1, 2013. Over the past 2 years, we have issued various reports highlighting challenges that CMS and the states have faced in implementing and operating health insurance marketplaces. For example, in an April 2013 report, we described the actions of seven states that were in various stages of developing an information technology infrastructure to establish marketplaces, including redesigning, upgrading, or replacing their outdated Medicaid and CHIP eligibility and enrollment systems. Six of the seven states were also building the IT infrastructure needed to integrate systems and allow consumers to navigate among health programs, but identified challenges with the complexity and magnitude of the IT projects, time constraints, and guidance for developing their systems. In September 2014, we reported that while CMS had taken steps to protect the security and privacy of data processed and maintained by the systems that support Healthcare.gov, weaknesses remained in both the processes used for managing information security and privacy as well as the technical implementation of IT security controls. Specifically, we noted that Healthcare.gov and the related systems had been deployed despite incomplete security plans and privacy documentation, incomplete security tests, and the lack of an alternate processing site to avoid major service disruptions. Accordingly, we recommended that CMS implement 22 information security controls. We also recommended that the agency improve its system security plans, privacy documentation, security tests, and alternate processing site for the systems that support Healthcare.gov. HHS concurred with all 22 of the recommendations to improve the effectiveness of its information security control and fully or partially concurred with our remaining information security program-related recommendations. The department stated that it intends to take steps to address the weaknesses, including updating its security plans, developing required computer matching agreements, and developing a backup site for Healthcare.gov. In December 2014, we reported that all states using the federal marketplace IT solution had faced challenges transferring applications to and from that system. We pointed out that none of the states using the federal marketplace IT solution in the first enrollment period were able to implement application transfers, which required the establishment of two IT connections: one connection to transfer applications found ineligible for Medicaid coverage from the state Medicaid agency to the federal marketplace IT solution, and another connection to transfer applications found ineligible for coverage from the federally facilitated marketplace to the state Medicaid agency. Most recently, in March 2015, we reported that several problems with the initial development and deployment of Healthcare.gov and its supporting systems had led to consumers encountering widespread performance issues when trying to create accounts and enroll in health plans. We noted, for example, that CMS had not adequately conducted capacity planning, adequately corrected software coding errors, or implemented all planned functionality. In addition, the agency did not consistently apply recognized best practices for system development, which contributed to the problems with the initial launch of Healthcare.gov and its supporting systems. In this regard, weaknesses existed in the application of requirements, testing, and oversight practices. Further, we noted that HHS had not provided adequate oversight of the Healthcare.gov initiative through its Office of the Chief Information Officer. We made recommendations aimed at improving requirements management, system testing processes, and oversight of development activities for systems supporting Healthcare.gov. HHS concurred with all of our recommendations and subsequently took or planned steps to address the weaknesses, including instituting a process to ensure functional and technical requirements are approved, developing and implementing a unified standard set of approved system testing documents and policies, and providing oversight for Healthcare.gov and its supporting systems through the department-wide investment review board. States reported to CMS that they spent federal marketplace grant funds, as well as Medicaid matching funds, on various IT projects to establish, support, and connect to health insurance marketplaces. Specifically, states reported spending about $1.45 billion in federal marketplace grant funds from September 2010 through March 2015. The states also reported spending federal funds designated for Medicaid eligibility and enrollment systems on marketplace-related IT projects, although the actual amount spent was uncertain, as only a selected number of states reported on our survey that they tracked or estimated this information. In this regard, from April 2011 through December 2014, states reported spending $2.78 billion in combined federal and state Medicaid funds, a portion of which was spent to support the marketplaces. States that chose to establish state-based marketplaces were responsible for the majority of the federal marketplace grant spending. These states’ efforts typically included developing web portals and supporting data processing systems to carry out key marketplace-related functions, and establishing electronic connections in order to exchange information with various states, federal partners, and issuers. Fourteen states with state-based marketplaces had developed and were operating IT systems to support their marketplaces; however, not all system functions were complete as of February 2015. In addition, according to a CMS status report, as of November 2014, 7 of 37 states using the federal marketplace IT solution could not transfer applications for health insurance coverage between their state Medicaid systems and the federal data services hub or had not completed testing or certification of these functions. According to CMS officials, states operating IT systems and states using the federal marketplace IT solution were continuing to improve the development and operation of their marketplaces in the second enrollment period. States reported to CMS spending approximately $1.45 billion in federal grant funds on IT projects to establish, support, and connect to health insurance marketplaces from September 2010 to March 2015. States that established state-based marketplaces, including state-based marketplaces using the federal marketplace IT solution, reported having spent approximately $1.37 billion of these funds. In addition, states with a federally facilitated marketplace reported spending approximately $47 million, while those with a federally facilitated partnership reported spending approximately $32 million. Table 1 provides a summary of the states’ reported use of marketplace grant funds for their IT projects as of March 2015. In addition to the $1.45 billion of reported IT spending, approximately $703 million of authorized grant funding for IT projects had not been spent as of mid-March 2015. For additional details on the amount of marketplace grant funding awarded and spent, see appendix II. States with state-based marketplaces were authorized by CMS to spend $2.02 billion for IT until December 2015, and this authorized amount per state ranged from approximately $55 million to $325 million as of March 2015. As shown in figure 4, the reported spending of grant funds among the 17 states that were approved to establish state-based marketplaces, (i.e., the 14 state-based marketplaces and the 3 state-based marketplaces using the federal marketplace IT solution), ranged from approximately $29 million (in Minnesota) to approximately $254 million (in California), as of March 2015. Regarding states with a federally facilitated marketplace or federally facilitated partnership, 19 of these states were authorized by CMS to spend $378 million for IT, and this authorized amount per state ranged from approximately $158,000 to $81 million as of March 2015. These states reported marketplace grant IT spending that ranged from approximately $30,000 (in Alabama) to approximately $20 million (in Iowa), as of March 2015 (see fig. 5). The 15 other states that used these two types of marketplaces were not authorized to spend grant funds for IT projects. In June 2015, CCIIO officials told us that, with the exception of Arkansas, Mississippi, and Utah, states with a federally facilitated marketplace or federally facilitated partnership are no longer authorized to spend marketplace grant funding for information technology because they are no longer investing in the long-term creation of a modern eligibility system to be shared between a state-based marketplace and the state Medicaid program. CMS required the states to report their grant spending for marketplace IT projects in five broad budget categories: contracts, consultants, personnel, equipment, and supplies. In this regard, the 17 states that established state-based marketplaces, including state-based marketplaces that used the federal marketplace IT solution, reported spending the following approximate amounts in these categories, as of March 2015: $1.13 billion on contracts, $76.18 million on consultants, $39.00 million on state personnel, $21.06 million on equipment, and $720,000 on supplies. The largest part of these reported expenditures—nearly 89 percent—was on contracts for services such as systems integration, project management, and independent validation and verification. In addition to costs in these five categories, CMS also asked the states to report the amount of early innovator IT marketplace grant funding that they had spent. In response, these states reported that they had spent approximately $112.4 million of such funding. The 34 states with a federally facilitated marketplace or federally facilitated partnership reported spending, as of March 2015, approximately $69.68 million on contracts, $2.19 million on consultants, $1.66 million on state personnel, $5.68 million on equipment, and $.03 million on supplies. These states also reported spending $.06 million of early innovator IT marketplace grant funding. Table 2 shows marketplace grant spending for IT, by category, as of March 2015. During the course of our work, in October 2014, CMS began collecting data on IT contract costs in new categories aimed to gather a greater level of detail across states with state-based marketplaces. These new reporting categories are system integration, project management, independent verification and validation, middleware software, rules engine software, and “other.” As of May 2015, 11 state-based marketplaces had reported costs in some of these new detailed cost categories. However, CMS’s documentation indicated that not all states reported using all the new categories. For example, not all states reported costs in the rules engine and middleware software categories because those costs were included in the system integration category or marked in the “other” category. Specifically, only five states reported costs for developing rules engine software or middleware software. According to CCIIO officials, CMS is following up with states on missing amounts. Following through on these efforts to collect more detailed information on states’ IT contract costs would increase CMS’s insight into states’ IT spending. States also spent Medicaid funds for marketplace-related IT projects, such as modifying Medicaid eligibility and enrollment systems to interface with the marketplaces. Specifically, states spent some portion of approximately $2.78 billion in combined federal and state Medicaid funding from April 2011 through December 2014 for marketplace-related IT projects. Of this amount, $2.42 billion was from 90/10 funding and $364 million was from 75/25 funding. An undetermined portion of this spending was used to develop and maintain eligibility and enrollment systems connections to the marketplaces. States that established state-based marketplaces, including state-based marketplaces using the federal marketplace IT solution, reported having spent approximately $757 million of the 90/10 Medicaid funds for Medicaid eligibility and enrollment systems. Further, states with a federally facilitated marketplace reported spending approximately $1.32 billion of these funds, and those with federally facilitated partnerships reported spending approximately $340 million. The amounts spent included expenditures for marketplace-related IT projects. Of the $364 million in 75/25 Medicaid funds, states that established state- based marketplaces, including state-based marketplaces using the federal marketplace IT solution, reported having spent approximately $56 million. Those with a federally facilitated marketplace reported spending approximately $285 million, and those with a federally facilitated partnership reported spending approximately $23 million. Table 3 provides a summary of states’ Medicaid 90/10 and 75/25 expenditures for Medicaid eligibility and enrollment systems by marketplace type, as of December 2014. While CMS required states to report the ratio of Medicaid funds to grant funds in allocating their planned spending for marketplace-related IT projects, the agency did not require states to track the actual amount of Medicaid funds spent specifically on these IT projects. Thus, the total portion of Medicaid funds spent for those purposes is unknown. However, as part of our survey, 26 states were able to track or estimate the portion of marketplace-related IT spending for Medicaid 90/10 funds, and 17 states were able to track or estimate the portion of marketplace- related IT spending for Medicaid 75/25 funds. The states that tracked or estimated their use of Medicaid funds reported spending approximately $750 million of these funds—both 90/10 and 75/25 funds—for marketplace-related IT projects through June 2014. The remaining states in our survey did not track the amount or could not provide the actual or estimated amount of Medicaid funds spent. Based on the survey responses, states may have tracked or estimated these amounts using a variety of approaches, thus state-reported data may not be consistent across states. Table 4 shows the approximate state-reported amounts of combined federal and state 90/10 and 75/25 Medicaid funding used for marketplace-related IT projects by marketplace type. Generally, the states used federal funds (both marketplace grant and Medicaid matching funds) for various IT projects, including the establishment and operation of their marketplaces and their connection to the federal marketplace. Accordingly, the nature and extent of their efforts varied depending on which marketplace type they chose to establish. The 17 states that were approved to establish state-based marketplaces, (i.e., the 14 state-based marketplaces and the 3 state-based marketplaces using the federal marketplace IT solution) undertook various IT projects to establish their marketplaces. These states generally used the funds to develop their IT solutions, including the web portal for individual consumer interaction (to set up user accounts, select health plans, and apply for health coverage); systems to perform the key marketplace functions (eligibility and enrollment, plan management, financial management, and consumer assistance); functionality for determining Medicaid and CHIP eligibility using new income standards; functionality for sharing marketplace enrollment data with qualified health plan issuers; and interfaces with federal systems through the federal data services hub (needed to conduct eligibility verifications). In documents provided to supplement the survey responses, states also reported using their funds to cover numerous other expenses for state personnel, systems integrator contracted services, interface development and maintenance, independent verification and validation services, project management, technical support, and software licenses. Among the 34 states with a federally facilitated marketplace or federally facilitated partnership, IT projects typically involved system development to connect the states’ existing Medicaid systems to CMS’s federal data services hub. In addition, 17 of these states reported on our survey that they conducted projects to explore the option of developing IT systems to support a state-based marketplace (even though they ultimately chose to participate in the federally facilitated marketplace). For example, one state reported to CMS that it used grant funds to develop technical requirements and an architectural design, along with a request for proposals to obtain a systems integrator for the implementation of a marketplace. Another state using the federally facilitated marketplace was awarded marketplace grant funds to support technology projects in anticipation of becoming a state-based marketplace. According to CMS officials, states that initially planned for, but did not pursue, a state-based marketplace were required to return the funds to CMS or to re-budget the funds for non-IT costs. In addition, two federally facilitated partnership states used marketplace grant funds to develop new integrated Medicaid eligibility and enrollment systems needed to support new requirements, such as determining income eligibility for Medicaid using new income standards. As of February 2015, the 14 states with state-based marketplaces had developed and were operating systems to support their marketplaces; however, not all IT functions were complete. In particular, CMS reported that these 14 states’ marketplace systems were performing some, but not all, key functions, including those related to eligibility and enrollment, financial management, hub services, and IRS reporting: With regard to eligibility and enrollment functions, CMS status reports indicated that eight state-based marketplace systems were fully operational and operating without interruptions in service. The other six state-based marketplace systems were partially operational, meaning that these functions were operational but did not work as intended and may have required manual processes to supplement automated functionality. States with partially operational functions used business process workarounds to complete eligibility and enrollment functions, such as manually entering and verifying individuals for healthcare coverage. For example, in one state, applications to the state-based marketplace were sent by Medicaid as portable document format (PDF) files and processed by data entry specialists. In another state, data transferred from the marketplace to Medicaid was automated, but other information was manually entered. With regard to financial management functions such as collecting premium payments, remitting payments to issuers, and payment calculation for reinsurance, 4 state-based marketplace systems were fully operational without interruptions in service and 8 state-based marketplace systems were partially operational and may have required manual workarounds. (These functions were not applicable for 2 state-based marketplace systems that decided to rely on issuers to conduct premium billing and processing functions.) Although all states developing state IT solutions had received approval from CMS to connect to the federal data hub, only 1 state- based marketplace state had fully completed development of hub services functions such as verifying an individual’s identity and citizenship and retrieving tax information for evaluating taxpayer eligibility for insurance affordability programs. Thirteen state-based marketplace states had partially completed hub services functions, meaning that they had not yet implemented all hub services because the testing or development had not been completed or independent verification and validation attestation had not yet been received. With regard to submissions to IRS regarding information such as premium tax credits, 1 state had fully completed performance testing of these functions, 10 states had partially completed performance testing, and 2 states had not completed any performance testing of these functions. Additionally, these functions were not applicable for 1 state, which used the federal IT system in the previous enrollment period and was not responsible for IRS reporting. The operational status of the state-based marketplace IT systems by functional category, as of February 2015 is summarized in table 5. Further, between the first and second enrollment periods, 6 of the 17 states with state-based marketplaces and state-based marketplaces using the federal marketplace IT solution changed their IT solution. In response to our survey, these states cited a variety of reasons for doing so, such as significant flaws in the system, unsuccessful system roll out, and non-working technology. The primary IT development and operations changes, as reported by the 6 states to CMS, were the following: Two states with state-based marketplaces, Oregon and Nevada, stopped development on their marketplace IT solutions and decided instead to use the federal marketplace IT solution (i.e., Healthcare.gov and related systems) for eligibility and enrollment functions. New Mexico had delays in developing and operating its marketplace and used the federal marketplace IT solution as its platform for eligibility and enrollment for the first enrollment period. For the second open enrollment, the state continued to use the federal marketplace IT solution for the eligibility and enrollment functionality and subsequently decided to continue using the federal marketplace IT solution indefinitely. Maryland changed its IT solution to one that had been successfully implemented in Connecticut for the second enrollment period. Massachusetts replaced its existing system and implemented a commercial-off-the-shelf technology solution for the second enrollment period. Idaho, which previously used the federal marketplace IT solution, developed and operated its own marketplace IT solution for the second enrollment period. According to CMS documentation regarding marketplaces using the federal marketplace IT solution, as of November 2014, 7 of 37 states using the federal marketplace IT solution could not transfer applications for health insurance coverage between their state Medicaid systems and the federal data services hub or had not completed testing or certification of these functions. Specifically, 3 of the states could not transfer—send and receive—applications for health insurance coverage between the state Medicaid and federal marketplace IT solution. The other 4 states had not completed testing and certification of those functions. CMS officials stated that the agency was continuing to work with the 7 states that had not fully implemented these functions to ensure implementation as soon as possible. In addition, as of April 2015, the transfer of applications between state Medicaid systems and the federal marketplace IT solution were not taking place in real time, and according to a CMCS official, achieving this capability is a goal for 2015 or 2016. For example, in one state, it took about 15 minutes to send applications between state Medicaid systems and the federal marketplace IT solution in either direction. In another example, a state held on to applications received and sent them at the end of the day. According to CMCS officials, states using the federal marketplace IT solution continue to focus on completing their eligibility system modernization, resolving defects, and making improvements to systems so that business processes require less manual intervention. To address the requirements of PPACA and its implementing policies, CMS engaged in various activities to oversee the states’ marketplace IT projects. In particular, the agency assigned oversight roles and responsibilities, put in place various reporting systems, and established a series of reviews that were to help ensure that states’ systems were adequately tested and functioning as intended. Nonetheless, even with these steps, CMS did not clearly document, define, and communicate its oversight roles and responsibilities to state officials, and it did not consistently involve senior executives in the review and approval of federal funding for states’ IT marketplace projects. In addition, CMS’s reviews of the states’ progress were not always effective in ensuring that systems and capabilities being developed to support the states’ marketplaces were fully tested before they became operational. States that established and operated their own (state-based) marketplaces generally used quasi-governmental entities to oversee their marketplace IT projects; they also relied on various oversight mechanisms, including executive steering committees, management change control boards, and technical review boards. Meanwhile, states with a federally facilitated marketplace or federally facilitated partnership oversaw their IT projects through existing state agencies. To oversee states’ efforts in undertaking IT projects to support the establishment and operation of their marketplaces, CMS identified numerous internal offices and groups to which it had assigned roles and responsibilities. As previously mentioned, three key offices—CCIIO, OTS, and CMCS—were responsible for overseeing states’ efforts in establishing the marketplaces. These three offices were to conduct oversight activities, such as being involved in joint grant reviews, Medicaid advanced planning document reviews, and IT gate reviews, to ensure that states followed a standardized funding process. Their primary roles and duties included the following: CCIIO led the marketplace implementation, and within that office, State Officers were assigned to be accountable for day-to-day communications with the state marketplace officials. CCIIO officials were also involved in grant funding decisions. OTS was responsible for systems integration and software development efforts to ensure that the functions of the marketplaces were carried out. A primary participant within OTS was the IT project manager, who was the individual responsible for monitoring, among other things, state-based marketplaces’ IT development activities and support for states that transitioned from one marketplace type to another. OTS officials also provided technical reviews to State Officers to inform grant funding decisions. CMCS was the office responsible for coordinating and approving Medicaid matching fund requests and implementation activities related to the state health insurance marketplaces. The office carried out these responsibilities in conjunction with CCIIO. CMCS officials identified the enrollment and eligibility specialists as the primary contacts within their office. In addition, CMS established a group called the Cross Component Committee to address marketplace-related issues across states. The committee, which included members from OTS, CCIIO, and CMCS, was tasked with overseeing the states’ progress to ensure that all marketplace requirements were aligned with CMS policy. Major policy issues identified through the committee were raised to business unit directors within the agency. CMS also informed us of other offices and groups within the agency that had roles and responsibilities for overseeing states’ marketplace IT projects. Based on written and oral descriptions of the various offices and groups, as provided by CCIIO, CMCS, and OTS officials, we compiled the information in table 6 to summarize CMS’s identified roles and responsibilities for overseeing state marketplace IT projects. In addition to establishing marketplace roles and responsibilities, CMS identified various reporting systems that were to be used to assist federal officials in overseeing state marketplace IT project funding and progress. For example, the agency relied on state marketplace information that it compiled in multiple computer systems to make funding decisions and provide technical assistance to state officials. CMS also maintained or utilized other systems that allowed states to apply for marketplace grant funding online and to transfer funds to states to establish and operate their marketplace. Additional systems allowed states to report to CMS on their grant IT expenditures; upload documentation related to their marketplace IT projects, such as project plans and testing and requirements documents; and share best practices with each other. Project management best practices emphasize the importance of clearly documenting, defining, and communicating project roles and responsibilities during the organizational planning process. During this process, to make the most effective use of the people involved with a project, best practices cite the importance of identifying, documenting, and clearly assigning project roles, responsibilities, and reporting relationships. Effective communication means that the information is provided in the right format, at the right time, to the right audience, and with the right impact. Adequate communications planning avoids problems such as delays in message delivery, insufficient communication to stakeholders, and misunderstanding or misinterpretation of the message communicated. According to best practices identified in the Project Management Institute’s Guide to the Project Management Body of Knowledge, a key document needed to ensure that communication is carried out effectively is a communications management plan. The communications management plan describes how project communications will be planned, structured, monitored, and controlled in a comprehensive document, including stakeholder communication requirements; the method of updating and refining the communications management plan as the project progresses and develops; and charts the information flow in the project. Among other things, it should include persons or groups who are responsible for communicating and receiving the information, the process and associated time frames for escalating issues that cannot be resolved at lower levels, and workflows that show the order of information authorization. In addition, according the Project Management Institute’s Guide to the Project Management Body of Knowledge, a communications management plan is a comprehensive document that contains the entire scope of the project and is updated regularly to reflect the current communication and stakeholders. However, while CMS established roles and responsibilities to help oversee marketplace activities, the agency did not always clearly document, define, and communicate marketplace IT project roles and responsibilities to the states. Despite the complexity inherent in overseeing marketplace IT project efforts across 50 states and the District of Columbia, CMS did not have a comprehensive communication plan that clearly documented and defined its state marketplace oversight structure and all the associated roles and responsibilities of key organizations and officials that were involved in state marketplace oversight. Instead, the agency’s definition and communication of roles and responsibilities were dispersed among various websites, operating procedures, and other documents, such as those we used in developing table 6. For example, roles for officials such as the CMS Administrator and Principal Deputy Administrator were located on the agency’s website, while other roles and responsibilities, such as those of the CCIIO State Officers, were described in one of the agency’s standard operating procedures. Additionally, CMS officials within CCIIO and CMCS stated that some roles and responsibilities are embedded in memorandums of agreement. Further, while the agency had documented selected stakeholder responsibilities in a matrix that CCIIO, OTS, and CMCS officials said applied to state marketplace IT projects, this document only identified responsibilities specifically associated with CMS’s development of the Healthcare.gov web portal supporting the federally facilitated marketplace and did not include all the personnel associated with oversight of the state marketplaces. Specifically, it did not identify all stakeholders that would be included in a more comprehensive communications plan developed for the management of state marketplace IT projects, including the CCIIO State Officers, the Marketplace Chief Executive Officer, and relevant state officials. The agency also provided a standard operating procedure for marketplace communications and technical assistance that contained selected CMS roles and instructions for providing technical assistance to states. However, the procedure was identified as a draft document from January 2013, and was limited to addressing technical assistance, which did not represent the full range of stakeholder and IT oversight activities. For example, the document did not identify all groups that are to receive pertinent information, a process identifying time frames and the management chain for escalating the communication of information, or workflows for issuing and disseminating guidance to states. Further, officials within CCIIO, CMCS, and OTS did not recognize certain organizations as having a role in marketplace IT activities, even though they should have done so. For example, while the officials told us that the Office of Communications does not have a role in states’ marketplace IT oversight, this office is identified as a member in the charters of key committees and boards responsible for state marketplace IT project oversight, including the Cross Component Committee, Marketplace Oversight Board, and Health Reform Operations Board. In discussing this matter, CCIIO and CMCS officials acknowledged that they had not created a comprehensive communication plan containing all relevant oversight roles and responsibilities. According to these officials, certain roles and responsibilities were not defined and documented because they were considered to be general public knowledge for which no detailed documentation was necessary. They added that, in the absence of a specific document or process, states were informed of who their points of contact were by e-mail or weekly calls. Further, these officials noted that all communications to the states were routed through the CCIIO State Officers, thus replacing the need for a comprehensive communications management plan. As previously described, CMS provided oversight and technical assistance to states in establishing their marketplaces. In responding to our survey, states with a state-based marketplace, including those using the federal marketplace IT solution, provided generally positive ratings of the clarity, completeness, and timeliness of CMS’s communication, while federally facilitated states, including federally facilitated partnerships, provided a higher rate of dissatisfaction. Similarly, state-based marketplace states provided generally positive ratings of the clarity, completeness, and timeliness of CMS’s guidance, while federally facilitated states provided a higher rate of dissatisfaction. While states with all marketplace types reported in our survey being generally satisfied with the level of CMS oversight and assistance, several states identified instances of delayed or insufficient communications with CMS. Specifically, of the 36 states that responded to our survey question regarding CMS’s overall oversight and assistance, 25 states rated it as just right, 4 rated it as more than enough, and 7 rated it as less than enough. Further, of the 17 states that provided comments, 5 spoke positively about CMS’s support and 1 spoke positively about the completeness and timeliness of CMS guidance. The remaining 11 states provided both mixed and negative comments regarding the completeness and timeliness of CMS guidance that included roles and responsibilities. For example, these states noted that they generally had experienced some type of delay in message delivery from CMS, insufficient communication with the stakeholders, and misunderstandings or misinterpretations of the messages communicated. For example, these states generally reported that they lacked complete and timely policy and business guidance from CMS, which impacted their IT development deadlines, created rework, and necessitated moving forward to develop solutions without knowing if the agency would approve or disapprove of their marketplace solutions. Overall, responses to our survey questions indicate that CMS may not have always provided the level of consistent and comprehensive communication of roles and responsibilities that is necessary to support states in effectively establishing and operating their marketplace systems. Having a comprehensive communications management plan that identifies and conveys the roles and responsibilities of key organizations and officials could be a valuable resource as states move forward on any further marketplace IT efforts. To oversee its own IT projects, such as the development of Healthcare.gov and related systems, CMS created a process called the eXpedited Lifecycle Process. This process required reviews and approvals by senior-level CMS executives, generally the Director or Deputy Director of the agency’s IT unit—OTS—and business units, including CCIIO, CMCS, and OAGM. According to the agency’s guidance, these senior-level executives should be individuals who have the authority to speak for, vote for, and otherwise make commitments on behalf of their business units. This approach is consistent with best practices in GAO’s IT investment management framework, which emphasizes the importance of having senior executive-level decision makers, such as the heads of IT and business units, involved in investment decisions. Such involvement by senior executives provides accountability for investment decisions and helps ensure that these decisions are consistent and reflect the goals of the agency. Similar to the eXpedited Lifecycle Process, CMS created its Establishment Review process, which states were required to comply with (as part of their cooperative agreements with CMS) in order to receive marketplace grant funding. The Establishment Review process is a structured grant monitoring approach that consists of multiple technical reviews for assessing the state’s progress and associated IT project documentation. States must obtain CMS approval to access restricted IT grant funds by passing technical review gates associated with the planning, design, and implementation of their projects. However, unlike the eXpedited Lifecycle Process that CMS uses to manage its own investments at the federal level, the Establishment Review process did not include representation from all relevant senior executives in the agency to review and approve the planned marketplace IT projects prior to releasing federal funding to the states. Specifically, CMS’s standard operating procedure for State Officers identified the IT and business units involved in the Establishment Review process, which included CCIIO, CMCS, OTS, and OAGM, among others. However, with the exception of the Director of CCIIO, it did not clearly require involvement by the heads of the other IT and business units involved in this process. For example: CMS did not demonstrate that senior-level executives from all relevant business and IT units were involved in the initial approval of grant awards. According to the operating procedure and officials from these business and IT units, the agency’s Objective Review Committee was tasked with reviewing state applications for federal marketplace grants. This committee consisted of subject matter experts from both inside and outside the federal government who scored applications during a review in which the State Officer participated to answer questions. The State Officer then prepared federal marketplace grant funding recommendations to OAGM and the Deputy Director of the State Exchange Group within CCIIO, who made the final decision on grant awards. However, it was unclear who these subject matter experts were or whether there were executives at the appropriate level involved with these decisions. CMS did not provide evidence that senior executives from all relevant business and IT units were involved in approving the release of restricted IT funds from marketplace grants as states progressed with their projects. According to CMS’s standard operating procedure and officials in CCIIO and OAGM, decisions to release restricted state IT funding were made by the Deputy Director of the State Exchange Group within CCIIO and OAGM grant management officers, who were responsible for reviewing and providing guidance on grant services for state marketplaces. These decisions were based on input from CCIIO State Officers, who served as primary points of contact to assigned states, and IT project managers in OTS, who were responsible for monitoring state-based marketplaces’ IT development activities. However, these officials did not hold executive-level positions. CMS did not provide evidence of executive-level involvement in the approval of Medicaid funds for marketplace IT projects. CMCS officials stated that they followed CMS’s Establishment Review process in order for states to receive Medicaid matching funds and that the approval of these funds was a coordinated effort between CCIIO and CMCS. However, they did not identify the specific officials responsible for approving these funds or provide evidence to show the approval process included senior executives from CMCS, CCIIO, and other relevant business units. CCIIO, CMCS, and OTS officials told us that they believed their Establishment Review process included the appropriate officials to review and approve state requests for federal funding. These officials added that they used their existing organizational structure to oversee decisions regarding marketplace grants and Medicaid funds. However, without the involvement of senior executives from all relevant IT units, such as OTS and business units such as CCIIO and CMCS to review and approve all federal funds invested in the state marketplace IT projects, CMS has less assurance that decisions are being coordinated among officials with a perspective across their respective business units and the agency as a whole. By ensuring such executive involvement, CMS would increase accountability for decisions to fund states’ IT projects and better ensure these decisions are well informed and make efficient use of federal funds. As part of its marketplace oversight, CMS established a process to review states’ progress on related IT projects. This framework, called the Enterprise Life Cycle, requires states to provide CMS specific artifacts supporting their projects, such as the concept of operations, system test documents, and project plans, among others. The framework focuses on incremental reviews of the projects at distinct stages, or “gates.” For each review, states are expected to show CMS an acceptable level of progress and maturity in their projects’ development before proceeding to the next project phase. Table 7 describes the various Enterprise Life Cycle gate reviews. These reviews were important because they were intended to demonstrate that the state marketplaces were ready to go live. In particular, during the operational readiness reviews, states establishing state-based marketplaces were required to demonstrate that they had met requirements, such as concluding all system testing, before the IT projects could proceed from development to operations. The Enterprise Life Cycle guidance defines this review as the agency’s determination that the state marketplace is ready to go into production. Based on these operational readiness reviews, CMS was to either approve the state’s system for operation or grant a conditional approval to proceed if the system was substantially compliant with the requirements of the review. However, the operational readiness reviews did not always meet the agency’s stated goal to ensure that states’ marketplace systems were ready for production. For the first enrollment period, CMS conducted operational readiness reviews of 15 state-based marketplaces in August and September 2013. However, CMS conditionally passed all of those states without fully ensuring that they had conducted all required system testing and demonstrated that their systems were ready for production as called for in its Enterprise Life Cycle guidance. For example, CMS documentation from these operational readiness reviews showed the following: Maryland demonstrated several eligibility and enrollment functions. However, the state had only completed approximately half of the planned user acceptance testing and had over 100 outstanding high- priority defects. In addition, almost 500 total defects had yet to be resolved. Nevada also demonstrated several eligibility and enrollment functions. However, the state had not submitted test reports for all end-to-end system testing, and user acceptance testing was in progress. The report identified 42 critical or major defects that needed to be addressed. Massachusetts demonstrated several eligibility and enrollment functions. However, the state had not completed testing and reported 1,170 open defects. Nonetheless, all state-based marketplace systems were conditionally approved and went live on October 1, 2013. Consumers in many states subsequently experienced widespread problems when using these IT solutions to apply for health insurance coverage during the first enrollment period, and in four states these problems were so severe that the states switched to a different solution. According to CMS officials, these four states implemented new marketplace IT solutions or used the federal marketplace IT solution in the second open enrollment period and successfully conducted enrollment even if some states had to create manual workarounds. However, according to CMS documentation, as of November 2014, eight states continued to have outstanding follow-up items from their operational readiness reviews that had not been addressed. In May 2015, officials in CCIIO, CMCS, and OTS stated they were actively working with these states to complete their outstanding open items. CCIIO officials further noted that the Enterprise Life Cycle gate reviews were not intended to be “pass or fail,” but to set the appropriate level of expectations for the status and progress of marketplace development and implementation and to identify areas where states may require assistance. In addition, CCIIO officials stated that, if all the milestones were not met during the gate review, they planned to conduct more frequent follow-up to improve the state’s position. They also said that although the IT component did not work for certain states, the agency granted conditional approvals because the states were able to build workarounds and put manual processes in place to allow individuals to submit applications and enroll in health coverage. Officials in OTS added that, although they made suggestions for improvements, states could choose whether or not to implement CMS’s recommendations. However, when CMS granted states conditional approval to go live, they did not ensure states’ systems had been fully tested, which is part of the structured and disciplined approach to oversight that is outlined in the agency’s Enterprise Life Cycle. By not ensuing that systems were completely tested, the agency lacked assurance that the states’ marketplace IT systems would performed as intended which, in some cases, resulted in applicants facing long waits for eligibility determinations, websites freezing midway through the process of applying for coverage, and systems being taken offline for days at a time, forcing applicants to enroll manually. The extent and manner of oversight that states exercised over marketplace IT projects depended in large part on the type of marketplace they chose to establish. For state-based marketplaces, state officials were responsible for overseeing various IT activities associated with the development and operations of their marketplaces. Specifically, states were required to oversee the planning involved with becoming a state-based marketplace. Thus, among other things, state officials were responsible for ensuring that key functionality requirements in areas such as eligibility and enrollment, plan management, consumer assistance, and financial management, were included in the development of the marketplace. Additionally, these states were responsible for overseeing contractors, who carried out various marketplace IT project-related activities, such as system integration, platform builds, project management, independent verification and validation, and security assessments. State officials were to follow CMS policy and guidance when establishing the marketplaces, including preparing project artifact deliverables, such as the marketplace concepts of operation, system test documents, and project plans. They also were to comply with financial and performance reporting requirements of CMS’s Enterprise Life Cycle process. To oversee their marketplaces, 13 of 17 states with state-based marketplaces, including those using the federal marketplace IT solution, reported on our survey that they established “quasi-governmental” entities, which were created by state legislation to oversee marketplace activities and interface with CMS to fulfill the state’s marketplace responsibilities. These entities are governed by a board made up of representatives from consumer groups and health insurance issuers, since CMS policy requires a balance of consumer and business interests on the board. The board is responsible for governance of the marketplace, making key marketplace decisions, and holding regularly scheduled meetings. By contrast, 4 of these 17 states reported on our survey that they chose to operate their marketplace through an existing state agency, such as a state department of health or Medicaid agency. If a state-based marketplace was housed within an existing state agency, then that marketplace was typically led by directors or an advisory board, and the leadership team typically reported to the governor’s office. States with state-based marketplaces, including those using the federal marketplace IT solution, reported on our survey that they also established various committees and boards to assist state officials in overseeing the marketplace’s IT funding and progress. These oversight committees and boards included steering committees, management change control boards, and technical review boards, among others. Steering committees: All states with state-based marketplaces had established this type of committee. A steering committee is to provide leadership, direction, and support for IT projects. For example, one state’s steering committee was reported to be made up of senior leadership from various agencies within the state and was responsible for ensuring that marketplace IT goals aligned with various state agencies’ goals. In addition, the committee served as a forum for project strategy development and operations, policy, and technology recommendations to its board of directors. Management change control boards: Thirteen of the 17 states with state-based marketplaces established this type of board. A management change control board is to oversee a project’s scope and requirements. For example, one state reported that its management change control board was chaired by its project director and oversaw not only changes to the scope and requirements, but also its marketplace project schedule, costs, and deliverables. Technical review boards: Nine of the 17 states with state-based marketplaces established this type of board. A technical review board provides technical findings and recommendations to project stakeholders. For example, one state reported that its technology committee provided leadership and helped to analyze the impact of the marketplace on existing IT standards and informed other teams and stakeholders about policy changes that could impact the project. In addition, 7 of the 17 states with state-based marketplaces, including those using the federal marketplace IT solution, reported on our survey that they used additional oversight mechanisms beyond these three. Specifically, one state reported that its marketplace and state administration established an integrated project management office to assist with coordination of Medicaid and tax credit applications and eligibility functions. Another state reported using a cross-agency group made up of agencies involved in marketplace eligibility functions from both IT and policy perspectives. Further, all states relying on the federally facilitated marketplace and federally facilitated partnerships that responded to our survey indicated that they used existing state agencies to oversee implementation of their marketplace IT projects. Existing state agencies included state departments of health or Medicaid agencies, which coordinated directly with CMS. In addition, these states’ officials oversaw the contractors who were responsible for various marketplace-related activities, such as building interfaces to connect the state systems to the federal data services hub for transferring information between the federally facilitated marketplace and state Medicaid programs. States encountered various challenges in their efforts to design, develop, and implement marketplace IT systems. States with state-based marketplaces reported experiencing challenges in each of five areas identified in our survey: project management and oversight, marketplace IT solution design, marketplace IT solution development, resource allocation and distribution, and marketplace implementation and operation. In addition, states with a federally facilitated marketplace reported facing challenges in two areas identified in the survey: project management and oversight and system design and development. While states operating both state-based and federally facilitated marketplace IT solutions reported in the survey that they faced similar issues, various challenges were more common for states developing their own IT solution because the scope of their efforts was larger than that of states with a federally facilitated marketplace. For example, those with state-based marketplaces generally reported experiencing issues with marketplace eligibility and enrollment functions; while for states with a federally facilitated marketplace, those functions were performed by CMS. To varying extents, states identified lessons learned and best practices from their experiences with and efforts to address the challenges. CMS was aware of state challenges and took various actions to provide technical assistance. It also has taken steps to facilitate the sharing of the lessons learned and related best practices, which will continue to be important as states work to complete the remaining functions for their marketplace systems. Compressed time frames was rated as the greatest challenge by officials of both states with a state-based marketplace and states with a federally facilitated marketplace. Specifically, 13 of 17 states with state- based marketplaces and 20 of 30 states with a federally facilitated marketplace considered compressed time frames a great or very great challenge, and it was also reported as a factor driving other challenges. State officials noted that their IT project schedules were constrained by the need to deliver functionality in time for the first enrollment period beginning on October 1, 2013. For example, one state-based marketplace official reported that compressed time frames affected the state’s development and testing time, which impacted all phases of testing (system, integration, performance, and user acceptance). Project governance, oversight, and decision making was also rated as one of the greatest challenges in the project management and oversight area by officials of both states with a state-based marketplace and states with a federally facilitated marketplace. Specifically, 10 of 17 states with state-based marketplaces and 8 of 30 states with a federally facilitated marketplace rated project governance, oversight, and decision making as a great or very great challenge. Based on our analysis of narrative survey responses, 14 states with state- based marketplaces and 15 states with a federally facilitated marketplace also identified lessons learned or best practices in the area of project management and oversight. For example, regarding compressed time frames, a best practice identified by 1 state was to double the amount of lead time normally expected when planning for implementation of complex IT projects. Another state reported a lesson learned regarding compressed time frames, which was related to IT systems design and development. This state learned that taking a two-phased approach whereby the state modified its legacy Medicaid eligibility system first, and then proceeded with a full-scale system upgrade, helped meet deadlines while avoiding significant problems that had arisen in other states. States also reported lessons learned and best practices related to project governance, oversight, and decision making. For example, one state reported reshaping its project management team and, thus, making progress for the second open enrollment season. A second state realized too late that it needed more governance and a dedicated program management office. This state’s officials also said that it was important to recognize that the marketplace is an IT project as well as an insurance project, and that it was critical to have a proper mix of both sides to ensure success. Developing interfaces and interoperability with insurers was rated as one of the greatest challenges by 9 of 17 states with state-based marketplaces. For example, 1 state reported challenges with a system that was supposed to allow users to pay for and enroll in insurance plans; however, that basic feature was not appropriately developed by launch or for months afterward. The state hired a contractor to reconcile enrollment and premium tax credit issues between its insurance carriers and its IT solution, but all issues were not resolved, and the state was still working through this process when officials responded to our survey. Developing state marketplace website eligibility functions for both state Medicaid and Qualified Health Plans was also rated as one of the greatest challenges by 9 of the 17 states. For example, one state official reported that their applicants could not have their eligibility determined for Qualified Health Plans, Medicaid, and premium tax credits without the assistance of specially trained customer service representatives or community partners and agents. Another state’s original IT solution was not working appropriately, so officials approached CMS, who offered to let the state use the Healthcare.gov platform for eligibility and enrollment. A third state cited numerous multi-stage workarounds to circumvent defects in eligibility and enrollment functionality. This included, for example, 100 percent manual validation of all enrollment files. Although 8 states with state-based marketplaces identified lessons learned or best practices in the marketplace IT solution design and development area, with one exception, states did not specifically identify lessons learned related to developing interfaces and interoperability with insurers or developing state marketplace website eligibility functions. One state reported that it learned that projects like this should begin with simple rules on eligibility, and then add complexity. Further, this state decided to maintain Medicaid and CHIP enrollees in its legacy system using a close approximation of eligibility rules to ensure that there was no disruption in coverage with the launch of a new system. New applications for Medicaid and CHIP were determined in the new system while renewals for current enrollees were determined in the legacy system. This was to enable more time for adequate testing and further development of Medicaid and CHIP rules in the new system. Conducting systems integration testing was rated as one of the greatest challenges by 12 of 30 states with a federally facilitated marketplace. For example, 1 state reported that limited development and testing time affected all phases of testing including system, integration, performance, and user acceptance testing. Another state reported that the interface between the state and the federally facilitated marketplace was delayed due to implementation delays in the federal marketplace IT solution. These delays resulted in last-minute changes to the federal systems, both known (but communicated late) and unknown. Each federal system change required the state to also change, and such changes and delays resulted in the state missing deadlines. Other states specifically cited a lack of end-to-end testing between the federal IT systems and states, as well as integrating and testing with the federal marketplace and the federal data services hub, as challenges. Changes to requirements was rated as one of the greatest challenges by 19 of 30 states with a federally facilitated marketplace. For example, one state official said that “the aggressive time frame made an impact to the design. Systems always evolve, but the aggressive schedule forced design trade-offs along the way.” A second state reported that the compressed time frame caused CMS to continually define requirements throughout implementation and into operations, resulting in the reprogramming of multiple design changes. Lastly, another state official commented on multiple challenges related to changes in requirements. This state official said that changes and delays due to clarification of CMS requirements in areas such as use of the federal data services hub and identity proofing caused significant rework and some critical functionality to be deferred, which, because of the aggressive time frame, impacted operations. A second state official emphasized developing a comprehensive set of requirements. The state invested time to develop a comprehensive set of requirements for all known areas of the system and included broad requirements referencing CMS guidance documents when detail from CMS was insufficient. The state then required vendors to explicitly identify which requirements would be met with delivered functionality, and which requirements would need to be augmented with customizations or additional software applications. This kept most of the systems development in scope and resulted in less than a 10 percent increase in the negotiated fixed price due to change orders. A third state identified a best practice regarding guidance and policy—which drive requirements— noting that they should be finalized before states are tasked with implementing system changes and testing. Our analysis of narrative survey responses showed that 14 states with a federally facilitated marketplace reported lessons learned or best practices related to IT systems design and development, including those associated with changes to requirements or the development of requirements. For example, one state official said that there were many changes leading all the way up to open enrollment. Only after this occurred did officials recognize that they needed to lock down the scope of work and disallow “nice-to-haves” to focus on critical functionality. Adequate number of staff was rated as one of the greatest challenges by 9 of 17 states with state-based marketplaces. In one case, a state official reported that the state had only approved the hiring of approximately one- third of the staff it requested and, as of October 2014, had never hired a certified project manager to oversee their state’s marketplace-related IT projects. Similarly, staffing limitations forced another state to ask its staff to work overtime, in some cases more than 60 hours a week for months on end, in order to complete the work required prior to open enrollment, resulting in burnout and the loss of key staff soon after the start of the first open enrollment period. Adequate funding to sustain a state’s marketplace system was rated as one of the greatest challenges by 6 of 17 states with state-based marketplaces. For example, one state official reported that, in order to meet open enrollment deadlines and reduce schedule risks, the state decided to use a commercial off-the-shelf product instead of open-source products, which led to an increase in life-cycle costs. Our analysis of narrative survey responses found that five states reported lessons learned or best practices related to resource allocation and distribution. For one state, the most significant lesson learned was the amount of testing resources required for all associated types of IT testing. Due to this, the state has identified a need for additional business analyst positions and subject matter expert knowledge. Call center operations was rated as one of the greatest challenges by 9 of the 17 states with state-based marketplaces. For example, one state official reported that due to challenges with system performance, their call center experienced high-traffic volume, and this affected the average time to handle a call, abandonment rates of calls, and operations. Another state reported that insufficient time for staff training led to inefficiencies in call center operations. System performance was rated as one of the greatest challenges by 7 of the 17 states with state-based marketplaces. For example, 1 state cited significant challenges in implementation and operation because its software did not work as advertised. Also, as mentioned above, system performance problems affected call center operations. This was compounded in part because of the surge in users attempting to use the online marketplace that occurred in the period immediately after going live. Our analysis of narrative survey responses showed that two states with state-based marketplaces identified best practices or lessons learned related to the operation and implementation of marketplace-related IT systems. For example, one state cited the importance of contingency planning that enabled state deployment of additional system capacity when volume exceeded expectations. Another state reported that the inability to develop and refine marketplace technology resulted in significant operational costs, which could have been avoided with a less aggressive time frame. Figure 6 summarizes the challenges in each of the five areas rated by states with state-based marketplaces. Figure 7 depicts the challenges that states with a federally facilitated marketplace rated in each of their two respective areas. CMS was aware of states’ challenges and responded to them by engaging in various outreach to and communication efforts with the states. According to CCIIO officials, once an issue or challenge was identified, CMS responded in a number of ways. Specifically, according to these officials, the agency provided technical assistance that included discussions with CMS subject matter experts to ensure that appropriate information and resources were available to address challenges. For example, the officials said they conducted site visits with state marketplace officials during which they discussed management and other issues and made recommendations for improvement, as needed. Other state challenges that CMS officials indicated they were aware of included issues with compressed schedules, state governance, legislative requirements, vendor management, personnel and resources, and call- center operations. Additionally, CMS made efforts to both directly share and facilitate the sharing of identified lessons learned and best practices among the states. CCIIO officials reported that lessons learned and best practices were shared through various methods such as discussion forums, including bi- weekly forum meetings with senior state officials, conference calls, and weekly newsletters distributed to grantees, and through various reporting and document sharing systems maintained by CMS. In taking steps to respond to state challenges, identify lessons learned, and share best practices with states, CMS performs an essential role of advising state officials and others involved with health insurance marketplace IT projects. It will be important for CMS to continue doing so as states work to complete the remaining functions for their marketplace systems. States spent approximately $1.45 billion in federal marketplace grant funds to help establish IT systems supporting their health insurance marketplaces, as well as a portion of Medicaid funds. As of the second enrollment period, states had largely established these systems, although some of their functions remain to be implemented. While CMS was tasked with overseeing states’ development of their marketplace IT systems, limitations in CMS’s efforts resulted in oversight that was not always effectively executed. Specifically, because roles and responsibilities were not always clearly defined, documented or communicated, as recommended by leading practices for project management, a number of states faced hurdles in communicating with stakeholders and receiving timely CMS guidance. In addition, although called for by leading practices in investment management, relevant senior executives in the agency were not always involved in overseeing decisions to fund states’ marketplace IT projects, resulting in less accountability for such decisions. Further, because CMS’s reviews of state IT projects did not ensure state systems were fully tested as called for in CMS’s guidance, systems were put into place that, in some cases, did not perform as intended. States also had a key oversight role, which varied depending on the type of marketplace. Finally, states reported a number of challenges and lessons learned in establishing their marketplaces, with state-based marketplaces encountering some unique challenges. CMS has taken various actions to facilitate the sharing of these challenges and lessons learned, as well as best practices among the states, and it will be important for CMS to continue these efforts as states work to complete the remaining functions for their marketplace systems. To improve the oversight of states’ marketplace IT projects, we recommend that the Secretary of Health and Human Services direct the Administrator of the Centers for Medicare & Medicaid Services to take the following three actions: clearly document, define, and communicate to all state marketplace officials and stakeholders the roles and responsibilities of those CMS officials involved in overseeing state marketplaces in a comprehensive communication management plan; ensure that all CMS senior executives from IT and business units who are involved in the establishment of state marketplace IT projects review and approve funding decisions for these projects; and ensure that states have completed all testing of marketplace system functions prior to releasing them into operation. We received written comments on a draft of this report, signed by HHS’s Assistant Secretary for Legislation. In the comments (reprinted in appendix III), the department stated that it concurred with all three of our recommendations. The department added that it had taken various actions that were focused on improving its oversight and accountability for states’ marketplace efforts. While the actions discussed are important, the department did not always identify specific activities being taken or planned that would address the full extent of the recommendations. Specifically, with respect to our recommendation that CMS clearly document, define, and communicate its roles and responsibilities for overseeing state marketplaces in a comprehensive communication management plan, the department noted that a State Officer is assigned to each state to serve as the primary point of contact and that CMS’s roles and responsibilities are communicated through this official. The department also stated that these roles and responsibilities are documented in several resources, including standard operating procedures and weekly newsletters to state officials. However, the department did not indicate that CMS would develop a communications management plan to provide a comprehensive and consistent means of identifying and conveying the roles and responsibilities of key CMS organizations to all states and the District of Columbia. As we noted in our report, CMS’s standard operating procedures and other documents did not identify all the relevant stakeholders or activities involved in its oversight process. Thus, we maintain that a comprehensive communications management plan would be a valuable resource as states move forward on any further marketplace IT efforts. With respect to our recommendation that CMS include senior executives from all relevant IT and business units in funding decisions for state marketplace IT projects, HHS stated that the department already includes senior executives in its funding decisions for these projects. However, as noted in our report, CMS did not provide evidence that key senior executives from CCIIO, CMCS, and OTS were involved in various funding decisions associated with the states’ IT projects. For example, CMS did not demonstrate that senior-level executives from all relevant business and IT units were involved in the initial approval of grant awards or the release of restricted IT funds from marketplace grants as states progressed with their projects. In addition, CMS did not provide evidence of senior executive involvement in the approval of Medicaid funds for marketplace IT projects. By ensuring such executive involvement, CMS would increase accountability for decisions to fund states’ IT projects and ensure that these decisions are well informed in order to make efficient use of federal funds. With respect to our recommendation to ensure that states have completed all testing of marketplace system functions prior to releasing them into operation, HHS noted that it will continue to follow its guidelines to determine if state marketplace system functions are ready for release. The department added that it will continue to work closely with state- based marketplaces to improve their systems and verify that system requirements are met. We agree that following its review guidance as defined is important. In particular, as noted in our recommendation, CMS should ensure that states’ systems are fully tested before approving them for release into production, rather than relying on workarounds and manual processes. HHS also provided technical comments, which we incorporated in the report as appropriate. Among these comments, the CMS liaison in the Office of Legislation sent an e-mail on September 10, 2015, stating that the amount of total marketplace grant spending for the District of Columbia that CMS provided to us based on its March 2015 report was incorrect. Accordingly, we revised our analysis and relevant areas of our report to reflect the new amount provided by the agency. We also provided relevant excerpts of this report to each of the 50 states and the District of Columbia and received responses, via e-mail or in writing, from officials in 15 states. Officials from 5 of these states (Alaska, Arizona, Maine, Nevada, and Rhode Island) said they had no comments. Among the remaining 10 states, 6 states (Alabama, Idaho, Indiana, Minnesota, Washington, and Wisconsin) commented on our discussion of their marketplace grant data. According to these states, the data we reported on marketplace grant funding were not always consistent with their own data. However, the grant funding discussed in our report reflects state-reported data that CMS provided and represents a consistent source and time frame of data for all states as of March 2015; thus, we did not revise our discussion of the reported data in the report. However, we did revise the report to clarify that the state-reported data that CMS provided could lag behind actual state marketplace grant data for a specific date. In addition, officials from 6 of the 10 states commented on the status of their systems development and operation. In e-mail comments, the Grant Compliance Officer of Covered California provided details on specific functionality Covered California was still implementing. For example, its small business marketplace was using manual workarounds for its automated payment functionality until the system is completed. Regarding the hub services and IRS reporting submission functions, the official said that California will continue to enhance and improve efficiencies of the hub services for the health insurance renewal process, and will complete performance testing of IRS reporting submissions. In written comments, the Executive Director of the District of Columbia Health Benefit Exchange Authority did not agree with some of the characterizations in our report. Specifically, the Executive Director concurred with our characterization of the status of the financial management functions as fully complete and IRS reporting functions as partially complete, but did not agree that the District of Columbia’s eligibility and enrollment and hub services functions were only partially complete. Regarding the eligibility and enrollment functions, the Executive Director said that our characterization was misleading and unsupported because these functions were only partially operational for one specific function and that the marketplace received permission from CMS to implement an alternate method for implementing another specific function; thus, the overall eligibility and enrollment function should have been considered fully operational. Our characterization of eligibility and enrollment functions as partially operational was based on CMS’s February 2015 operational status report which consisted of a larger list of functions than the Executive Director cited and states were expected to automate all these functions. While we recognize that the District of Columbia was able to enroll applicants through its system, CMS’s report indicated that these specific functions, which support important provisions of PPACA, were not complete or fully automated. Regarding hub services, the Executive Director said that the District of Columbia requested and received permission from CMS not to deploy a specific function for plan year 2015 but has begun testing this function for plan year 2016. Since the District of Columbia was still testing this hub service, it had not fully developed, tested, and implemented this functionality required by CMS. The District of Columbia Health Benefit Exchange Authority’s comments are reprinted in appendix IV. In e-mail comments, the Executive Director of the Office of the Kentucky Health Benefit Exchange requested that we clarify the partial rating for IRS required submissions because the Executive Director believed that the state had been fully compliant with these requirements. However, according to CMS’s February 2015 operational status report, Kentucky had not completed the most recent annual submission of IRS data which is used to ensure that individuals received the correct amount of premium tax credit. In written comments, the Interim Chief Executive Officer of MNsure, the Minnesota marketplace, generally agreed with the operational status ratings for the functional categories. But the official also noted that while the functions may be rated as partially operational, our report did not recognize that MNsure delivered the required services and in some cases used manual workarounds to temporarily meet the functional requirements. We recognized that states implemented workarounds to deliver services, but our report focuses on the status of fully automated functionality delivered by states’ IT projects. For example, regarding eligibility and enrollment functions, although MNsure sent automated notices for most consumers, due to system limitations it was unable to issue automated notices to some consumers renewing coverage and therefore created manual notices for these consumers. In addition, regarding financial management functions, the Interim Chief Executive Officer said MNsure was billing small business customers using a manual process in February 2015, but has since incorporated automation into the process. Further, the official noted that MNsure opted to have certain financial management functions performed by CMS. While MNsure made progress in this area, we are reporting the status according to CMS’s February 2015 operational status report, which is a consistent source and time frame of data for all states, and these financial management functions were categorized as not operational in the report. Regarding hub services, the Interim Chief Executive Officer generally agreed with the status and stated that MNsure will continue to plan for testing of these functions. Regarding IRS reporting, the official generally agreed with the status and stated that the delays for submitting files to IRS were due to additional quality assurance work. The MNsure Minnesota marketplace’s comments are reprinted in appendix V. In e-mail comments, the Deputy Director of the New York State Department of Health disagreed that financial management, hub services and IRS reporting file submissions functions were partially operational as of February 2015, and believed that the ratings should reflect fully operational or fully complete. In addition, the Deputy Director stated that the state should not receive partial ratings because it opted to have CMS perform certain financial management functions, determined alternate methods for completing certain hub services functions, and was waiting for solutions from CMS regarding IRS reporting file submissions. Although New York opted to have certain financial management functions performed by CMS, the agency’s February 2015 operational status report categorized these functions as not operational. Further, while CMS may have allowed certain alternate methods or workarounds for hub services functions, CMS’s operational status report indicated that these specific functions were not complete or fully automated. Even though New York may have been waiting for a solution from CMS to complete its IRS reporting file submissions, CMS’s report noted that this function was not fully complete. In written comments, the Chief Executive Officer of the Washington Health Benefit Exchange concurred with our characterization of the status of eligibility and enrollment functions and IRS reporting file submissions but did not agree that its financial management and hub services functions were only partially operational. The Chief Executive Officer stated that our report lacked the necessary details for him to review in order to respond to these characterizations. We later provided details from CMS’s February 2015 operational status report that we evaluated to determine the status of the state’s marketplace. Subsequently, the official stated that certain financial management functions were incomplete because the state opted to have these functions performed by CMS. Nonetheless, CMS’s February 2015 operational status report categorized these functions as not operational. For hub services, the official noted that the Washington Healthplanfinder successfully used multiple services offered by the federal hub to verify Social Security numbers, citizenship, lawful presence, income, and other eligibility factors and that the marketplace has tested these services. However, CMS’s February 2015 operational status report noted that it had only partially completed certain hub services for verifying eligibility. The Washington Health Benefit Exchange’s comments are reprinted in appendix VI. Other technical comments provided via e-mail by marketplace and Medicaid officials within these states were considered and incorporated into our final report as appropriate We are sending copies of this report to the Secretary of Health and Human Services and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. Should you or your staffs have questions on matters discussed in this report, please contact me at (202) 512-6304. I can also be reached by e- mail at melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VII. Our objectives were to (1) determine how states have used federal funds for IT projects to establish, support, and connect to health insurance marketplaces, including amounts spent, and the overall status of their development and operation; (2) determine CMS’s and states’ roles in overseeing these state IT projects; and (3) describe IT challenges that states have encountered in developing and operating their marketplaces and connected systems, and lessons learned from their efforts. To address the three objectives, we designed and administered a web- based survey to collect information about the state health insurance marketplace IT projects in the 50 states and the District of Columbia. We developed two versions of this survey: one for states with state-based marketplaces, including those using the federal marketplace IT solution, and one for states with a federally facilitated marketplace or federally facilitated partnership. Seventeen states received the state-based version of the survey, and 34 states received the federally facilitated version. Generally, the survey asked state program officials about federal and state funding for developing and operating state marketplace-related IT projects, state marketplace and project types, CMS’s and state’s marketplace oversight roles and tools, and challenges and lessons learned with state marketplace IT development and operations. Out of the original population of state health marketplaces in the 50 states and the District of Columbia, 46 states and the District of Columbia submitted survey responses; however, not all respondents provided answers to every question. We did not independently verify the data the states provided in each case, but we did, in selected cases, compare them to equivalent CMS data. We also relied on CMS-provided data, rather than survey data, in most cases because we received more up-to- date and complete information from CMS. The survey was administered between September 30, 2014, and November 19, 2014. The status of state marketplace types is as of the end of the second enrollment period—which ended on February 15, 2015. Several weeks before the survey period began, we notified recipients that they would be receiving it and confirmed that they were the appropriate state contacts. We also followed up with non-respondents several times before the survey period ended. In developing the surveys, we took steps to ensure the accuracy and reliability of responses. We pre-tested the survey with marketplace and Medicaid officials from seven states to ensure that the questions were clear, comprehensive, and unbiased, and to minimize the burden the questionnaire placed on respondents. To determine how states have used federal funds to establish, support, and connect to health insurance marketplaces and the overall status of their development and operation, we reviewed CMS guidance regarding federal funding and development for marketplaces such as the marketplace grant funding opportunity announcement, instructions for marketplace reporting, guidance for marketplace and Medicaid IT systems, and blueprint guidance for approval of state marketplace types. We also reviewed best practices for IT investment management and managing program costs. We then reviewed CMS funding and status documentation, including notices of grant awards and state IT spending and status summaries. We also analyzed state survey responses on costs and development status, including state documentation on federal grant and Medicaid costs. To assess the reliability of CMS’s data on state-reported IT spending to establish, support, and connect to marketplaces, we assessed the reliability of the systems used to collect the information. We asked officials responsible for entering and reviewing the grants information a series of questions about the accuracy and reliability of the data. Among the sources of data used for our study, we reviewed a spreadsheet compiled by CMS Center for Consumer Information and Insurance Oversight officials that contained state-reported grant funding data and marketplace IT project status information drawn from three separate information systems: CMS’s On-Line Data Collection System, Grant Solutions, and the Payment Management System. The spreadsheet was a consistent source of information that reflected the same cost factors for all states as of March 2015. Specifically, the spreadsheet tracked, among other things, the type and total amount of grant funding provided and available to each state, as well as the time period for expending those funds. We also reviewed the data to determine if there were any outliers and other obvious errors in the data. For any anomalies in the data, we followed up with CMS to either understand or correct those anomalies. We determined that the data were sufficiently reliable for our purposes and noted any limitations in our report. While our report discusses state-reported IT spending based on CMS data, we did not verify the accuracy of the data states reported to CMS. We also reviewed our recent report on Medicaid funding for eligibility IT system changes, which addressed state-reported Medicaid expenditure data from CMS-64—a form that states complete quarterly to obtain federal reimbursement for services provided or administrative costs incurred. We updated our review of states’ reported expenditures, beginning with the quarter ending June 30, 2011, the first quarter for which 90/10 funds were available to states, through the quarter ending December 31, 2014. To determine the reliability of the CMS-64 data, we reviewed related documentation and our prior records of interviews with CMS officials describing how these data are collected and processed; we also examined other research that has used these data to report state expenditures. We determined that the data we used in this report were sufficiently reliable and noted any limitations in our report. In addition, we reviewed and analyzed CMS documentation of states’ marketplace status and operation progress and challenges to summarize the status of marketplaces. We reviewed states’ survey responses regarding changes in and the status of developing and operating their marketplace IT solutions. We also reviewed CMS state marketplace operational status reports as of February 2015 and the CMS State Exchange Resource Tracking System as of April 2015. We did not independently verify the accuracy of CMS’s data on states’ operational status. We also obtained input from CMS regarding funding and status of marketplaces through interviews with knowledgeable officials. To determine CMS’s and states’ roles in overseeing these state IT projects, we analyzed applicable federal laws and regulations, CMS marketplace policies and guidance, documentation on applicable CMS marketplace roles and responsibilities and state marketplace governance structures, state survey responses regarding their governance structures, and state survey responses and ratings regarding the effectiveness of CMS guidance, oversight, and related systems. We also compared CMS’s policies and procedures to best practices included in GAO’s IT investment management framework and to the Project Management Institute’s A Guide to the Project Management Body of Knowledge (PMBOK® Guide) to determine whether CMS had roles and responsibilities clearly documented and communicated in its policies and procedures. Further, we reviewed CMS’s funding oversight processes and compared them to relevant sections of GAO’s IT investment management framework to determine if CMS followed best practices for overseeing IT investments. We used our survey results to describe how the states viewed CMS’s oversight and guidance in regard to the marketplace-related IT projects. We also reviewed CMS’s Enterprise Life Cycle guidance for systems development reviews and reports from states’ operational readiness reviews from August and September 2013 to assess the extent to which CMS followed its process. In addition, we reviewed state survey responses and other state-provided documents to determine states’ marketplace oversight roles. Further, we interviewed CMS officials responsible for the oversight and implementation of the state marketplaces to obtain their perspective on their marketplace roles. To describe IT challenges encountered in developing and operating the marketplace and connected systems as well as lessons learned from these efforts, we analyzed state survey responses related to challenges, lessons learned, and best practices identified by state officials and documentation such as CMS meeting presentations. For the state surveys, we identified a variety of marketplace-related IT challenges based on our analysis of CMS and state documentation and interviews, and grouped these challenges according to several broad areas. State- based marketplace challenges were divided into five areas in the survey (project management and oversight, marketplace IT solution design, marketplace IT solution development, resource allocation and distribution, and marketplace implementation and operation), while federally facilitated challenges were divided into two areas (project management and oversight and system design and development) based on the IT work each marketplace performs. For the purposes of our report, we consolidated the marketplace IT solution design and marketplace IT solution development challenge areas for the state-based marketplaces. In both the state-based and federally facilitated versions of our survey, we asked states to rate their experience with each of these identified challenges using a 5-point scale with the following response options: very great challenge, great challenge, moderate challenge, somewhat of a challenge, or little or no challenge. In our report, we combined the very great and great state ratings. We then analyzed states’ ratings of challenges and using counts of the “very great” and “great” responses, we selected the greatest (i.e., the top two) challenges from each area for discussion in this report. If a challenge area applied to both states using a state-based marketplace and states with a federally facilitated marketplace, the greatest challenges from each marketplace type were selected. Further, we asked each state to identify whether they had identified best practices or lessons learned within each challenge area of our survey, and to include specific examples of those best practices and lessons. We reviewed all written survey responses regarding states’ lessons learned to ensure these lessons were appropriately categorized into each identified challenge area. Based on our qualitative analysis of the states’ survey responses, we identified the number of states that provided lessons learned and then provided examples of the best practices or lessons learned that related to the greatest challenges in each area, if there were any. We also interviewed CMS and state officials responsible for the oversight and implementation of the state marketplaces to determine what the agency did to identify and share states’ challenges, best practices, and lessons learned. We conducted this performance audit from April 2014 to September 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. To help states establish a marketplace, the Patient Protection and Affordable Care Act (PPACA) authorized the Department of Health and Human Services (HHS) to award federal exchange (now referred to as marketplace) grants for planning and implementation activities, as well as for the first year of a marketplace’s operation. States were required to report marketplace grant spending, including IT spending, to HHS’s Centers for Medicare & Medicaid Services (CMS). The following table shows the amount of marketplace grants awarded; the amount of grants spent or drawn down; the amount authorized for IT; and the amount spent for IT as of March 2015, for the four different marketplace types—state-based, state-based using the federal marketplace IT solution, federally facilitated, and federally facilitated partnership marketplaces. In addition to the contact named above, Tammi Kalugdan (assistant director), Christie Motley (assistant director), Christopher Businsky, Debra Conner, Sandra George, David Hong, Kendrick Johnson, Lee McCracken, Monica Perez-Nelson, Jerome Sandau, Brandon Sanders, Andrew Stavisky, Karin Wallestad, Merry Woo, and Elizabeth Wood made key contributions to this report.

Summary: The Patient Protection and Affordable Care Act required the establishment of health insurance exchanges—or marketplaces—to allow consumers and small employers to compare, select, and purchase health insurance plans. States can elect to establish a state-based marketplace, or cede this authority to CMS to establish a federally facilitated marketplace. To assist states in establishing their marketplaces and supporting IT systems, federal funding was made available, including grants and Medicaid matching funds. CMS has responsibilities for overseeing states' use of these funds and the establishment of their marketplaces. The objectives of this study were to (1) determine how states have used federal funds for IT projects to support their marketplaces and the status of the marketplaces, (2) determine CMS's and states' roles in overseeing these projects, and (3) describe IT challenges states have encountered and lessons learned. To do this, GAO surveyed the 50 states and the District of Columbia, reviewed relevant documentation from the states and CMS, and interviewed CMS officials. States reported to the Department of Health and Human Services' (HHS) Centers for Medicare & Medicaid Services (CMS) that they spent about $1.45 billion in federal marketplace grant funding on information technology (IT) projects supporting health insurance marketplaces, as of March 2015. The majority of this spending was for state-based marketplaces (i.e., marketplaces established and operated by the states). These marketplaces reported spending nearly 89 percent of the funds on IT contracts, and CMS has ongoing efforts to track states' IT spending in more detailed categories. States also reported spending, as of December 2014, $2.78 billion in combined federal and state funds designated for Medicaid eligibility and enrollment systems—a portion of which was used for marketplace IT projects. However, the specific amount spent on marketplace-related projects was uncertain, as only a selected number of states reported to GAO that they tracked or estimated this information. Regarding the status of states' marketplace IT projects, 14 states with state-based marketplaces had developed and were operating IT systems to support their marketplaces, but, as of February 2015, not all system functions were complete. In addition, as of November 2014, 7 of 37 states using the federal marketplace system could not transfer health insurance applications between their state Medicaid systems and a key component of the federal marketplace or had not completed testing or certification of these functions. According to CMS officials, states operating their own IT systems and states using the federal marketplace system were continuing to improve the development and operation of their marketplaces in the enrollment period that began in November 2014. CMS tasked various offices with responsibilities for overseeing states' marketplace IT projects. However, the agency did not always clearly document, define, or communicate its oversight roles and responsibilities to states as called for by best practices for project management. According to some states, this resulted in instances of poor communication with CMS, which adversely affected states' deadlines, increased uncertainty, and required additional work. CMS also did not involve all relevant senior executives in decisions to approve federal funding for states' IT marketplace projects; such involvement, according to leading practices for investment management, can increase accountability for decision making. Further, while CMS established a process that required the testing of state marketplace systems to determine whether they were ready to be made operational, these systems were not always fully tested, increasing the risk that they would not operate as intended. For their part, states oversaw their IT projects through state agencies or quasi-governmental entities, depending on marketplace type, as well as using other oversight mechanisms. States reported a number of challenges in establishing the systems supporting their marketplaces. These fell into several categories, including project management and oversight, system design and development, resource allocation and distribution, and marketplace implementation and operation. States also identified lessons learned from dealing with such challenges, including the need for strong project management and clear requirements development. CMS has taken various actions to respond to state challenges, identify lessons learned, and share best practices with states; continuing these efforts will be important as states work to complete their marketplace systems. GAO is recommending that CMS define and communicate its oversight roles and responsibilities, ensure senior executives are involved in funding decisions for state IT projects, and ensure that states complete testing of their systems before they are put into operation. HHS concurred with GAO's recommendations.