Information Technology: Centers for Medicare &amp; Medicaid Services Needs to Establish Critical Investment Management Capabilities
Published: 2005-10-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: To carry out its mission of ensuring health care security for beneficiaries, the Centers for Medicare & Medicaid Services (CMS) relies heavily on information technology (IT) systems. In fiscal year 2005, CMS's total IT appropriations was about $2.55 billion, of which about $760 million, or 30 percent, was to support internal investments, and $1.79 billion was to fund the Medicaid Management Information Systems (MMIS) that states use to support their Medicaid programs. (GAO is using the term "internal" to refer to all of CMS' IT investments excluding state MMISs.) In light of the size and significance of these investments, GAO's objectives were to (1) evaluate CMS's capabilities for managing its internal investments, (2) determine any plans the agency might have for improving these capabilities, and (3) examine CMS's process for approving and monitoring state MMISs.
section_title: What GAO Found
paragraphs: Judged against GAO's framework for IT investment management, which measures the maturity of an organization's investment management process, CMS's capabilities for effectively managing its internal investments are limited. Specifically, the agency has established a little over half of the foundational practices it needs to manage individual investments and has executed 2 of the 27 key practices needed to manage investments as a portfolio. Until CMS fully establishes foundational and portfolio-level practices, executives will lack the assurance that they are managing the agency's collection of investments in a manner that minimizes risks and maximizes returns. CMS has initiated steps to improve its investment management process; however, these steps do not fully address the weaknesses GAO identifies in this report, nor are they coordinated with other needed improvement efforts into a plan that (1) is based on an assessment of strengths and weaknesses; (2) specifies measurable goals, objectives, and milestones; (3) specifies needed resources; (4) assigns clear responsibility and accountability for accomplishing tasks; and (5) is approved by senior-level management. Without such a plan and procedures for implementing it, CMS will be challenged in sustaining the commitment it needs to fully establish its investment management process. The process for approving requests for federal funding of MMIS activities (including development, operations, and maintenance activities) is characterized by standard procedures, guidance, and reported information to CMS's Center for Medicaid and State Operations. In contrast, the process for monitoring MMIS activities lacks standard procedures, guidance, and reporting requirements. Without these elements for monitoring MMIS activities, CMS may not be able to easily determine whether the state MMISs in which CMS invests close to $1.7 billion annually are facilitating the delivery of Medicaid benefits in the most effective and beneficial manner.

section_title: Letter
paragraphs: The Centers for Medicare & Medicaid Services (CMS), formerly called the  Health Care Financing Administration, within the Department of Health  and Human Services (HHS), is responsible for overseeing the Medicare and  Medicaid programs. In 1990, we designated the Medicare program as high- risk, in part, because of its sheer size and complexity. Similarly, in 2003, we  placed the Medicaid program on our high-risk list, noting the growing  concerns about the quality of fiscal oversight. In our latest high-risk series,  issued in January 2005, we continued to designate both these programs as  high risk. While the Medicare program is financed and administered by the  federal government, the Medicaid program is jointly financed by the federal  government and the states and is administered directly by the states.
To carry out its responsibilities, CMS depends on hundreds of information  technology (IT) systems to maintain information on Medicare  beneficiaries, providers, and medical services provided as well as to carry  out its oversight of the states’ Medicaid programs for low-income  Americans. For example, IT systems support the Medicare program, which  enrolls about 41 million elderly and disabled beneficiaries and, in fiscal  year 2004, had estimated outlays of $297 billion in health care benefits. The  agency also provides funding assistance (through grants) to the states to  develop and operate automated systems, known as Medicaid Management  Information Systems (MMIS), to support their Medicaid programs. While  the responsibility for managing CMS’s internalIT investments falls to its  Information Technology Investment Review Board, the responsibility for  approving requests for federal funding of state MMIS activities and for  monitoring these activities falls to CMS’s Center for Medicaid and State  Operations and the 10 regional offices. For fiscal year 2005, CMS’s total IT  appropriations was about $2.55 billion, of which about $1.79 billion, or 70  percent, was to be used to support Medicaid state IT investments.
This report is one of two we prepared in response to your request that we  review HHS’s and CMS’s IT management processes. It focuses on CMS’s  processes for making IT investment management decisions and evaluates  how well these processes compare with the accepted practices presented  in our IT Investment Management framework. This framework provides a  method for evaluating and assessing how well an agency is selecting and  managing its IT resources. As we agreed with your office, our objectives  were to (1) evaluate CMS’s capabilities for managing its internal IT  investments, (2) determine any plans the agency might have for improving  these capabilities, and (3) examine CMS’s processes for approving and  monitoring the state MMISs it funds. To address these objectives, we  analyzed documents and interviewed agency officials to (1) validate and  update CMS’s self-assessment of key practices in the framework, (2)  evaluate the agency’s plans for improving its capabilities, and (3) examine  CMS’s processes for approving and monitoring the state MMISs. We  performed our work from January 2005 through September 2005 in  accordance with generally accepted government auditing standards.  Appendix I contains further details on our objectives, scope, and  methodology.
subsections: section_title: Background
paragraphs: CMS has become the largest purchaser of health care in the United States,  serving nearly 83 million Medicare and Medicaid beneficiaries. The agency  administers the Medicare program, enacted in 1965, which provides health  insurance to people who are aged 65 years and over and to some people  with disabilities who are under aged 65 years. The agency also works with  the states to administer the Medicaid program, enacted in 1965 as a jointly  funded program in which the federal government matches state spending  according to a formula to provide medical and health-related services to  low-income Americans.
In fiscal year 2005, CMS will reportedly spend about $519 billion: 63 percent  for Medicare, 35 percent for Medicaid and Medicaid administration, and the  remaining 2 percent for the State Children’s Health Insurance Program and  other administrative costs. CMS estimates that its total budget in fiscal year  2006 will be $622 billion.
The agency carries out its responsibilities from its national headquarters  located in Baltimore, Maryland, and its 10 regional offices located  throughout the nation. It is organized around three centers (to support its  key functions): the Center for Medicare Management, the Center for  Beneficiary Choices, and the Center for Medicaid and State Operations.  Numerous other offices throughout the agency support these centers.
subsections: section_title: CMS’s Use of Information Technology
paragraphs: IT systems play a vital role in helping CMS to fulfill its responsibilities in  carrying out the Medicare and Medicaid programs. These systems help to  maintain Medicare information on the millions of beneficiaries, providers,  and medical services provided. For example, CMS’s Medicare Fee-for- Service claims processing systems process more than 1 billion claims  annually and make benefit payments for the 41 million elderly and disabled  beneficiaries. In fiscal year 2004, the Medicare program had estimated  outlays of $297 billion in health care benefits.
Similarly, IT systems are relied on to manage the Medicaid program. In  fiscal year 2003, this program provided benefits totaling about $261 billion  to nearly 54 million people. Of this amount, the federal share was about  $153 billion.
To assist the states in developing and operating MMISs used to process  Medicaid claims and administer the program, CMS provides funding  assistance through grants. In fiscal year 2005, about $1.79 billion, or 70  percent, of CMS’s nearly $2.55 billion total appropriations for IT went to  support Medicaid state investments. The remaining approximately $0.76  billion, or 30 percent, was used for CMS’s internal investments. Figure 1  shows the breakdown of this funding between CMS’s internal IT  investments and Medicaid state IT investments.
section_title: Weaknesses Previously Identified in CMS’s IT Investment Management Processes
paragraphs: In September 2001, we reported that CMS’s processes for managing its IT  investments omitted key review, approval, and evaluation steps. We  recognized that the agency was making efforts to strengthen its IT planning  and had developed guidance for an improved management process, but  stated that it would need to make considerable progress in implementing  these changes to ensure that its ongoing modernization efforts stayed on  track. To improve its investment management processes, we made several  recommendations to the CMS Administrator, including establishing  sufficient and written criteria to ensure a consistent process for funding IT  projects agencywide, and establishing a systematic process for evaluating  completed IT projects that included cost, milestone, and performance data.
section_title: CMS’s Approach to Investment Management
paragraphs: Several groups and individuals play a role in CMS’s process to manage its  internal IT investments, including an investment board for establishing the  IT investment governance principles. However, a different process is used  to oversee the Medicaid IT systems that the agency jointly funds with the  states. This process is carried out by CMS’s Center for Medicaid and State  Operations and 10 regional offices. Both of these processes, along with the  roles and responsibilities of the groups and individuals involved, are  described below.
section_title: Process for Managing Internal Investments
paragraphs: The groups and individuals who play a role in CMS’s internal IT investment  management process include the Information Technology Investment  Review Board, Executive Steering Committees, Enterprise Architecture  Group, and Component Leads.
Information Technology Investment Review Board (ITIRB). This  board was established in January of 2004 to provide a corporate  perspective in evaluating IT investments against CMS’s business  priorities. Its members consist of senior leadership from CMS centers,  offices, and regional offices, and it is chaired by the agency’s Chief  Information Officer (CIO). Initially, the primary ITIRB responsibility was  overseeing investments associated with the Medicare Presciption Drug,  Improvement, and Modernization Act of 2003 (MMA) and with CMS’s  revitalization initiative. These investments made up about one-third of  CMS’s fiscal year 2005 Operating Plan for internal systems. In the spring  of 2005, the role of the board was expanded to include all internal IT  investments. To assist the ITIRB in its activities, CMS staff from the  Office of Information Services and the Office of Financial Management  provide administrative support. According to its charter, the board is  responsible for   establishing the criteria for the selection, control, and evaluation of  CMS’s portfolio of IT projects;   developing the agency’s IT operation plan and responding to the  President’s budget request;  reviewing the performance of IT investments using the criteria and  checkpoints in meeting cost, schedule, risk, and benefit expectations  and taking corrective actions when expectations are not being met;   ensuring that IT investments in operation are periodically evaluated  to determine whether they should be retained, modified, replaced, or  terminated; and   comparing the results of implemented investments with the  expectations that were set for them and developing a set of lessons  learned for future process improvement.
Executive Steering Committees (ESC). The ESCs were established to  support the ITIRB in carrying out its responsibilities. Each ESC is  responsible for managing IT projects (or investments) that are grouped  together into a portfolio for each of CMS’s business components. This  responsibility includes maintaining the appropriate mix of IT  investments in its portfolio, managing the investments in its portfolio,  and providing funding recommendations to the ITIRB for these  investments. The membership of each ESC depends on the IT  investments contained in the portfolio, but, at a minimum, every CMS  component that sponsors a project is to have a representative on the  ESC.
Enterprise Architecture Group. This group, formally known as the IT  Architecture Planning Staff, supports the IT investment management  process, by, among other things, reviewing business case analyses for  new investments and major enhancements to ensure that they are  consistent with the enterprise architecture, by making  recommendations based upon that review that are aimed at the optimal  leveraging of assets.
Component Leads. These individuals provide support in the IT  investment management process by serving as liaisons between the  Office of Information Services and individual project managers.  Component Leads are to assist project managers in understanding  CMS’s investment management process and other operational policies  and processes. They can also provide project managers with key  contacts for various IT services that project owners may require during  implementation of a project.
In the spring of 2005, CMS implemented a new budget formulation process  and used it to select its IT investments. This process begins with an  information request from the CIO asking that each component submit  information on all of its investments, both new and ongoing. This  information is to include (1) a score sheet for each investment that shows  how it compared with prescribed criteria, such as alignment with business  drivers and IT strategic goals, and (2) a prioritized list of all investments for  the component. For new investments, the components also are to submit  an IT Fact Sheet (an investment proposal) that the ITIRB support staff; the  Enterprise Architecture Group; and, ultimately, the board review to  determine if the need for the new investment is justified. If the need is  found to be justified, project managers receive funding to develop a  Business Case Analysis (smaller projects may not require such a  document), which goes through the same review process as the IT Fact  Sheet.
The ITIRB support staff review all information submitted in response to the  information request and prepare it for the ESCs’ review. The ESCs  reevaluate the investments against the criteria, making adjustments to the  scoring if necessary, and make funding recommendations to the board. The  ITIRB makes strategic and funding recommendations regarding CMS’s IT  capital investment portfolio to CMS’s Chief Operating Officer who, in turn,  provides recommendations to the CMS’s Office of Financial Management  for integration into the agency’s overall budget. Figure 2 illustrates CMS’s  process for selecting its internal IT investments.
To date, the ITIRB’s role in controlling (overseeing) IT investments has  been primarily limited to those associated with the MMA and revitalization  initiatives. According to CMS officials, efforts to define procedures for the  board to control all internal investments, in accordance with the  responsibilities described in its charter, are currently under way.
subsections: section_title: Process for Approving and Monitoring State Medicaid IT Investments
paragraphs: The ITIRB plays no role in approving and monitoring state Medicaid IT  investments. Instead, the process for approving states’ requests for  matching funds for MMIS activities—including the design, development,  and installation of new MMISs, and the operations, maintenance, and  enhancement of existing MMISs—is the shared responsibility of CMS’s  Center for Medicaid and State Operations (hereafter referred to as the  central office) and its 10 regional offices. According to regulations, the  State Medicaid Manual, and officials we interviewed at CMS’s central  office and 5 regional offices, CMS’s process for approving states’ requests  generally consists of the activities discussed below:   To request federal funds for state MMIS activities, states must prepare  an advance planning document (APD), which identifies, among other  things, the purpose, scope, benefits, and preliminary cost estimates for  the activities they want to undertake. States submit this document to the  regional office, which reviews the APD for completeness and technical  content. Regional office staff generally ensure that requests support the  Medicaid program, are in compliance with federal requirements, and  represent cost-effective solutions. Also, the regional office may have  suggestions for the states to improve their APDs. Some of the officials  we interviewed told us that they work with the states to complete the  APDs to expedite the review and approval process.
Once regional office staff determine that an APD adequately justifies the  request for funding and the request is approved by that regional office’s  Associate Regional Administrator for Medicaid, the CMS central office  and HHS are notified of the approval through a process referred to as  the Office of the Secretary Notice process. Once the central office  concurs, the regional office can send an approval letter to the state.
The states typically hire contractors to perform the MMIS activities.
With the approval of an APD, a state is given the clearance to develop  the request-for-proposals for soliciting contractor proposals. While the  APD is a high-level justification for funding, the request-for-proposals is  to contain the more detailed requirements of the MMIS activities. Before  it is issued, the request-for-proposals must be approved by the CMS  regional office through a process similar to that used for the APD.
The states review the proposals received and evaluate them in order to  make the final selection. While regional office staff do not formally  approve a state’s evaluation process, they do review the process to  ensure that it allows for open and free competition, to the maximum  extent practicable.
The states draft a contract for the MMIS activities. Prior to its award, the  contract is reviewed by regional office staff and approved by the  Associate Regional Administrator for Medicaid. The state then makes an  award to the contractor whose bid or offer is responsive to the  solicitation and most advantageous to the state—considering price,  quality, and other factors.
When the contracted MMIS activities start, regional office staff begin  monitoring the status of these activities through a variety of  mechanisms, including reviews of status reports; on-site visits; and  meetings with external groups, such as industry associations, provider  groups, and vendors.
Once MMISs are built and become operational, CMS establishes a team  consisting of headquarters and regional office staff with expertise in  relevant areas to do on-site reviews, referred to as certification reviews.  During these reviews, which are to be conducted about 6 months after a  system has been in operation, the team makes sure that the system  satisfies the terms of the state’s APD, meets minimal federal  requirements, and complies with current regulations and policy. CMS  has written guidance for conducting these reviews, which it is in the  process of revising.
Regional office staff are to continue monitoring MMIS activities through  the previously mentioned mechanisms.
section_title: Information Technology Investment Management Maturity Framework
paragraphs: The Information Technology Investment Management (ITIM) framework is  a maturity model comprising five progressive stages of maturity that an  agency can achieve in its investment management capabilities. The ITIM  framework was developed on the basis of our research into the IT  investment management practices of leading private- and public-sector  organizations. It identifies critical processes for making successful IT  investments, organized into the five increasingly mature stages. These  maturity stages are cumulative; that is, in order to attain a higher stage of  maturity, the agency must have institutionalized all of the requirements for  all of the lower stages in addition to the higher stage.
The ITIM framework can be used to assess the maturity of an agency’s  investment management processes and as a tool for organizational  improvement. The overriding purpose of the framework is to encourage  investment processes that increase business value and mission  performance, reduce risk, and increase accountability and transparency in  the decision process. We have used the framework in several of our  evaluations, and a number of agencies have adopted it. These agencies  have used ITIM for purposes ranging from self-assessment to the redesign  of their IT investment management processes.
The ITIM framework’s five maturity stages represent steps toward  achieving stable and mature processes for managing IT investments. Each  stage builds on the lower stages; the successful attainment of each stage  leads to improvement in the organization’s ability to manage its  investments. With the exception of the first stage, each maturity stage is  composed of “critical processes” that must be implemented and  institutionalized in order for the organization to achieve that stage. These  critical processes are further broken down into key practices that describe  the types of activities that an organization should be performing to  successfully implement each critical process. An organization may be  performing key practices from more than one maturity stage at the same  time. This is not unusual, but efforts to improve investment management  capabilities should focus on becoming compliant with lower-stage  practices before addressing higher-stage practices.
Stage 2 of the ITIM framework encompasses building a sound investment  management process by establishing basic capabilities for selecting new IT  projects. It also involves developing the capability to control projects so  that they finish predictably within established cost and schedule  expectations and the capability to identify potential exposures to risk and  put in place strategies to mitigate that risk. The basic selection processes  established in Stage 2 lays the foundation for more mature selection  capabilities in Stage 3.
Stage 3 requires that an organization continually assess both proposed and  ongoing projects as parts of a complete investment portfolio—an  integrated and competing set of investment options. It focuses on  establishing a consistent, well-defined perspective on the IT investment  portfolio and maintaining mature, integrated selection (and reselection),  control, and evaluation processes that can be evaluated during  postimplementation reviews. This portfolio perspective allows decision  makers to consider the interaction among investments and the  contributions to organizational mission goals and strategies that could be  made by alternative portfolio selections, rather than focusing exclusively  on the balance between the costs and benefits of individual investments.  Organizations implementing Stages 2 and 3 have in place the selection,  control, and evaluation processes that are required by the Clinger-Cohen  Act of 1996.
Stages 4 and 5 require the use of evaluation techniques to continuously  improve both the investment portfolio and the investment processes in  order to better achieve strategic outcomes. At Stage 4 maturity, an  organization has the capacity to conduct IT succession activities and,  therefore, can plan and implement the deselection of obsolete, high-risk, or  low-value IT investments. An organization with Stage 5 maturity conducts  proactive monitoring for breakthrough information technologies that will  enable it to change and improve its business performance. Stages 4 and 5  define key attributes that are associated with the most capable  organizations.
Figure 3 shows the five ITIM stages of maturity and the critical processes  associated with each stage.
As defined by the model, each critical process consists of “key practices”  that must be executed to implement the critical process.
section_title: CMS’s Capabilities to Manage Its Internal Investments Are Limited
paragraphs: In order to have the capabilities to effectively manage IT investments, an  agency, at a minimum, should (1) build an investment foundation by putting  basic, project-level control and selection practices in place (Stage 2  capabilities) and (2) manage its projects as a portfolio of investments,  treating them as an integrated package of competing investment options  and pursuing those that best meet the strategic goals, objectives, and  mission of the agency (Stage 3 capabilities).
CMS has executed 20 of the 38 key practices that are required to build a  foundation for IT investment management. In addition, because CMS has  focused primarily on establishing the Stage 2 practices, it has executed  only 2 of the 27 Stage 3 key practices. Until CMS implements all of the key  practices associated with building the investment foundation and managing  its investments as a portfolio, the agency will not have much assurance that  it has selected the mix of investments that best supports its strategic goals,  or that it will be able to manage the investments to successful completion.
subsections: section_title: CMS Has Established about Half of the Foundational Practices for Investment Management
paragraphs: At the ITIM Stage 2 level of maturity, an organization has attained  repeatable, successful IT project-level investment control processes and  basic selection processes. Through these processes, the organization can  identify expectation gaps early and take the appropriate steps to address  them. According to the ITIM framework, critical processes at Stage 2  include (1) defining IT investment board operations, (2) identifying the  business needs for each IT investment, (3) developing a basic process for  selecting new IT proposals and reselecting ongoing investments,  (4) developing project-level investment control processes, and  (5) collecting information about existing investments to inform investment  management decisions. Table 1 describes the purpose of each of these  Stage 2 critical processes.
Because IT investment management has only recently become an area of  management attention, CMS has put in place 20 of the 38 Stage 2 key  practices required for basic project-level selection and control. The agency  has satisfied the majority of the key practices associated with establishing  an IT investment review board, capturing investment information, and  meeting business needs. CMS also has recently established a process for  selecting investments, but it has not yet established a process for the IT  investment review board to provide investment oversight. Figure 4  summarizes the status of CMS’s critical processes for Stage 2, showing how  many key practices CMS has executed in managing its internal IT  investments.
The creation of decision-making bodies or boards is central to the IT  investment management process. At the Stage 2 level of maturity,  organizations define one or more boards, provide resources to support  their operations, and appoint members who have expertise in both  operational and technical aspects of the proposed investments. The boards  operate according to a written IT investment process guide that is tailored  to the organization’s unique characteristics, thus ensuring that consistent  and effective management practices are implemented across the  organization. Once board members are selected, the organization ensures  that they are knowledgeable about policies and procedures for managing  investments. Organizations at the Stage 2 level of maturity also take steps  to ensure that executives and line managers support and carry out the  decisions of the investment board. According to the ITIM framework, an IT  investment management process guide should (1) be a key authoritative  document that the organization uses to initiate and manage IT investment  processes and (2) provide a comprehensive foundation for policies and  procedures developed for all other related processes. (The complete list of  key practices is provided in table 2.)
CMS has executed 5 of the 8 key practices for this critical process. For  example, in January 2004, the agency established the ITIRB to manage  internal investments and provide business-driven leadership to its  operations and development. While the ITIRB was initially only responsible  for overseeing MMA and revitalization initiatives, its responsibilities were  expanded this past spring to include management and oversight  responsibilities for all internal investments. ITIRB members are senior- level officials from both business and IT areas who understand board  policies and procedures.
The ITIRB is adequately resourced to maintain its operations. For example,  the Program Management and Support Group within the Office of  Information Services assists the board in such ways as coordinating and  integrating the investment management process. This group serves as the  principal contact and entry point for all new and proposed IT projects. In  addition, nine Executive Steering Committees were recently established to  support the work of the ITIRB by managing a subset of investments  grouped together according to business function. Their responsibilities  include, among other things, scoring and ranking IT investments, and  recommending investments to the ITIRB for funding.
Notwithstanding these strengths, CMS does not have an IT investment  process guide that reflects the agency’s current investment management  practices. For example, the agency uses ESCs to work with the board on  specific areas of IT investments, but its process guide does not identify this  critical group. Moreover, the process guide does not mention the agency’s  move to classify its IT investments in line with the department’s  classification scheme. (The new classification scheme consists of three  levels in which projects are rated as major, supporting, or tactical.) Instead,  the process guide outlines a four-level classification scheme that identifies  investments as A, B, C, or D, depending on the nature and sensitivity of the  project. According to CMS officials, the guide has not yet been updated  because the agency has made a priority of fully defining its processes  before documenting them. Documenting the process, however, does not  preclude it from future revisions or improvements, but does provide a basis  for consistent implementation across the agency. Until CMS’s documented  IT investment process guidance is updated, executives are at risk of  inconsistently performing key investment decision-making activities and  inaccurately communicating management practices. Such updated  guidance would also provide a process that could lead to greater  accountability about future IT investment outcomes, which would be  helpful to new members joining the board.
Another key weakness is that CMS’s ITIRB has not operated in accordance  with its assigned roles and responsibilities. For example, the ITIRB has not  yet been involved in systematically controlling investments nor has it  actively maintained the documented investment management process.  Until the ITIRB fully carries out its assigned roles and responsibilities,  executives will not have assurance that the whole IT investment  management process is functioning smoothly and effectively as intended.
Table 2 shows the rating for each key practice required to implement the  critical process for instituting the investment board at the Stage 2 level of  maturity. Each of the “executed” ratings shown below represents instances  where, on the basis of the evidence provided by CMS officials, we  concluded that the specific key practices were executed by the agency.
Defining business needs for each IT project helps to ensure that projects  and systems support the organization’s business needs and meet users’  needs. This critical process ensures that a link exists between the  organization’s business objectives and its IT management strategy.  According to the ITIM, effectively meeting business needs requires, among  other things, (1) documenting business needs with stated goals and  objectives, (2) identifying specific users and other beneficiaries of IT  projects and systems, (3) providing adequate resources to ensure that  projects and systems support the organization’s business needs and meet  users’ needs, and (4) periodically evaluating the alignment of IT projects  and systems with the organization’s strategic goals and objectives. (The  complete list of key practices is provided in table 3.)
CMS has in place 5 of the 7 key practices for meeting business needs. The  agency’s IT Investment Management Process Guide and Business Case  Analysis Development Guide require business needs for both proposed  and ongoing IT projects and systems to be identified in an IT fact sheet and,  in some instances, a business case analysis document. The agency also has  detailed procedures for developing these documents that call for  identifying users. We verified that the four projects we reviewed identified  specific users and also documented how the projects linked back to CMS  business needs. Resources for ensuring that IT projects and systems  support the organization’s business needs and meet users’ needs include  Component Leads, the Enterprise Architecture Group, and detailed  procedures and associated templates for developing the IT fact sheet and  business case analysis document.
Although CMS has performed most of the key practices associated with  meeting business needs, a few weaknesses remain. Specifically, officials  told us they rely on the HHS strategic plan to guide their efforts because  CMS’s strategic plan documenting the agency’s business mission, goals, and  objectives is outdated. However, the primary tool used to justify funding  for investments does not tie into the HHS plan but provides high-level  business drivers for aligning these investments with business needs.  While, according to agency officials, these business drivers reflect a  common understanding of the agency’s goals and objectives, they are not  descriptive enough to drive IT investments. Until CMS develops a current  strategic plan or other detailed statement of business mission with  supporting goals and objectives, the agency is at risk of not being able to  thoroughly communicate critical information on its goals and objectives or  to provide clear and transparent direction for its IT investment  management process.
Finally, CMS’s budget formulation process serves as a mechanism to  reevaluate the alignment of projects and systems with the organization’s  goals and objectives. However, the ITIRB selected investments for the first  time this past spring and, therefore, has not yet had to reevaluate projects’  and systems’ alignment with organizational goals and objectives. When  CMS executes all key practices associated with this critical process, it will  have greater assurance that its projects effectively meet the agency’s  business needs.
Table 3 shows the rating for each key practice required to implement the  critical process for meeting business needs at the Stage 2 level of maturity  and summarizes the evidence that supports these ratings.
Selecting new IT proposals and reselecting ongoing investments require a  well-defined and disciplined process to provide the agency’s investment  board, business units, and developers with a common understanding of the  process and the cost, benefit, schedule, and risk criteria that will be used  both to select new projects and to reselect ongoing projects for continued  funding. According to the ITIM, this critical process requires, among other  things, (1) making funding decisions for new proposals according to an  established process; (2) providing adequate resources for investment  selection activities; (3) using a defined selection process to select new  investments and reselect ongoing investments; (4) establishing criteria for  analyzing, prioritizing, and selecting new IT investments and for reselecting  ongoing investments; and (5) creating a process for ensuring that the  criteria change as organizational objectives change. (The complete list of  key practices is provided in table 4.)
CMS has executed 4 of the 10 key practices associated with selecting an  investment. Specifically, CMS used a process it defined in February  2005—its budget formulation process—to select new investments and  reselect existing investments using a set of limited criteria. We confirmed  that the four projects we reviewed were reselected using this new process.  In addition, by using the budget formulation process to select investments,  executives had assurance that funding decisions were aligned with  selection decisions. Officials indicated that adequate resources were  provided for identifying and selecting investments.
However, weaknesses remain in the selection area. Although CMS has a  number of documents that address investment selection and reselection,  these documents are not linked to provide a clear understanding of the  selection and reselection process. In addition, they do not define (1) the  roles and responsibilities for each participating unit involved in the project  selection process and (2) the decision-making procedures. CMS officials  told us they chose to first implement the selection process and then go  back to document it. Another key weakness in the selection area is that,  although selection and reselection criteria have been defined, they do not  include cost, benefit, schedule, and risk factors. Officials indicated that  because the Executive Steering Committees and the ITIRB had a short  amount of time to perform selection activities this year, they defined a  limited set of criteria to evaluate projects. Further, CMS does not have a  mechanism to ensure that its selection criteria continue to reflect  organizational objectives.
Until CMS implements all of the key practices associated with selecting  investments, executives will not be adequately assured that they are  consistently and objectively selecting projects that meet the needs and  priorities of the agency in a cost-effective and risk-insured manner.
Table 4 shows the rating for each key practice required to implement the  critical process for selecting an investment at the Stage 2 level of maturity  and summarizes the evidence that supports these ratings.
An organization should provide effective oversight for its IT projects  throughout all phases of their life cycles. Its investment board should  maintain adequate oversight and observe each project’s performance and  progress toward predefined cost and schedule expectations as well as each  project’s anticipated benefits and risk exposure. The investment board  should also employ early warning systems that enable it to take corrective  action at the first sign of cost, schedule, or performance slippages. This  board has ultimate responsibility for the activities within this critical  process. According to the ITIM framework, effective project oversight  requires, among other things, (1) having written policies and procedures  for management oversight; (2) developing and maintaining an approved  management plan for each IT project; (3) making up-to-date cost and  schedule data for each project available to the oversight boards; (4) having  regular reviews by each investment board of each project’s performance  against stated expectations; and (5) ensuring that corrective actions for  each underperforming project are documented, agreed to, implemented,  and tracked until the desired outcome is achieved. (The complete list of  key practices is provided in table 5.)
CMS has only executed 1 of the 7 key practices associated with effective  project oversight. While CMS’s IT Investment Management Process Guide  addresses management oversight of IT projects and systems, it does not  include specific procedures for the ITIRB’s oversight of IT projects and  systems. In addition, while the board is receiving performance data for  some investments, including revitalization investments, it is not yet  performing oversight of projects on a systematic basis. CMS officials  indicated that the ITIRB’s involvement in overseeing investments to date  has been limited because the board was first focusing on selecting  investments. However, CMS recognizes the importance of the ITIRB’s  involvement in oversight of IT investments, and, according to officials, the  agency is currently developing an approach to address this issue.
While CMS is in the process of developing a structured process for the  ITIRB to oversee investments, other entities are involved in the oversight of  projects. For example, performance information for one of the projects we  reviewed was not provided to CMS’s ITIRB, but instead was provided to  senior-level management, such as the Chief Technology Officer and  directors from some CMS components. Until the ITIRB systematically  oversees CMS’s investments, the oversight process will not benefit from the  corporate perspective that is gained by having an enterprisewide board. As  a result, executives may not be able to easily determine the impact  individual project decisions may have on other projects and on the  attainment of organizational goals and objectives.
Table 5 shows the rating for each key practice that is required to implement  the critical process for project oversight at the Stage 2 level of maturity and  summarizes the evidence that supports these ratings.
To make good IT investment decisions, an organization must be able to  acquire pertinent information about each investment and store that  information in a retrievable format. During this critical process, an  organization identifies its IT assets and creates a comprehensive repository  of investment information. This repository provides information to  investment decision makers to help them evaluate the impacts and  opportunities that would be created by proposed or continuing  investments. It can provide insights and trends about major IT cost and  management drivers. The repository can take many forms and does not  have to be centrally located, but the collection method should identify each  IT investment and its associated components. This critical process may be  satisfied by the information contained in the organization’s current  enterprise architecture, augmented by additional information—such as  financial information and information on risk and benefits—that the  investment board may require to ensure that informed decisions are being  made. According to the ITIM framework, effectively managing this  repository requires, among other things, (1) developing written policies  and procedures for identifying and collecting the information, (2) assigning  responsibility for ensuring that the information being collected meets the  needs of the investment management process, (3) identifying IT projects  and systems and collecting relevant information to support decisions about  them, and (4) making the information easily accessible to decision makers  and others. (The complete list of key practices is provided in table 6.)
CMS has in place 5 of the 6 key practices associated with capturing  investment information. For example, CMS’s IT Investment Management  Process Guide identifies specific information that is needed in the  investment management process, such as how each IT project relates to  the business needs of CMS. According to officials, adequate resources are  provided to support the collection of investment information, such as the  agency’s IT Investment Tracking Database, and an individual assigned  responsibility for ensuring that the necessary information is collected to  meet the needs of the investment management process.
CMS is collecting specific information about IT investments to support  decisions about these investments, including projects’ scores against  selection criteria and earned value management information. We verified  that this information was collected for the four projects we reviewed.  However, although the ITIRB has used investment information to support  selection decisions, it has not yet used it to systematically oversee projects.  According to CMS officials, specific procedures for the ITIRB’s oversight of  IT projects and systems are currently being defined.
Table 6 shows the rating for each key practice required to implement the  critical process for capturing investment information at the Stage 2 level of  maturity and summarizes the evidence that supports these ratings.
section_title: CMS Lacks the Key Capabilities Needed to Manage Its Investments as a Portfolio
paragraphs: During Stage 3, the investment board enhances the investment  management process by developing a complete investment portfolio. An  investment portfolio is an integrated, agencywide collection of investments  that are assessed and managed collectively on the basis of common  criteria. Managing investments within the context of such a portfolio is a  conscious, continuous, and proactive approach to expending limited  resources on an organization’s competing initiatives in light of the relative  benefits expected from these investments. Taking an agencywide  perspective enables an organization to consider its investments  comprehensively, so that, collectively, the investments optimally address  the organization’s missions, strategic goals, and objectives. Managing  investments with a portfolio approach also allows an organization to  determine priorities and make decisions about which projects to fund, and  continue to fund, on the basis of analyses of the relative organizational  value and risks of all projects, including projects that are proposed, under  development, and in operation. For an organization to reap the full benefits  of the portfolio process, it should collect all of its investments into an  enterprise-level portfolio that is overseen by its senior investment board.  Although investments may initially be selected into subordinate  portfolios—on the basis of, for example, the lines of business or life-cycle  stages-—and managed by subordinate investment boards, they should  ultimately be aggregated into this enterprise-level portfolio.
According to our ITIM framework, critical processes performed by Stage 3  organizations include (1) defining the portfolio criteria, (2) creating the  portfolio, (3) evaluating the portfolio, and (4) conducting  postimplementation reviews. Table 7 shows the purpose of each critical  process in Stage 3.
CMS has executed very few key practices—2 of 27—associated with Stage  3 critical processes. Specifically, under the critical process for defining the  portfolio criteria, CMS provided evidence that it had designated a working  group to be responsible for developing and modifying portfolio selection  criteria. Under the critical process for creating the portfolio, CMS provided  evidence that it was capturing and maintaining investment information for  future reference. In its self-assessment, the agency stated that it was not  executing any other Stage 3 key practices. According to officials, CMS has  not concentrated on implementing Stage 3 key practices because the  agency is first focusing its resources on establishing the practices  associated with Stage 2. Until CMS fully implements the critical processes  associated with managing its investments as a complete portfolio, it will  not have the data or enterprisewide perspective it needs to make informed  decisions about its collection of investments.
section_title: CMS Does Not Have a Comprehensive Plan to Coordinate and Guide Its Improvement Efforts
paragraphs: CMS has initiated efforts to improve its investment management process.  While these efforts do not fully address any of the weaknesses we identify  in this report, they enhance the agency’s ability to perform key activities.  Specifically:  CMS has begun to implement a tool for capturing project information.
According to officials, the tool will bring together investment  information currently residing in various locations, including project  description information captured in its IT Investment Tracking  Database, information such as project scores collected to support  project selection activities, and earned value management data.
Although information to support investment decisions does not have to  be in one location, doing so will improve accessibility and facilitate its  use by decision makers.
CMS recently established Executive Steering Committees to support the  ITIRB in carrying out its investment management responsibilities. These  groups played a key role in selecting investments for the fiscal year 2007  budget by reviewing investment information and making  recommendations for funding to the investment board. They are  currently determining procedures for overseeing investments.  According to officials, once procedures for the Executive Steering  Committee oversight have been determined, CMS plans to focus on  defining procedures for determining how and when to involve the  investment board in oversight—a key weakness identified in this report.
Although CMS has initiated these improvement efforts, it has not  coordinated them with the additional efforts needed to address the  weaknesses identified in this report in a comprehensive plan that  (1) specifies measurable goals, objectives, and milestones; (2) specifies  needed resources; (3) assigns clear responsibility and accountability for  accomplishing tasks; and (4) is approved by senior-level management. We  have previously reported that such a plan is instrumental in helping  agencies coordinate and guide improvement efforts.
CMS officials recognize the value of having a comprehensive plan and told  us they have begun to develop one; however, a time frame for completing  the plan has not been established. Until CMS develops this plan, the agency  risks not being able to put in place an effective management process that  will provide appropriate executive-level oversight for minimizing risks and  maximizing returns.
section_title: Process for Monitoring MMISs Could Benefit from Standard Procedures, Guidance, and Reporting Requirements
paragraphs: As we previously noted, the responsibility for approving and monitoring  MMISs that CMS funds jointly with the states falls to CMS’s central office  and its 10 regional offices, with the bulk of the activities being performed  by the regional offices. Although the process for approving states’ funding  requests for MMIS activities is characterized by (1) standard procedures  performed consistently across the regional offices, (2) guidance that staff  can rely on in carrying out their duties, and (3) requirements for reporting  information to the central office, the process for monitoring MMIS  activities is not.
subsections: section_title: Standard Procedures, Guidance, and Reporting Requirements Exist for the Approval Process
paragraphs: The process for approving states’ requests for federal funding of MMISs is  characterized by a defined set of activities that are performed consistently  across the regional offices. These activities include regional office staff  review and approval of the standard documentation (i.e., the APDs,  request-for-proposals, and contracts) that the states prepare to justify their  requests. Specifically, as we previously described:   States prepare an APD to request funding for MMISs. Regional office  staff review the document to ensure that states’ requests support the  Medicaid program, are in compliance with federal requirements, and  represent cost-effective solutions. Once regional office staff determine  that the APD adequately justifies the request, they issue a formal  approval letter to the states (with concurrence from CMS’s central  office).
The request-for-proposals that the states prepare to solicit contractor  bids for MMIS activities, including development and operations, is  reviewed and approved by regional office staff through a process similar  to that used to approve the APDs.
Regional office staff review the states’ process for reviewing  contractors’ proposed bids.
Regional office staff review and approve the contract, after which the  state makes an award to the contractor whose bid or offer is responsive  to the solicitation and is most advantageous to the state—considering  price, quality, and other factors.
Regional office staff told us that they rely on the State Medicaid Manual  and the Code of Federal Regulation for guidance in performing activities  for approving states’ requests for federal funding. Regional staff are also  required to inform the CMS central office of all approval actions through  the Office of the Secretary Notice process previously mentioned.
section_title: Process for Monitoring State MMISs Lacks Standard Procedures, Guidance, and Reporting Requirements
paragraphs: In contrast to the approval process, the process for monitoring MMIS  activities lacks (1) standard procedures regional office staff must perform  to carry out their responsibilities, (2) guidance for staff to rely on, and (3)  requirements for staff to report on the results of their monitoring efforts to  the central office.
First, regional office staff use a variety of mechanisms to monitor MMIS  activities. These mechanisms include reviews of project status reports; site  visits; telephone calls; and meetings with external groups, such as industry  associations, provider groups, and vendors. In addition, regional office staff  determine if and when to use these mechanisms. Table 8 shows the  different mechanisms used by the regional office staff we interviewed and  the number of regional offices who used them.
Second, CMS has no guidance for regional office staff to use in monitoring  MMIS activities. While CMS has a Regional Office Manual that includes  guidance for monitoring MMIS activities, this manual is not used by  regional office staff because, according to officials, it has not been  maintained throughout the years, and it no longer reflects current  processes.
Third, there are no requirements for regional office staff to report to CMS’s  central office on their monitoring of states’ federally funded MMISs  activities. Monthly teleconferences are conducted between the central  office and regional offices to discuss activities performed by these offices,  including activities to monitor state MMISs. According to CMS officials,  there is some communication outside of the scheduled teleconferences to  discuss any issues that might arise regarding the status of state MMISs. In  addition, according to officials, the certification reviews performed about 6  months after the MMISs have become operational provide opportunities to  determine firsthand how systems are performing. Despite these  mechanisms, the central office has no requirements for regional office staff  to regularly report on the results of their efforts to monitor MMIS activities.
According to CMS officials, the central office has traditionally placed  greater emphasis on the front-end approval of requests for federal funding.  The central office, however, now recognizes the need for and value of  adopting an approach for maintaining the visibility of MMISs from  beginning to end. To address this need, central office staff told us that they  plan to ask the regional offices to provide them with quarterly reports on  the status of MMIS activities in their states as part of a broader effort that is  currently under way to improve the administration of the Medicaid  program. Central office staff stated this effort would also result in  standard procedures and guidance to support regional office staff’s  monitoring efforts. While these activities would strengthen the monitoring  process, during our review central office staff did not yet have specific  plans or time frames for implementing them.
Until CMS defines standard procedures for monitoring MMIS activities,  guidance for staff to rely on, and reporting requirements, CMS’s central  office may not be able to easily determine whether state MMISs are  facilitating the delivery of Medicaid benefits in the most effective and  beneficial manner.
section_title: Conclusions
paragraphs: Because IT investment management has only recently become an area of  management focus, CMS capabilities to manage its internal investments are  limited. Specifically, the agency has established about half of the practices  for building the investment foundation, but few practices to manage its  investments as a portfolio. Although the foundational practices have  equipped CMS with the capabilities it needs to improve its management of  individual investments, the agency is hampered in its ability to manage  them as a portfolio because it has not implemented the practices for doing  so. Until CMS fully establishes the key practices required to build the  investment foundation and manage its investments as a portfolio, it will not  have the capabilities it needs to ensure that investments supporting its  multibillion-dollar Medicare and Medicaid programs are being managed to  minimize risks and maximize returns.
Critical to CMS’s success in going forward will be the development of an  implementation plan that (1) is based on an assessment of strengths and  weaknesses; (2) specifies measurable goals, objectives, and milestones; (3)  specifies needed resources; (4) assigns clear responsibility and  accountability for accomplishing tasks; and (5) is approved by senior-level  management. Although the agency has initiated improvement efforts, it has  not developed a comprehensive plan to guide these and other efforts  needed to improve its investment management process. Without such a  plan and procedures for implementing it, CMS will be challenged in  sustaining the commitment it needs to fully establish its investment  management process.
Finally, the process for approving states’ funding requests for MMIS  activities is characterized by standard procedures that are performed  consistently across the regional offices, guidance, and requirements for  informing the central office of regional office staff activities. The process  for monitoring the development and operations of state MMIS, on the other  hand, has no standard procedures for regional office staff, no guidance, and  no requirement to report information to the central office. Without these  elements for monitoring MMIS activities, CMS’s central office may not be  able to easily determine whether state MMISs are facilitating the delivery of  Medicaid benefits in the most effective and beneficial manner.
section_title: Recommendations for Executive Action
paragraphs: To strengthen CMS’s capability to manage its internal IT investments and  address the related weaknesses addressed in this report, we recommend  that the Secretary of the Department of Health and Human Services direct  the CMS Administrator to develop and implement a plan for improving  CMS’s IT investment management processes. The plan should address the  weaknesses described in this report. The plan should (1) first focus on  correcting the weaknesses in Stage 2 critical processes, and next focus on  the Stage 3 critical processes, and (2) at a minimum, provide for  accomplishing the following 12 actions:   Update the agency’s investment management guide to reflect current  investment management processes.
Establish a process for the board to actively maintain the agency’s  documented investment management process.
Use an updated strategic plan or other detailed statement of business  mission with supporting goals and objectives to align investments with  business needs.
Ensure that the board periodically evaluates the alignment of IT projects  and systems with strategic goals and objectives and take corrective  actions when misalignment occurs.
Fully document procedures that address investment selection and  reselection and (1) provide a clear understanding of the selection and  reselection process, (2) define the roles and responsibilities for each  participating unit involved in the project reselection process, and (3)  define the decision-making procedures.
Document procedures for integrating funding with investment selection.
Revise the ITIRB’s selection and reselection criteria to include cost,  benefit, schedule, and risk factors, and establish a mechanism to ensure  these criteria continue to reflect organizational objectives.
Define, document, and implement procedures for the ITIRB’s oversight  of projects and systems.
Implement processes to use investment information to fully support  investment management decisions.
Implement the Stage 3 critical processes for defining portfolio criteria,  creating the portfolio, evaluating the portfolio, and conducting  postimplementation reviews, which are necessary for portfolio  management.
We also recommend that the Secretary for Health and Human Services  direct the CMS Administrator to ensure that the plan draw together  ongoing efforts and additional efforts that are needed to address the  weaknesses identified in this report. The plan should also (1) specify  measurable goals, objectives, and milestones; (2) specify needed  resources; (3) assign clear responsibility and accountability for  accomplishing tasks; and (4) be approved by senior-level management. In  implementing the plan, the Administrator should ensure that progress is  measured and reported periodically to the Secretary of Health and Human  Services.
To improve CMS’s process for monitoring states’ progress in developing  and maintaining Medicaid management information systems, we are  recommending that the Secretary of the Department of Health and Human  Services direct the CMS Administrator to take the following two actions:   Define standard procedures and supporting guidance for regional  offices to monitor MMIS activities.
Require regional offices to regularly report on their MMIS monitoring  activities to CMS’s central office.
section_title: Agency Comments and Our Evaluation
paragraphs: The Administrator of CMS provided written comments on a draft of this  report (reprinted in app. II). In these comments, CMS identified actions it is  taking or plans to take to address our recommendations and stated that  effective management of IT investments is a critical priority at the agency.  CMS contended that many of the agency’s improvements to its IT  investment management process were not fully reflected in the report, and  took exception with the need for up-to-date, documented processes to  ensure consistency.
Concerning our description of progress in implementing investment  management processes, CMS commented that the report indicates that the  agency has only established 2 out of 27 key practices needed to manage  investments as a portfolio. CMS stated that this is misleading since the  report also indicates that the agency has accomplished 20 of 38  foundational IT investment management practices. CMS also provided  examples of the practices it has implemented, such as establishing an  investment review board. In our report, we make a distinction between  foundational practices, which are the Stage 2 key practices for establishing  basic project-level selection and control capabilities, and portfolio-level  practices, which are the Stage 3 key practices for managing investments as  an integrated set of competing options. We also note that both of these sets  of key practices are needed to implement the processes required by the  Clinger-Cohen Act of 1996. On the basis of this, we state that CMS does not  have the full suite of capabilities to manage its internal investments  because it has only established a little over half of the foundational  practices and 2 of 27 portfolio-level key practices, and we reiterate the  need to fully establish both sets of practices to increase assurance that  executives are selecting and managing the mix of investments that best  meets the agency’s needs and priorities. In our report, the sections in which  we discuss the implementation of specific key practices associated with  critical processes from our IT investment management framework each  describe CMS’s efforts and accomplishments to improve its IT investment  management processes. These include all of the examples of  accomplishments CMS provided in its comments.
In its comments, CMS took issue with our reporting that its IT investment  management guide did not reflect the current process, and that its  procedures for selecting and reselecting IT investments were not fully  documented. Although the agency fully agreed that an up-to-date guide  would constitute a piece of an effective process, it commented that the  emphasis should be on strengthening the process first and updating  documentation later. CMS made three points: (1) it is not practical to  publish an updated guide without having the effective and repeatable  underlying process in place and noted that it is not provided the latitude to  do this; (2) in the section of the report discussing instituting the investment  board, the noted successful execution of key practices appears to be  negated by the statement that the investment management processes are  not documented; and (3) in the same section of the report, we are implying  that an updated guide would improve rather than explain the process. We  disagree with CMS that the process needs to be repeatable and  strengthened before it can be documented. Documented procedures could  actually serve to strengthen and improve the process by ensuring it is  performed consistently. Finally, we are not negating the successful  implementation of key practices to institute the investment board. We are  simply emphasizing the importance of having documentation to drive the  investment management process.
In its comments, CMS also noted actions it is taking to (1) develop a plan to  implement key practices in Stages 2 and 3; (2) revise existing  documentation to reflect processes in place that are not formally  documented; and (3) develop a plan that will be approved by senior  management that will incorporate goals, objectives, and milestones  required to further close the gaps between existing processes and our ITIM  framework. Regarding our recommendation to improve its process for  monitoring state MMIS activities and reporting to the central office, CMS  stated that it is developing standard procedures and supporting guidance  for the regional office(s) for monitoring these systems activities and  reporting to the central office. We agree with CMS that these actions would  address many of the weaknesses we identify in this report.
CMS also provided some technical comments, which we have incorporated  into the report as appropriate.
As agreed with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days from the  date of this letter. At that time, we will send copies to other interested  congressional committees, the Secretary of Health and Human Services,  the CMS Administrator, the CMS Chief Information Officer, and other  interested parties. Copies will also be made available at no charge on our  Web site at www.gao.gov. If you have any questions on matters discussed in  this report, please contact David A. Powner at (202) 512-9286, or at  pownerd@gao.gov, or Leslie G. Aronovitz at (312) 220-7600, or at  aronovitzl@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made major contributions to this report are listed in  appendix III.
section_title: Objectives, Scope, and Methodology
paragraphs: The objectives of our review were to (1) evaluate Centers for Medicare &  Medicaid Services (CMS) capabilities for managing its internal information  technology (IT) investments, (2) determine any plans the agency might  have for improving these capabilities, and (3) examine CMS’s process for  approving and monitoring the state Medicaid management systems it  funds.
To address our first objective, we reviewed the results of the agency’s self- assessment of Stages 2 and 3 practices using our Information Technology  Investment Management framework (ITIM) and validated and updated the  results of the self-assessment through document reviews and interviews  with officials. We reviewed written policies, procedures, and guidance and  other documentation providing evidence of executed practices, including  CMS’s IT Investment Management Process Guide, CMS’s Policy for IT  Investment Management, and CMS’s fiscal year 2006/2007 budget process.  We also reviewed the CMS Information Technology Investment Review  Board (ITIRB) meeting minutes. We did not assess CMS’s progress in  establishing the capabilities found in Stages 4 and 5 of the ITIM framework  because CMS acknowledged that it had not executed any of the key  practices in these higher maturity stages. In addition, we conducted  interviews with officials from the Office of Information Services who have  responsibility for the development and implementation of CMS’s IT  investment management process.
We compared the evidence collected from our document reviews and  interviews with the key practices in our ITIM framework. We rated the key  practices as “executed” on the basis of whether the agency demonstrated  (by providing evidence of performance) that it had met the criteria of the  key practice. A key practice was rated as “not executed” when we found  insufficient evidence of a practice being executed or when we determined  that there were significant weaknesses in CMS’s execution of the key  practice. In addition, CMS was provided with the opportunity to produce  evidence for key practices rated as “not executed.”
As part of our analysis, we selected four CMS IT projects as case studies to  verify that the critical processes and key practices were being applied. The  projects were selected because they (1) supported different functional  areas, (2) were in various life-cycle phases, and (3) required different levels  of funding. The four projects are described below:   Healthcare Integrated General Ledger Accounting System—The project  is intended to standardize the collection, recording, and reporting of  Medicare financial information by contractors. It is to replace the  cumbersome ad hoc spreadsheets and “cuff” systems being used by  Medicare contractors to accumulate and report financial information to  CMS. The project’s life-cycle cost is estimated at about $567 million.
Medicare Claims Processing Redesign—This project is intended to  integrate and modernize the Common Working File system and  Redesign and the Medicare Shared Systems enterprise claims  processing applications and data systems. The modernization and  unification of these systems is to allow CMS to significantly enhance  program capabilities, integrity, performance, efficiencies, and  maintainability; reduce program change implementation time frames;  improve accuracy, timeliness, and quality of Medicare transaction  processing; reduce system exposure to security risks; and facilitate use  of the Internet. The project’s life-cycle cost is estimated at nearly $494  million.
Medicare Managed Care System—This project is intended to cover the  redesign of CMS’s managed care family of systems, including the legacy  Group Health Plan system. It is to provide the platform for implementing  requirements under the MMA. The project is intended to replace aging  operations and to continue to support the agency’s managed care  business needs until all functions are migrated to a new system. Its life- cycle cost is estimated at about $111 million.
National Plan and Provider Enumeration System—The project is  intended to implement a Health Insurance Portability and  Accountability Act requirement to issue a unique identifier to each  covered health care provider in the United States. It is expected to result  in administrative savings by simplifying a complicated, multifaceted  enumeration scheme, whereby a provider is issued different identifiers  for electronic transactions by each health plan with which it does  business, and sometimes multiple identifiers from a single plan. It will  impact several million providers and health plans in the nation. The  project’s life-cycle cost is estimated at about $38 million.
For these projects, we reviewed project management documentation, such  as project plans, business cases, status reports, and documentation on how  these projects were selected by the ITIRB. We also interviewed the project  managers for these projects.
To address our second objective, we examined documentation on what  management actions had been taken and what initiatives had been planned  by the agency. This documentation included a requirements document for a  tool CMS is currently implementing that is to help the agency with IT  investment management, among other things. We also interviewed officials  from the Office of Information Services to determine what efforts CMS had  undertaken to improve IT investment management processes.
To address our third objective, we reviewed documentation supporting  CMS’s implementation of processes for (1) approving states’ requests for  funding their Medicaid Management Information Systems (MMIS) and (2)  monitoring these MMISs, including related legislation, policy, and  implementing guidance. We also interviewed officials at CMS headquarters  and at the 5 CMS regional offices with the highest fiscal year 2004  expenditures for administrative services, which includes MMISs.
We conducted our work at CMS headquarters in Washington, D.C., and at 5  CMS regional offices located in New York, New York; Philadelphia,  Pennsylvania; Chicago, Illinois; San Francisco, California; and Atlanta,  Georgia, from January 2005 through September 2005 in accordance with  generally accepted government auditing standards.
section_title: Comments from the Centers for Medicare & Medicaid Services
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the persons named above, William G. Barrick, Shaunessye  Curry, Mary Beth McClanahan, Sabine R. Paul, and Amos Tevelow made  key contributions to this report.