Information Technology: SSA Has Taken Key Steps for Managing Its Investments, but Needs to Strengthen Oversight and Fully Define Policies and Procedures
Published: 2008-09-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Social Security Administration (SSA) spends about $1 billion annually to support its information technology (IT) needs. Given the size and significance of the agency's ongoing and future investments in IT, it is crucial that the agency manages these investments wisely. Accordingly, GAO was requested to determine whether SSA's investment management approach is consistent with leading investment management best practices. To accomplish this, GAO used its IT investment management framework and associated methodology, with a focus on the framework's Stages 2 and 3, which are based on the investment management provisions of the Clinger-Cohen Act of 1996.
section_title: What GAO Found
paragraphs: SSA's investment management approach is largely consistent with leading investment management practices. It has established most of the practices needed to manage its projects as investments and is making progress towards managing IT investments as a portfolio; however, it is not applying its investment management process to all of its investments. Specifically: (1) The agency is executing a majority of the key practices needed to build the foundation for managing its IT projects as investments. Of the 5 processes and their 38 associated key practices, SSA is executing 31 practices. However, the agency's investment board, which should provide executive oversight of investments, is not adequately monitoring the performance of IT projects. (2) SSA has made progress in establishing the key practices for managing investments as a portfolio--it is executing 18 out of 27 key practices. The agency has made important progress in defining and creating the investment portfolio, but it has not developed enterprisewide portfolio selection criteria. The agency also has not established procedures for evaluating the portfolio, and its postimplementation reviews do not determine whether projects meet the agency's strategic goals. (3) SSA is not applying its investment management process to a major portion of its IT budget. Specifically, IT products and services acquired with its acquisition budget ($610 million of the $1 billion IT budget for fiscal year 2008) are not managed by the board as investments. SSA's executive-level review board is not responsible for overseeing the acquisition budget. Consequently, executive management has limited insight into investments acquired with these funds, and the agency has limited ability to ensure that the budget is spent in the most efficient and effective manner. Until it establishes oversight of all investments and fully defines policies and procedures for overseeing both individual projects and an agencywide portfolio, SSA risks not being able to select and control these investments consistently and completely, thus increasing the chance that investments will not meet mission needs in the most cost-effective and efficient manner.

section_title: Letter
paragraphs: The Social Security Administration (SSA) manages and funds a variety of  information technology (IT) initiatives ranging from those supporting the  processing and payment of disability and supplemental security income  benefits to those that facilitate the calculation and withholding of  Medicare premiums. For fiscal year 2008, SSA plans to spend about $1  billion to support its IT needs. Given the size and significance of its  ongoing and future investments in information technology, it is crucial that  the agency manages these investments wisely. At your request, we  conducted an evaluation to determine whether SSA’s investment  management approach is consistent with leading investment management  best practices. These practices are identified in our IT Investment  Management (ITIM) framework by which we evaluate the maturity of an  agency’s investment management processes focusing on the framework’s  Stages 2 and 3, based on the investment management provisions of the  Clinger-Cohen Act of l996.
To accomplish our objective, we analyzed SSA’s self-assessment and  supporting documents to determine whether the agency has developed the  structures, policies, and procedures associated with executing those key  practices in the ITIM framework. We also interviewed relevant agency  officials about investment management practices. We selected three  projects as case studies to determine if certain critical processes and key  practices were applied. We conducted this performance audit from  October 2007 through September 2008 in accordance with generally  accepted government auditing standards. Those standards require that we  plan and perform the audit to obtain sufficient, appropriate evidence to  provide a reasonable basis for our findings and conclusions based on our  audit objectives. We believe that the evidence obtained provides a  reasonable basis for our findings and conclusions based on our audit  objective. For more details on our objective, scope, and methodology, see  appendix I.
subsections: section_title: Background
paragraphs: SSA’s mission is to advance the nation’s economic security through  compassionate and vigilant leadership in shaping and managing America’s  Social Security programs. This includes one of the nation’s largest  entitlement programs––federal Old-Age, Survivors, and Disability  Insurance benefits––commonly referred to as Social Security. The  program provides monthly benefits to retired and disabled workers, their  spouses and children, and the survivors of insured workers. SSA also  administers Supplemental Security Income, a needs-based program for the  aged, blind, and disabled that pays monthly benefits to individuals. Over 54  million people, one-sixth of the total U.S. population, receive monthly  Social Security or Supplemental Security Income benefit payments. The  agency’s estimated 2008 budget of about $657 billion includes an  administrative budget of $9.7 billion to support these programs, including  about $1 billion for IT.
Organizationally, SSA is headed by the Commissioner, who is assisted by a  deputy commissioner and various other executive officials, including the  Deputy Commissioner, Budget, Finance and Management; Chief  Information Officer (CIO); Chief Strategic Officer; and nine deputy  commissioners responsible for the agency’s various business components.  The organizational structure of the agency is depicted in figure 1.
The Commissioner is supported by approximately 60,000 employees  located at headquarters and throughout a decentralized network of over  1,400 offices that include regional offices, field offices, teleservice centers,  processing centers, state Disability Determination Services, program  service centers, and hearing offices. Of these employees, approximately  3,300 IT staff and contractors are assigned to the Office of Deputy  Commissioner, Systems. According to SSA, its organizational structure is  designed to provide timely, accurate, and responsive service to the  American public.
subsections: section_title: SSA Relies on IT to Deliver Services
paragraphs: The agency relies extensively on information technology to administer its  programs and to support related administrative needs. In this regard, IT is  used to, among other things:    evaluate evidence and make determinations of eligibility for benefits on  issue new and replacement Social Security cards,   process earnings items for crediting to workers’ earnings records,    handle millions of transactions on SSA’s toll-free telephone number,  issue Social Security statements,   process continuing disability reviews, and    process nondisability Supplemental Security Income redeterminations.
The agency’s IT budget for fiscal year 2008 is approximately $1 billion. Of  this amount, $400 million is for work year support of software  development projects in the Office of Deputy Commissioner, Systems and  about $610 million is for acquisition of IT-related products and services.  The agency expects to spend about 80 percent of its acquisition budget on  infrastructure.
section_title: Investment Management Is Critical to Effective Use of IT
paragraphs: A corporate approach to IT investment management is characteristic of  successful public and private organizations. Recognizing this, Congress  enacted the Clinger-Cohen Act of 1996, which requires the Office of  Management and Budget (OMB) to establish processes to analyze, track,  and evaluate the risks and results of major capital investments in IT  systems made by executive agencies. In implementing the Clinger-Cohen  Act and other statutes, OMB has developed policy and issued guidance for  the planning, budgeting, acquisition, and management of federal capital  assets. We have also issued guidance in this area that defines institutional  structures, such as investment boards; processes for developing  information on investments (such as cost/benefit); and practices to inform  management decisions (such as whether a given investment is aligned with  an enterprise architecture).
section_title: IT Investment Management: A Brief Description
paragraphs: IT investment management is a process for linking IT investment decisions  to an organization’s strategic objectives and business plans. Consistent  with this, the federal approach to IT investment management focuses on  selecting, controlling, and evaluating investments in a manner that  minimizes risks while maximizing the return on investment.
During the selection phase, the organization (1) identifies and analyzes  each project’s risks and returns before committing significant funds to any  project and (2) selects those IT projects that will best support its mission  needs.
During the control phase, the organization ensures that projects, as they  develop and investment expenditures continue, meet mission needs at the  expected levels of cost and risk. If the project is not meeting expectations  or if problems arise, steps are quickly taken to address the deficiencies.
During the evaluation phase, expected results are compared with actual  results after a project has been fully implemented. This comparison is  done to (1) assess the project’s impact on mission performance, (2)  identify any changes or modifications to the project that may be needed,  and (3) revise the investment management process based on lessons  learned.
section_title: Overview of GAO’s ITIM Maturity Framework
paragraphs: Our ITIM framework consists of five progressive stages of maturity for any  given agency relative to selecting, controlling, and evaluating its  investment management capabilities. (See fig. 2 for the five ITIM stages of  maturity.) This framework is grounded in our research of IT investment  management practices of leading private and public sector organizations.  The framework can be used to assess the maturity of an agency’s  investment management processes and as a tool for organizational  improvement. The overriding purpose of the framework is to encourage  investment processes that increase business value and mission  performance, reduce risk, and increase accountability and transparency in  the decision process. We have used the framework in many of our  evaluations,  and a number of agencies have adopted it.
ITIM’s five maturity stages represent steps toward achieving stable and  mature processes for managing IT investments. Each stage builds on the  lower stages and the successful attainment of each stage leads to  improvement in the organization’s ability to manage its investments. With  the exception of Stage 1, each maturity stage is composed of “critical  processes” that must be implemented and institutionalized in order for the  organization to achieve that stage. These critical processes are further  broken down into key practices that describe the types of activities that an  organization should be performing to successfully implement each critical  process. It is not unusual for an organization to perform key practices  from more than one maturity stage at the same time. However, our  research has shown that agency efforts to improve investment  management capabilities should focus on implementing all lower stage  practices before addressing the higher stage practices. Figure 2 provides  an overview of the five ITIM stages of maturity and the critical processes  associated with each stage.
In the ITIM framework, Stage 2 critical processes lay the foundation for  sound IT investment management by helping the agency to attain  successful, predictable, and repeatable investment management processes  at the project level. Specifically, Stage 2 encompasses building a sound  investment management foundation by establishing basic capabilities for  selecting new IT projects. This stage also involves developing the  capability to control projects so that they finish predictably within  established cost and schedule expectations and developing the capability  to identify potential exposures to risk and put in place strategies to  mitigate that risk. It also involves instituting an IT investment board,  which includes defining its membership, guidance policies, operations,  roles, responsibilities, and authorities. The basic selection processes  established in Stage 2 lay the foundation for more mature management  capabilities in Stage 3, which represents a major step forward in maturity,  in which the agency moves from project-centric processes to an  agencywide portfolio approach.
Stage 3 requires that an organization continually assess both proposed and  ongoing projects as part of a complete investment portfolio—an integrated  and competing set of investment options. It focuses on establishing a  consistent, well-defined perspective on the IT investment portfolio and  maintaining mature, integrated selection (and reselection), control, and  evaluation processes. This portfolio perspective allows decision makers to  consider the interaction among investments and the contributions to  organizational mission goals and strategies that could be made by  alternative portfolio selections, rather than focusing exclusively on the  balance between the costs and benefits of individual investments.  Organizations that have implemented Stage 2 and 3 practices have  capabilities in place that assist in establishing selection; control; and  evaluation structures, policies, procedures, and practices that are required  by the investment management provisions of the Clinger-Cohen Act.
Stages 4 and 5 require the use of evaluation techniques to continuously  improve both the investment portfolio and the investment processes in  order to better achieve strategic outcomes. At Stage 4, an organization has  the capacity to conduct IT succession activities and, therefore, can plan  and implement the deselection of obsolete, high-risk, or low-value IT  investments. An organization with Stage 5 maturity conducts proactive  monitoring for breakthrough information technologies that will enable it  to change and improve its business performance.
section_title: SSA’s Current Investment Management Approach
paragraphs: SSA’s investment management process is intended to meet the objectives  of the Clinger-Cohen Act by providing a framework for selecting,  controlling, and evaluating investments that helps to ensure it meets the  strategic and business objectives of the agency. The investment  management process is documented in the agency’s Capital Planning and  Investment Control (CPIC) Guide.
The CPIC Guide assigns the responsibility for the investment management  process to SSA executive-level managers. In this regard, the Information  Technology Advisory Board (ITAB) is responsible for assigning resources  to projects reported in the 2-year Agency IT Plan, which specifies which  projects and systems the agency will build and operate. The board, which  meets quarterly, is comprised of the deputy commissioners and other  senior executives, such as the general counsel and the Deputy  Commissioner, Budget, Finance and Management and it is chaired by the  CIO. The CIO is the key decision maker in the CPIC process. He provides  advice to the Commissioner and Deputy Commissioner of Social Security  to ensure that IT is acquired and information resources are managed in a  manner that is consistent with the policies and procedures of the Clinger- Cohen Act. The CIO is the chairman of the investment board and makes  final IT budget recommendations to the Commissioner. The Deputy  Commissioner, Systems is responsible for monitoring all development and  operations projects included in the Agency IT Plan. Each deputy  commissioner responsible for a portfolio has a portfolio manager and  portfolio team to assist in the day-to-day management of the  corresponding investment portfolio within each business component.
Table 1 identifies the key participants that have a role in the agency’s  investment management process and their responsibilities.
SSA uses its established CPIC process to manage the work years  associated with its in-house software development projects. (The  acquisition budget is managed by a separate process discussed later in this  report.) The CPIC process is as follows:    During the investment selection phase, new projects are proposed by a  sponsor––either from a business unit for mission-related projects or from  the Deputy Commissioner, Systems’ organization for supporting  acquisitions, such as telephone systems—and are assigned to 1 of 11  portfolios. Proposals that identify business needs are developed based on  the Commissioner’s priorities or gap analyses performed by each portfolio  team that identify future business needs. The ITAB issues guidelines to the  portfolio teams on the number of work years that each portfolio will have  available for projects. In response, each portfolio team develops a  prioritized list of proposed and ongoing projects within their work year  allocations. Prioritization is based on a vote by portfolio team  representatives. According to SSA’s documented procedures, prioritization  criteria can include relative benefits, costs, and risks. However, portfolio  teams have discretion in how they weigh these and any other criteria.  Next, the prioritized lists are combined into a proposed Agency IT Plan for  approval by the ITAB. The plan is comprised of proposed investments for  the next 2 fiscal years, and provides information on work year  requirements. In addition, expected benefits and return on investment are  included for new development projects. The ITAB approves or modifies  the proposed plan once a year, including allocating work years to the  portfolios. At this point, the selection phase of the annual cycle is basically  complete, though portfolio teams can propose additional projects that  arise in the middle of a cycle.
During the control phase, the Deputy Commissioner, Systems holds  monthly meetings with his staff who are assigned to monitor projects in  development. During these meetings, projects that are not meeting cost  and schedule expectations are identified, and corrective actions are  initiated. According to SSA guidance, the objective of the Deputy  Commissioner, Systems’ meetings with his staff is to resolve problems  related to underperforming projects without elevating them to the ITAB.  During the months in which ITAB quarterly meetings are scheduled, the  Deputy Commissioner, Systems meets with his staff prior to these  meetings to prepare to address concerns about investments that may be  raised during the meetings. If concerns are raised at the meeting, the  Deputy Commissioner, Systems provides information about these  investments. In addition, the ITAB receives investment profiles on the  status of each of the agency’s major IT investments. These profiles include  reports on actual and expended work years, cost, schedule, and any  variances.
During the evaluation phase, the CPIC Guide calls for the CIO to conduct  postimplementation reviews on projects that have been completed and  deployed for at least 3 months. The purpose of these reviews is to compare  actual project results against planned results in order to assess  performance and identify areas where future decision making can be  improved. Figure 3 illustrates SSA’s current investment management  process as specified in agency guidance.
section_title: SSA Has Taken Key Steps to Manage Investments, but Gaps Remain in Oversight and in Defining Policies and Procedures
paragraphs: SSA has executed a majority of the key practices—82 percent––needed to  effectively manage its IT projects as investments, but it has not fully  implemented many of the related oversight responsibilities and  procedures that our ITIM framework outlines. Of the five Stage 2 critical  processes specified by the ITIM, it has (1) established most of the key  practices needed for instituting the investment board, (2) developed  procedures for ensuring that projects meet business and user needs, (3)  established a process for selecting an investment, and (4) developed tools  for capturing investment information. However, the critical process of  providing oversight is not being fully executed. Also, the agency has made  progress in establishing the critical processes and key practices for  managing IT investments as a portfolio. It is executing 18 out of 27 key  practices from this stage of the ITIM. However, it has not established  enterprisewide portfolio selection criteria and has executed few key  practices for evaluating the portfolio. In addition, its postimplementation  reviews are not achieving key objectives. Further, a gap exists in the  agency’s management of its IT in that more than half of its budget—its  acquisition budget—is not overseen as part of the agency’s current  investment management process. While SSA has taken key steps for  managing its investments, until key practices are fully implemented and  coverage of its management processes is extended to all investments, it  will not be fully postured to ensure that its investments achieve their  intended results and address the strategic goals, objectives, and mission of  the organization.
subsections: section_title: SSA Has Established Most of the Foundation for Managing IT Investments, but It Has Not Established Some Processes and Procedures
paragraphs: At the ITIM Stage 2 level of maturity, an organization has attained  repeatable, successful IT project-level investment control and basic  selection processes. Through these processes, the organization can  identify expectation gaps early and take the appropriate steps to address  them. According to ITIM, critical processes at Stage 2 include (1) defining  IT investment board operations, (2) identifying the business needs for  each IT investment, (3) developing a basic process for selecting new IT  proposals and reselecting ongoing investments, (4) developing project- level investment control processes, and (5) collecting information about  existing investments to inform investment management decisions.
Table 2 describes the purpose of each of these Stage 2 critical processes.
Within these 5 critical processes are 38 key practices for effective project- level management. SSA has implemented 31 of these practices.
Specifically, the agency has satisfied all the key practices associated with  meeting business needs and capturing investment information and most of  those associated with instituting an investment board and selecting an  investment. However, the agency has not executed most of the key  practices related to providing investment oversight. Moreover, the agency  has not developed some policies and procedures required for the critical  process areas, including providing investment oversight.
Table 3 summarizes the status of SSA’s Stage 2 critical processes, showing  the number of associated practices that have been implemented, as they  apply to the agency’s management of its IT work year budget for in-house  projects.
The establishment of decision-making bodies or boards is a key  component of the IT investment management process. At the Stage 2 level  of maturity, organizations define one or more boards, provide resources to  support their operations, and appoint members who have expertise in both  operational and technical aspects of proposed investments. The board  operates according to a written IT investment process guide that is  tailored to the organization’s unique characteristics, thus ensuring that  consistent and effective management practices are implemented across  the organization. Once board members are selected, the organization  ensures that they are knowledgeable about policies and procedures for  managing investments. Organizations at the Stage 2 level of maturity also  take steps to ensure that executives and line managers support and carry  out the decisions of the IT investment board. An IT investment  management process guide should be an authoritative document that the  organization uses to initiate and manage IT investment processes and  should provide a comprehensive foundation for the policies and  procedures that are developed for all of the other related processes. (The  complete list of key practices is provided in table 4.)
SSA has executed seven of the eight key practices for instituting the  investment board. In particular, it has established the ITAB as its  investment board. As previously discussed, the board is chaired by the  CIO, and includes deputy commissioners and other agency senior  executives, such as the Deputy Commissioner, Budget, Finance and  Management. Further, the agency has a documented investment  governance process and provides resources for the board. Management  controls have been established for ensuring that the investment board’s  decisions are carried out.
However, the agency is not executing one of the key practices associated  with this process. The board is not implementing one of the three stages of  the IT investment governance process based on the Clinger-Cohen Act.  Specifically, it is not evaluating IT investments, including performing  postimplementation reviews. Rather, the CIO alone is assigned this  responsibility and the investment board does not receive the results of  these reviews. Until all relevant IT governance becomes the responsibility  of the ITAB, SSA may have insufficient high-level executive involvement in  its investment management process and will not benefit from the  contributions of those executives who are in the best position to make the  full range of decisions needed for the agency to carry out its mission most  effectively.
Further, although SSA has established its investment board, the policies  and procedures to define and implement the investment governance  process are not fully established for all of the key practices. For example,  the procedures for elevating underperforming investments to the board  are not established. Further, although the CIO and Deputy Commissioner,  Systems agree that the CPIC guide and other guidance they provided are  official agency documents, these documents had not been officially  approved by SSA’s management. Without policy guidance that is agreed to  and approved by all the appropriate levels of the organization, consistent  and repeatable investment management practices cannot be assured.
Table 4 summarizes our findings relative to SSA’s execution of the eight  key practices for instituting the investment board.
Defining business needs for each IT project helps to ensure that projects  and systems support the organization’s business needs and meet users’  needs. According to ITIM, effectively meeting business needs requires,  among other things, (1) documenting business needs with stated goals and  objectives; (2) identifying specific users and other beneficiaries of IT  projects and systems; (3) providing adequate resources to ensure that  projects and systems support the organization’s business needs and meet  users’ needs; and (4) periodically evaluating the alignment of IT projects  and systems with the organization’s strategic goals and objectives. (The  complete list of key practices is provided in table 5).
SSA has in place all seven key practices for meeting business needs. The  agency’s CPIC Guide and IT Planning Training Package require that  sponsors identify the current and future business needs for proposed and  ongoing projects and systems. Business needs are to be aligned with the  SSA Strategic Plan. Resources for ensuring that IT projects and systems  support the organization’s business needs and meet users’ needs include  the ITAB, project sponsors and reviewers, the Systems Planning and  Reporting System (which documents business needs information on  proposed and ongoing projects), and the project scope agreement (which  documents the business needs that the developer agrees will meet user  needs). In reviewing selected agency projects as part of our study, we  verified that the new and ongoing projects had these scope agreements.
Table 5 shows the analysis for each key practice of the critical process for  meeting business needs and summarizes the supporting evidence.
Selecting new IT proposals and reselecting ongoing investments requires a  well-defined and disciplined process to provide the agency’s investment  boards, business units, and developers with a common understanding of  the process and the cost, benefit, schedule, and risk criteria that will be  used both to select new projects and to reselect ongoing projects for  continued funding. According to ITIM, this critical process requires,  among other things, (1) making funding decisions for new proposals  according to an established process; (2) providing adequate resources for  investment selection activities; (3) using a defined selection process to  select new investments and reselect ongoing investments; (4) establishing  criteria for analyzing, prioritizing, and selecting new IT investments and  for reselecting ongoing investments; and (5) creating a process for  ensuring that the criteria change as organizational objectives change. (The  complete list of key practices is provided in table 6.)
SSA has in place 9 of 10 key practices for selecting investments. For  example, the agency has established policies and procedures for  integrating funding with the process of selecting an investment; the IT  Planning Training Package states that the ITAB is to specify the resources  available to each portfolio team for its investments. According to SSA  officials, resources are provided for selecting investments, including  managerial attention and tracking systems. Criteria have been established  for selecting and reselecting investments, including return on investment,  the business value of the investment, and investment cost.
The agency ensures that selection criteria reflect organizational goals by  aligning project selection with the organizational priorities set by the  Commissioner each year. The IT Planning Training Package and ITAB  meeting notes document the predefined selection criteria and process for  selection of new investments. We verified that the three case study  projects we reviewed were selected using the predefined selection process  and criteria, and that these funding decisions were based on the selection  information for the projects.
However, SSA is not fully executing the key practice requiring policies and  procedures for selecting new IT investment proposals. While the CPIC  Guide has policies for identifying and evaluating new IT proposals, the IT  Planning Training Package does not have documented procedures for  prioritizing investment proposals. SSA officials said they do not have  documented procedures because predefined criteria might result in not  selecting a proposal that a portfolio team determines is required for  operations. However, without predefined criteria for prioritizing  investments consistently in each portfolio, SSA risks having less critical  investments selected over investments that are more critical to  accomplishing the portfolio’s objective.
Table 6 shows the rating for each key practice required to implement the  critical process for selecting investments at the Stage 2 level of maturity  and summarizes the evidence that supports these ratings.
An organization should provide effective oversight for its IT projects  throughout all phases of their life cycles. Its investment board should  maintain adequate oversight and observe each project’s performance and  progress toward predefined cost and schedule expectations as well as  each project’s anticipated benefits and risk exposure. The investment  board should also employ early warning systems that enable it to take  corrective action at the first sign of cost, schedule, or performance  slippages. This board has ultimate responsibility for the activities within  this critical process. According to ITIM, effective project oversight  requires, among other things, (1) having written policies and procedures  for management oversight; (2) developing and maintaining an approved  project management plan for each IT project; (3) providing adequate  resources for supporting the investment board; (4) having regular reviews  by each investment board of each project’s performance against stated  expectations; and (5) ensuring that corrective actions for each  underperforming project are documented, agreed to, implemented, and  tracked until the desired outcome is achieved.
The agency is executing two of seven key practices for providing  oversight. The agency provides resources for oversight, and the board  reviews summary reports on projects’ cost and schedule performance.  Also, the agency maintains project plans, including cost and schedule  milestones for its investments.
However, the agency is not executing the remaining five key practices  related to providing oversight of IT projects. Although SSA provides the  investment board with summary data on projects’ performance related to  cost, schedule, and benefits, the board does not receive information on  projects’ risks. Also, the board does not regularly track the  implementation of corrective actions for each underperforming project.
The board’s meeting agenda allows individual deputy commissioners to  raise concerns about project performance at quarterly meetings, but,  based on our analysis of the ITAB meeting minutes, this opportunity is  infrequently exercised. Specifically, during 2007, the meeting minutes  showed that underperforming investments were discussed at only one of  the quarterly meetings. Also, SSA officials have not specified the criteria  for terminating projects that are underperforming. The Deputy  Commissioner, Systems told us that he takes corrective actions to address  underperforming projects but does not document these actions or report  them to the ITAB.
Table 7 shows the status of each key practice required to provide  investment oversight at the project level and summarizes the supporting  evidence.
To make good IT investment decisions, an organization must be able to  acquire pertinent information about each investment and store that  information in a retrievable format. During this critical process, an  organization identifies its IT assets and creates a comprehensive  repository of investment information. This repository provides  information to investment decision makers to help them evaluate the  potential impacts and opportunities created by proposed or continuing  investments. The repository can take many forms and need not be  centrally located, but the collection method should, at a minimum, identify  each IT investment and its associated components. According to ITIM,  effectively managing this repository requires, among other things, (1)  developing written policies and procedures for identifying and collecting  the information; (2) assigning responsibilities for ensuring that the  information being collected meets the needs of the investment  management process; (3) identifying IT projects and systems and  collecting relevant information to support decisions about them; and (4)  making the information easily accessible to decision makers and others.  (The complete list of key practices is provided in table 8.)
SSA has in place all six key practices associated with capturing investment  information. For example, the agency’s Project Resource Guide  documents policies and procedures for submitting, updating, and  maintaining relevant project information. One policy document, the Office  of Systems Project Management Directive, identifies project management  activities and work products for all projects approved by the investment  board. SSA’s Systems Process Improvement team is responsible for  developing and maintaining the monthly health reports on project  performance that are provided to the Deputy Commissioner, Systems to  track actual project work years. In addition, projects must be recorded in  the Systems Planning and Reporting System and each item to be  considered by the investment board must be documented, including  project dollar and work year estimates. The automated project status  reports provide comprehensive status information for all development  projects, including activities completed, activities in progress, and  activities planned.
We verified that information for three of the agency’s IT projects we  examined was collected in the Systems Planning and Reporting System  and they all had a project scope agreement, which described the business,  user, customer, and systems functions required. Also, project performance  was reported in the monthly IT project health reports for all three projects.
Table 8 summarizes the status of the six key practices for capturing  investment information.
section_title: SSA Has Established Processes for Managing Investments as an Enterprisewide Portfolio, but Key Practices Remain Not Executed
paragraphs: Once an agency has attained Stage 2 maturity, it needs to implement  critical processes for managing its investments as an enterprisewide  portfolio (Stage 3). An investment portfolio is an integrated, agencywide  collection of investments that are assessed and managed collectively  based on common criteria. Managing investments as a portfolio is a  conscious, continuous, and proactive approach to allocating limited  resources among an organization’s competing initiatives in light of the  relative benefits expected from these investments. Taking an agencywide  perspective enables an organization to consider its investments  comprehensively, so that collectively the investments optimally address  the organization’s mission, strategic goals, and objectives. Managing IT  investments as a portfolio also enables an organization to determine its  priorities and make decisions about which projects to fund and continue  to fund based on analyses of the relative organizational value and risks of  all projects, including projects that are proposed, under development, and  in operations. Although investments may initially be organized into  separate portfolios—based on, for example, business lines or life-cycle  stages—and managed by subordinate investment boards, they should  ultimately be aggregated into this enterprise-level portfolio.
According to the ITIM, Stage 3 maturity includes (1) defining the portfolio,  (2) creating the portfolio criteria, (3) evaluating the portfolio, and (4)  conducting postimplementation reviews. Table 9 summarizes the purpose  of each critical process in Stage 3.
Within these 4 critical processes are 27 key practices associated with  portfolio-level management. For the work year budget managed by its  investment review board, SSA has executed 18 of the 27 key practices.  SSA has executed all of the key practices for creating the portfolio and  most of those for defining the criteria and conducting postimplementation  reviews. However, the agency has not executed nine key practices,  including establishing enterprisewide selection criteria and managing all  of its investments as an enterprisewide portfolio. SSA has implemented  postrelease reviews of its investments, but does not include evaluations of  quantitative data and analyses, such as the investments’ contributions  toward achieving both the strategy and the objectives of the organization’s  IT strategic plan.
Table 10 summarizes the status of SSA’s Stage 3 critical processes and key  practices.
Developing an IT investment portfolio involves defining appropriate  investment cost, benefit, schedule, and risk criteria to ensure that the  organization’s strategic goals, objectives, and mission will be satisfied by  the selected investments. Portfolio selection criteria reflect the strategic  and enterprisewide focus of the organization and build on the criteria that  are used to select individual projects. When IT projects are not considered  in the context of a portfolio, criteria based on narrow, lower-level  requirements may dominate enterprisewide selection criteria.
SSA is executing five of seven key practices associated with defining the  portfolio criteria, including assigning responsibility to the ITAB for  developing and modifying portfolio guidance and providing thresholds for  selecting investments to the portfolio teams. According to SSA officials,  the agency also has adequate resources for portfolio selection activities,  including people and tools. Further, project management personnel are  aware of the portfolio selection criteria.
However, SSA is not executing two key practices. The agency has not fully  documented policies and procedures, such as key procedures for creating  and modifying IT portfolio selection criteria. Further, the investment  board approved the core criteria for selection, but it has delegated the  weighting of core criteria to the portfolio teams. This delegated approach  conflicts with the need articulated in the ITIM framework to manage  investments in a strategic, enterprisewide manner so that the investments  address not only the objectives of individual programs, or lines of  business, but also the impact that projects have on one another and the IT  portfolio’s overall benefit to the organization. Lacking complete  enterprisewide portfolio criteria, SSA risks optimizing individual business  processes while producing stovepiped systems, as well as not maximizing  overall benefits to the agency.
Table 11 shows the status for each key practice required to implement the  critical process for defining the portfolio criteria and summarizes the  evidence that supports these ratings.
At ITIM Stage 3, organizations create a portfolio of IT investments to  ensure that (1) they are analyzed according to the organization’s portfolio  selection criteria and (2) an optimal investment portfolio with manageable  risks and returns is selected and funded. According to ITIM, creating the  portfolio requires organizations to, among other things, document policies  and procedures for analyzing, selecting, and maintaining the portfolio;  provide adequate resources, including people, funding, and tools for  creating the portfolio; and capture the information used to select, control,  and evaluate the portfolio and maintain it for future reference. In creating  the portfolio, the investment board should also (1) examine the mix of  new and ongoing investments and their respective data and analyses and  select investments for funding and (2) approve or modify the performance  expectations for the IT investments they have selected. (The complete list  of key practices is provided in table 12.)
SSA is executing the seven key practices associated with creating the  portfolio. For example, according to SSA officials, the agency has  adequate resources for selecting the portfolio, including the ITAB  executives, other supporting staff, and a system that tracks proposal  information. The ITAB also considers a list of proposed IT investments  and assigns IT staffing resources to the investment portfolios.
Table 12 shows the status for each key practice required to implement the  critical process for creating the portfolio and summarizes the evidence  that supports these ratings.
This critical process builds upon the Stage 2 critical process related to  providing investment oversight by adding the elements of portfolio  performance to an organization’s investment control capacity. Compared  to less mature organizations, Stage 3 organizations will have the  foundation they need to control the risks faced by each investment and to  deliver benefits that are linked to mission performance. In addition, a  Stage 3 organization will have the benefit of good performance data  generated by Stage 2 processes. Expanding this focus to the entire  portfolio provides the organization with longer-term assurances that the IT  investment portfolio will deliver mission value at acceptable cost.
SSA has executed two of the seven key practices associated with this  process: ensuring adequate resources, including staff and tools for  reviewing the investment portfolio, and ensuring that the ITAB is familiar  with the process for evaluating and improving investments. The remaining  five key practices were not executed, partly because SSA has delegated  portfolio management and partly because it is not executing the Stage 2  prerequisite critical process, providing investment oversight, which  collects information on projects. As we have discussed, the ITAB does not  receive information on nonperforming projects, because performance  monitoring has been delegated to the Deputy Commissioner, Systems. SSA  officials agreed that they were not evaluating the portfolio as a whole.  Until SSA executes all the key practices associated with this critical  process, senior executives will not have the information they need to  determine whether the investments they have selected are delivering  mission value at the expected cost and risk.
Table 13 shows the status for each key practice required to implement the  critical process for evaluating the portfolio and summarizes the evidence  that supports these ratings.
The purpose of a postimplementation review is to evaluate an investment  after it has completed development in order to validate whether the  estimated return on investment was actually achieved. Specifically, the  review is conducted to (1) examine differences between estimated and  actual investment costs and benefits and possible ramifications for  unplanned funding needs in the future and (2) extract “lessons learned”  about the investment selection and control processes that can be used as  the basis for management improvements. Postimplementation reviews  should also be conducted for investment projects that were terminated  before completion to readily identify potential management and process  improvements.
SSA has executed four of the six key practices associated with this  process: policies and procedures are defined, adequate resources are  provided, individuals assigned to conduct postimplementation reviews are  familiar with the processes, and projects for which reviews will be  conducted are identified.
The remaining two key practices were not executed: quantitative  investment data are not collected and analyzed and lessons learned are not  conducted on investment processes for selection, control, and evaluation.  Without analyzing quantitative data on benefits achieved, SSA cannot  determine whether the project has delivered anticipated benefits. Further,  without knowledge of what benefits are actually achieved from projects,  the portfolio cannot be evaluated, and Stage 4 and 5 practices cannot be  carried out effectively. Also, without developing lessons learned from  postimplementation reviews to improve the CPIC’s select, control, and  evaluate phases, the agency will be unable to use the reviews to improve  its investment management processes.
Table 14 shows the status for each key practice required to implement the  critical process for conducting postimplementation reviews and  summarizes the evidence that supports these ratings.
section_title: More Than Half of SSA’s IT Budget Is Not Subject to Its Current Investment Management Process
paragraphs: Even though SSA is executing most Stage 2 and Stage 3 key practices for  the work year budget managed by its investment board, IT products and  services acquired with the acquisition budget ($610 million in acquisitions  in fiscal year 2008—58 percent of the IT budget) are not managed as  investments under SSA’s CPIC process, and are not reviewed by the ITAB.  These products and services include, among other things, engineering  support services, network infrastructure, mainframe capacity  infrastructure, hardware maintenance, software maintenance, local  telecom services, telephone systems maintenance, and an agencywide  support service contract.
These acquisition budget expenditures are under the overall direction of  the Deputy Commissioner, Systems and are determined by funding  requests from the business units and subsequent negotiations. Each  deputy commissioner and the associate commissioners who report to the  Deputy Commissioner, Systems, submit requests for funds based on the  unit’s acquisition needs. These requests are analyzed by the Deputy  Commissioner, Systems staff, requests are reconciled with the available  resources, a budget is developed, and the CIO reviews and signs it.
Although this process involves a large budget and important assets, it is  not subject to the CPIC select, control, and evaluate phases. For example,  acquisitions of IT products and services are not selected by a board in a  disciplined fashion, such as using the agency’s CPIC select and control  procedures, but instead are largely selected by one individual—–the  Deputy Commissioner, Systems. While the ITAB is provided a list of  proposed projects for the Agency IT Plan, the list does not include the  acquisition budget expenses associated with projects. However, the  investment board does receive a report summarizing the total amount of  the funds expended.
Agency officials gave several reasons why the acquisition budget is not  managed by the investment board. Specifically, in SSA’s view, just as the  other deputy commissioners have discretion to manage funding allocated  to their portfolios, the Deputy Commissioner, Systems should have the  same discretion to allocate funding in the infrastructure portfolio. Further,  the officials stated that many items included in this budget are very  technical and might not be well understood by senior business  management; thus, review at this level is not thought to be effective. In  addition, officials said that many items in the acquisition budget (such as  telephones) are not optional, but necessary to keep the agency running,  and thus do not require a decision process.
Given the large amount of funds involved, senior management  involvement and oversight are essential to ensure effective management of  and full accountability for acquisitions of IT products and services.  Further, until the agency manages all of its investments from an  enterprisewide perspective, it will be unable to consider its investments  comprehensively, and ensure that the investments optimally address the  organization’s mission, strategic goals, and objectives.
section_title: SSA Is Beginning Initiatives Intended to Address High-Level ITIM Processes
paragraphs: Organizations that achieve the Stage 4 level of maturity evaluate their IT  investment processes and portfolios to identify opportunities for  improvement. At the same time, these organizations are able to maintain  the mature control and selection processes that are characteristic of Stage  3 in the ITIM model. At Stage 4, organizations are capable of systematically  planning for and implementing decisions to discontinue or deselect  obsolete, high-cost, and low-value IT investments and planning for  successor investments that better support strategic goals and business  needs.
Organizations acquire Stage 5 capabilities when they create opportunities  to shape strategic outcomes by learning from other organizations and  continuously improving the manner in which they use IT to support and  improve business outcomes. Thus, organizations at Stage 5 benchmark  their IT investment processes relative to other best-in-class organizations  and conduct proactive monitoring for breakthrough information  technologies that will allow them to significantly improve business  performance.
Table 15 shows the purpose of each critical process in Stages 4 and 5.
Because the ITIM is cumulative, agencies cannot fully implement Stage 4  and 5 processes without first executing Stage 2 and 3. Nonetheless, SSA  officials said they have begun two initiatives related to a Stage 4 objective  (improving the investment process) and a Stage 5 objective (leveraging IT  for strategic outcomes). The first initiative, Application Portfolio  Management, was established to improve the agency’s information  technology decision-making process. When fully implemented, the  initiative is intended to address the Stage 4 critical process (managing the  succession of information systems). The Application Portfolio  Management review is used to analyze and quantify the health of existing  software applications to determine whether they are eligible to be retired,  renovated, or maintained. According to the agency, SSA has released  version 1.0 of Application Portfolio Management and has begun identifying  software applications that are eligible to be retired, renovated, or  maintained.
The second initiative, the Technology Infusion Process, is beginning to  address the second Stage 5 critical process—using IT to drive strategic  business change. The Technology Infusion Process was established to  evaluate and implement new technologies or new uses of existing  technologies that will facilitate SSA’s ability to achieve the agency’s  strategic goals. SSA has begun to identify various technologies for  research and has begun to review technology projects submitted by a  component sponsor as candidates for the Technology Infusion Process.  However, Application Portfolio Management has not identified hardware  or infrastructure projects for retirement, renovation, or maintenance.
section_title: Conclusions
paragraphs: Given the importance of IT to SSA’s mission, it is vital that the agency  manages its investments effectively. To its credit, SSA has established  many of the basic practices needed to build the foundation for managing  its projects as investments and for managing its investments as a portfolio.  However, weaknesses remain. For example, although the agency has  established an investment board as the decision-making body that defines  and implements the investment governance process, key policies and  procedures for the investment management process are not fully defined,  and the investment board does not provide oversight of underperforming  investments. Moreover, the agency does not track corrective actions for its  underperforming projects. SSA has also taken the important step of  creating an investment portfolio. However, it has not fully established the  policies and procedures essential to managing the portfolio, such as for  reviewing, evaluating, and improving the performance of the portfolio.  Further, the agency’s postimplementation reviews do not evaluate whether  the expected benefits were achieved or identified lessons learned for  improving the investment management processes.
Moreover, the agency’s IT acquisition budget, used to acquire IT-related  products and services, is not allocated or overseen by the investment  board and is not managed using investment governance processes. Failure  to apply these processes to the acquisition budget makes it impossible for  SSA executive management tasked with overseeing the agency’s  investments to ensure that this portion of the budget is spent in the most  efficient and effective manner.
section_title: Recommendations for Executive Action
paragraphs: To strengthen SSA’s investment management capability and address  weaknesses discussed in this report, we recommend that the  Commissioner of Social Security take the following actions:  To fully implement the key practices for building the investment  foundation (Stage 2) for current and project-level future IT investments’  success, direct the Chief Information Officer to    establish comprehensive policies and procedures for defining the  investment governance process that specify (1) investment board  operating procedures, (2) delegations of authority, and (3) criteria for  prioritizing new and ongoing investments;   strengthen and expand the board’s oversight responsibilities for  underperforming projects and evaluations of projects; and   establish a mechanism for tracking corrective actions for underperforming  investments.
To fully implement the key practices for developing a complete investment  portfolio (Stage 3), direct the Chief Information Officer to    establish policies and procedures for defining the portfolio criteria;    establish portfolio-level performance evaluation policies and procedures  and criteria for assessing portfolio performance; and    evaluate quantitative measures during postimplementation reviews, and  lessons learned for improving select, control, and evaluate processes.
To ensure senior management involvement and full accountability for the  agency’s investments, direct the Chief Information Officer to    develop and implement policies and procedures to manage IT acquisitions  as investments and manage them using the investment management  framework.
section_title: Agency Comments and Our Evaluation
paragraphs: The Commissioner of Social Security provided written comments on a  draft of this report (comments are reproduced in appendix II). In its  comments SSA agreed with six of our recommendations and disagreed  with one.
Regarding those recommendations with which it agreed, SSA stated that it  had initiated actions to document existing investment management  processes and that it plans to strengthen and expand the role of the  investment board in the oversight of underperforming projects and in the  evaluations of investments. The agency also stated that it plans to  establish a mechanism for tracking corrective actions for underperforming  investments. Further, to achieve a complete IT investment portfolio, SSA  plans to establish procedures for defining the portfolio criteria within the  context of the existing delegation of authority to the portfolio sponsors. In  addition, regarding postimplementation reviews, the agency stated it plans  to evaluate quantitative measures and lessons learned for improving  select, control, and evaluate processes.
SSA disagreed with our recommendation that it develop policies and  procedures for managing its IT acquisitions as investments and manage  them using the investment board and investment management processes.  The agency stated that its existing budget development process already  treats these acquisitions as investments and maintains them by using an  investment management framework, though not the one described in our  ITIM framework. However, under SSA’s current process, these  acquisitions are not subject to the agency’s investment management select,  control, and evaluate processes and are not managed by its investment  board. Given that the IT products and services make up the majority of  SSA’s IT budget, the investment board’s involvement is essential to helping  ensure effective management of and full accountability for acquisitions of  IT products and services. As we previously noted, by the agency not  applying its investment management process to the acquisition budget, it  limits the ability of SSA’s executive management tasked with overseeing  the agency’s investments to ensure that this portion of the budget is spent  in the most efficient and effective manner.
SSA also provided technical and other comments, which we have  incorporated as appropriate. Among the comments, the agency stated that  it had pursued the adoption of industry best practices developed by  institutions such as the Software Engineering Institute of Carnegie Mellon  University and believed it had achieved comprehensive and mature IT  management practices. SSA added that our assessment had provided an  opportunity for the agency to think carefully about many aspects of its  investment management processes, and had enabled it to better  understand the strengths and weaknesses of its current approach to  managing investments.
As agreed with your office, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days from the  date of this letter. At that time, we will send copies of the report to  interested congressional committees, the Director of the Office of  Management and Budget, and the Commissioner of Social Security. Copies  of this report will be made available to other interested parties on request.  This report will also be available at no charge on our Web site at  http://www.gao.gov.
Should you or your staff have questions on matters discussed in this  report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix III.
section_title: Appendix I: Objective, Scope, and Methodology
paragraphs: Our objective was to determine whether Social Security Administration’s  (SSA) investment management approach is consistent with leading  investment management best practices. Our analysis was based on best  practices contained in GAO’s Information Technology Investment  Management (ITIM) framework and the framework’s associated  evaluation methodology, and focused on the agency’s implementation of  critical processes and key practices for managing its business systems  investments.
To address our objective, we asked the agency to complete a self-  assessment of its investment management process and provide the  supporting documentation. We then reviewed the results of the agency’s  self-assessment of Stages 2 and 3 practices and compared them against  our ITIM framework. We focused on Stages 2 and 3 because these stages  represent the processes needed to meet the standards of the Clinger- Cohen Act and they establish the foundation for effective acquisition  management. We also validated and updated the results of the self- assessment through document reviews and interviews with officials, such  as the CIO, Deputy Commissioner, Systems, and other staff in these  offices. In doing so, we reviewed written policies, procedures, and  guidance that provided evidence of documented practices, including SSA’s  IT Capital Planning and Investment Control (CPIC) Guide and IT Planning  Training Package. We also reviewed the fiscal year 2008-2009 Agency IT  Plan and the board’s meeting minutes and other documentation providing  evidence of executed practices.
We compared the evidence collected from our document reviews and  interviews to the key practices in ITIM. We rated the key practices as  “executed” on the basis of whether the agency demonstrated (by providing  evidence of performance) that it had met the criteria of the key practice. A  key practice was rated as “not executed” when we found insufficient  evidence of a practice during the review or when we determined that there  were significant weaknesses in SSA’s execution of the key practice. In  addition, SSA was provided with the opportunity to produce evidence for  key practices rated as “not executed.”
We did not assess investments made with SSA’s IT acquisition budget  because SSA acknowledged that the acquisition budget is not managed  using SSA’s investment management process. This budget includes items  that are not projects, but are technology items that support projects, or  general infrastructure such as mainframe computers, desktop computers,  data storage, or telecommunications services.
As part of our analysis, we selected three IT projects as case studies to  verify whether certain critical processes and key practices were being  applied. SSA officials participated in the selection of these case studies.  We selected projects that (1) supported different SSA functional areas, (2)  were in different life-cycle phases, and (3) involved different funding  amounts. These three projects are described below.
Ready Retirement is a project that automates the processing of  retirement applications. It allows individuals to file for benefits using a  Web interface. This investment is expected to increase online claims filing,  minimize the number of recontacts required to complete an application,  and provide progress indicators to inform applicants of where they are in  the application process. Ready Retirement is intended to prepare the  agency for the growing retirement workload expected as baby boomers  become eligible for retirement by enabling applicants to prepare their own  applications. According to the agency, this project is estimated to require  about 27 staff years for fiscal year 2008, which corresponds to costs of  about $3.1 million.
Appeals Council Case Processing is a software development project  that automates the handling of case files in appeals of disability  determinations. It is intended to provide the capability to process all  disability cases electronically at all adjudicative levels. Further, the system  can obtain claims, medical evidence, and supporting documentation over  the Internet in a secured environment. The users have the capability to  complete all disability case-related actions electronically. This project is  expected to eliminate backlogs, reduce reliance on paper folders, and  increase decisional and documentation accuracy and decisional  consistency. SSA estimates that this project will require about 56 staff  years in fiscal year 2008, which corresponds to costs of about $6.4 million.
Mainframe Architecture is a large infrastructure investment that  involves both developmental and operations and maintenance  components, and includes both software development and hardware.
SSA’s mainframes are the hardware platform for many critical systems.  The agency states that its objective is to provide 100 percent reliability and  availability to mainframe users. Tasks for the project include  enhancements to hardware and software technology, annual upgrades to  the operating system, routine additions to mainframe capacity dictated by  workload growth, and migration to the current software versions of over  100 vendor products. The agency estimates that this project will require  about 54 staff years for developmental projects and about 28 staff years for  operations and maintenance work in fiscal year 2008, which corresponds  to costs of about $9.5 million. In addition, the project is expected to  require about $84 million from the acquisition budget for a total cost of  about $94 million.
For these projects, we reviewed project management documentation, such  as project proposals, project plans, and performance reports on costs and  benefits. We also conducted interviews with the agency’s CIO and Deputy  Commissioner, Systems, as well as other managers responsible for the  agency’s investment management processes.
We conducted our work at SSA headquarters in Baltimore, Maryland from  October 2007 through September 2008 in accordance with generally  accepted government auditing standards. Those standards require that we  plan and perform the audit to obtain sufficient, appropriate evidence to  provide a reasonable basis for our findings and conclusions based on our  audit objectives. We believe that the evidence obtained provides a  reasonable basis for our findings and conclusions based on our audit  objective.
section_title: Appendix II: Comments from Social Security Administration
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact person named above, key contributors to this  report were Cynthia Scott, Assistant Director; Faiza Baluch; Rebecca  LaPaze; Sabine Paul; Tomás Ramirez; Glenn Spiegel; Niti Tandon; and  Daniel Wexler.