Information Technology: SSA Has Taken Key Steps for Managing Its Investments, but Needs to Strengthen Oversight and Fully Define Policies and Procedures
Released: 2008-10-14T00:00:00

Letter

Background

SSA’s mission is to advance the nation’s economic security through compassionate and vigilant leadership in shaping and managing America’s Social Security programs. This includes one of the nation’s largest entitlement programs––federal Old-Age, Survivors, and Disability Insurance benefits––commonly referred to as Social Security. The program provides monthly benefits to retired and disabled workers, their spouses and children, and the survivors of insured workers. SSA also administers Supplemental Security Income, a needs-based program for the aged, blind, and disabled that pays monthly benefits to individuals. Over 54 million people, one-sixth of the total U.S. population, receive monthly Social Security or Supplemental Security Income benefit payments. The agency’s estimated 2008 budget of about $657 billion includes an administrative budget of $9.7 billion to support these programs, including about $1 billion for IT.

Organizationally, SSA is headed by the Commissioner, who is assisted by a deputy commissioner and various other executive officials, including the Deputy Commissioner, Budget, Finance and Management; Chief Information Officer (CIO); Chief Strategic Officer; and nine deputy commissioners responsible for the agency’s various business components. The organizational structure of the agency is depicted in figure 1.

The Commissioner is supported by approximately 60,000 employees located at headquarters and throughout a decentralized network of over 1,400 offices that include regional offices, field offices, teleservice centers, processing centers, state Disability Determination Services, program service centers, and hearing offices. Of these employees, approximately 3,300 IT staff and contractors are assigned to the Office of Deputy Commissioner, Systems. According to SSA, its organizational structure is designed to provide timely, accurate, and responsive service to the American public.

SSA Relies on IT to Deliver Services

The agency relies extensively on information technology to administer its programs and to support related administrative needs. In this regard, IT is used to, among other things: evaluate evidence and make determinations of eligibility for benefits on issue new and replacement Social Security cards, process earnings items for crediting to workers’ earnings records, handle millions of transactions on SSA’s toll-free telephone number, issue Social Security statements, process continuing disability reviews, and process nondisability Supplemental Security Income redeterminations.

The agency’s IT budget for fiscal year 2008 is approximately $1 billion. Of this amount, $400 million is for work year support of software development projects in the Office of Deputy Commissioner, Systems and about $610 million is for acquisition of IT-related products and services. The agency expects to spend about 80 percent of its acquisition budget on infrastructure.

Investment Management Is Critical to Effective Use of IT

A corporate approach to IT investment management is characteristic of successful public and private organizations. Recognizing this, Congress enacted the Clinger-Cohen Act of 1996, which requires the Office of Management and Budget (OMB) to establish processes to analyze, track, and evaluate the risks and results of major capital investments in IT systems made by executive agencies. In implementing the Clinger-Cohen Act and other statutes, OMB has developed policy and issued guidance for the planning, budgeting, acquisition, and management of federal capital assets. We have also issued guidance in this area that defines institutional structures, such as investment boards; processes for developing information on investments (such as cost/benefit); and practices to inform management decisions (such as whether a given investment is aligned with an enterprise architecture).

IT Investment Management: A Brief Description

IT investment management is a process for linking IT investment decisions to an organization’s strategic objectives and business plans. Consistent with this, the federal approach to IT investment management focuses on selecting, controlling, and evaluating investments in a manner that minimizes risks while maximizing the return on investment.

During the selection phase, the organization (1) identifies and analyzes each project’s risks and returns before committing significant funds to any project and (2) selects those IT projects that will best support its mission needs.

During the control phase, the organization ensures that projects, as they develop and investment expenditures continue, meet mission needs at the expected levels of cost and risk. If the project is not meeting expectations or if problems arise, steps are quickly taken to address the deficiencies.

During the evaluation phase, expected results are compared with actual results after a project has been fully implemented. This comparison is done to (1) assess the project’s impact on mission performance, (2) identify any changes or modifications to the project that may be needed, and (3) revise the investment management process based on lessons learned.

Overview of GAO’s ITIM Maturity Framework

Our ITIM framework consists of five progressive stages of maturity for any given agency relative to selecting, controlling, and evaluating its investment management capabilities. (See fig. 2 for the five ITIM stages of maturity.) This framework is grounded in our research of IT investment management practices of leading private and public sector organizations. The framework can be used to assess the maturity of an agency’s investment management processes and as a tool for organizational improvement. The overriding purpose of the framework is to encourage investment processes that increase business value and mission performance, reduce risk, and increase accountability and transparency in the decision process. We have used the framework in many of our evaluations, and a number of agencies have adopted it.

ITIM’s five maturity stages represent steps toward achieving stable and mature processes for managing IT investments. Each stage builds on the lower stages and the successful attainment of each stage leads to improvement in the organization’s ability to manage its investments. With the exception of Stage 1, each maturity stage is composed of “critical processes” that must be implemented and institutionalized in order for the organization to achieve that stage. These critical processes are further broken down into key practices that describe the types of activities that an organization should be performing to successfully implement each critical process. It is not unusual for an organization to perform key practices from more than one maturity stage at the same time. However, our research has shown that agency efforts to improve investment management capabilities should focus on implementing all lower stage practices before addressing the higher stage practices. Figure 2 provides an overview of the five ITIM stages of maturity and the critical processes associated with each stage.

In the ITIM framework, Stage 2 critical processes lay the foundation for sound IT investment management by helping the agency to attain successful, predictable, and repeatable investment management processes at the project level. Specifically, Stage 2 encompasses building a sound investment management foundation by establishing basic capabilities for selecting new IT projects. This stage also involves developing the capability to control projects so that they finish predictably within established cost and schedule expectations and developing the capability to identify potential exposures to risk and put in place strategies to mitigate that risk. It also involves instituting an IT investment board, which includes defining its membership, guidance policies, operations, roles, responsibilities, and authorities. The basic selection processes established in Stage 2 lay the foundation for more mature management capabilities in Stage 3, which represents a major step forward in maturity, in which the agency moves from project-centric processes to an agencywide portfolio approach.

Stage 3 requires that an organization continually assess both proposed and ongoing projects as part of a complete investment portfolio—an integrated and competing set of investment options. It focuses on establishing a consistent, well-defined perspective on the IT investment portfolio and maintaining mature, integrated selection (and reselection), control, and evaluation processes. This portfolio perspective allows decision makers to consider the interaction among investments and the contributions to organizational mission goals and strategies that could be made by alternative portfolio selections, rather than focusing exclusively on the balance between the costs and benefits of individual investments. Organizations that have implemented Stage 2 and 3 practices have capabilities in place that assist in establishing selection; control; and evaluation structures, policies, procedures, and practices that are required by the investment management provisions of the Clinger-Cohen Act.

Stages 4 and 5 require the use of evaluation techniques to continuously improve both the investment portfolio and the investment processes in order to better achieve strategic outcomes. At Stage 4, an organization has the capacity to conduct IT succession activities and, therefore, can plan and implement the deselection of obsolete, high-risk, or low-value IT investments. An organization with Stage 5 maturity conducts proactive monitoring for breakthrough information technologies that will enable it to change and improve its business performance.

SSA’s Current Investment Management Approach

SSA’s investment management process is intended to meet the objectives of the Clinger-Cohen Act by providing a framework for selecting, controlling, and evaluating investments that helps to ensure it meets the strategic and business objectives of the agency. The investment management process is documented in the agency’s Capital Planning and Investment Control (CPIC) Guide.

The CPIC Guide assigns the responsibility for the investment management process to SSA executive-level managers. In this regard, the Information Technology Advisory Board (ITAB) is responsible for assigning resources to projects reported in the 2-year Agency IT Plan, which specifies which projects and systems the agency will build and operate. The board, which meets quarterly, is comprised of the deputy commissioners and other senior executives, such as the general counsel and the Deputy Commissioner, Budget, Finance and Management and it is chaired by the CIO. The CIO is the key decision maker in the CPIC process. He provides advice to the Commissioner and Deputy Commissioner of Social Security to ensure that IT is acquired and information resources are managed in a manner that is consistent with the policies and procedures of the Clinger- Cohen Act. The CIO is the chairman of the investment board and makes final IT budget recommendations to the Commissioner. The Deputy Commissioner, Systems is responsible for monitoring all development and operations projects included in the Agency IT Plan. Each deputy commissioner responsible for a portfolio has a portfolio manager and portfolio team to assist in the day-to-day management of the corresponding investment portfolio within each business component.

Table 1 identifies the key participants that have a role in the agency’s investment management process and their responsibilities.

SSA uses its established CPIC process to manage the work years associated with its in-house software development projects. (The acquisition budget is managed by a separate process discussed later in this report.) The CPIC process is as follows: During the investment selection phase, new projects are proposed by a sponsor––either from a business unit for mission-related projects or from the Deputy Commissioner, Systems’ organization for supporting acquisitions, such as telephone systems—and are assigned to 1 of 11 portfolios. Proposals that identify business needs are developed based on the Commissioner’s priorities or gap analyses performed by each portfolio team that identify future business needs. The ITAB issues guidelines to the portfolio teams on the number of work years that each portfolio will have available for projects. In response, each portfolio team develops a prioritized list of proposed and ongoing projects within their work year allocations. Prioritization is based on a vote by portfolio team representatives. According to SSA’s documented procedures, prioritization criteria can include relative benefits, costs, and risks. However, portfolio teams have discretion in how they weigh these and any other criteria. Next, the prioritized lists are combined into a proposed Agency IT Plan for approval by the ITAB. The plan is comprised of proposed investments for the next 2 fiscal years, and provides information on work year requirements. In addition, expected benefits and return on investment are included for new development projects. The ITAB approves or modifies the proposed plan once a year, including allocating work years to the portfolios. At this point, the selection phase of the annual cycle is basically complete, though portfolio teams can propose additional projects that arise in the middle of a cycle.

During the control phase, the Deputy Commissioner, Systems holds monthly meetings with his staff who are assigned to monitor projects in development. During these meetings, projects that are not meeting cost and schedule expectations are identified, and corrective actions are initiated. According to SSA guidance, the objective of the Deputy Commissioner, Systems’ meetings with his staff is to resolve problems related to underperforming projects without elevating them to the ITAB. During the months in which ITAB quarterly meetings are scheduled, the Deputy Commissioner, Systems meets with his staff prior to these meetings to prepare to address concerns about investments that may be raised during the meetings. If concerns are raised at the meeting, the Deputy Commissioner, Systems provides information about these investments. In addition, the ITAB receives investment profiles on the status of each of the agency’s major IT investments. These profiles include reports on actual and expended work years, cost, schedule, and any variances.

During the evaluation phase, the CPIC Guide calls for the CIO to conduct postimplementation reviews on projects that have been completed and deployed for at least 3 months. The purpose of these reviews is to compare actual project results against planned results in order to assess performance and identify areas where future decision making can be improved. Figure 3 illustrates SSA’s current investment management process as specified in agency guidance.

SSA Has Taken Key Steps to Manage Investments, but Gaps Remain in Oversight and in Defining Policies and Procedures

SSA has executed a majority of the key practices—82 percent––needed to effectively manage its IT projects as investments, but it has not fully implemented many of the related oversight responsibilities and procedures that our ITIM framework outlines. Of the five Stage 2 critical processes specified by the ITIM, it has (1) established most of the key practices needed for instituting the investment board, (2) developed procedures for ensuring that projects meet business and user needs, (3) established a process for selecting an investment, and (4) developed tools for capturing investment information. However, the critical process of providing oversight is not being fully executed. Also, the agency has made progress in establishing the critical processes and key practices for managing IT investments as a portfolio. It is executing 18 out of 27 key practices from this stage of the ITIM. However, it has not established enterprisewide portfolio selection criteria and has executed few key practices for evaluating the portfolio. In addition, its postimplementation reviews are not achieving key objectives. Further, a gap exists in the agency’s management of its IT in that more than half of its budget—its acquisition budget—is not overseen as part of the agency’s current investment management process. While SSA has taken key steps for managing its investments, until key practices are fully implemented and coverage of its management processes is extended to all investments, it will not be fully postured to ensure that its investments achieve their intended results and address the strategic goals, objectives, and mission of the organization.

SSA Has Established Most of the Foundation for Managing IT Investments, but It Has Not Established Some Processes and Procedures

At the ITIM Stage 2 level of maturity, an organization has attained repeatable, successful IT project-level investment control and basic selection processes. Through these processes, the organization can identify expectation gaps early and take the appropriate steps to address them. According to ITIM, critical processes at Stage 2 include (1) defining IT investment board operations, (2) identifying the business needs for each IT investment, (3) developing a basic process for selecting new IT proposals and reselecting ongoing investments, (4) developing project- level investment control processes, and (5) collecting information about existing investments to inform investment management decisions.

Table 2 describes the purpose of each of these Stage 2 critical processes.

Within these 5 critical processes are 38 key practices for effective project- level management. SSA has implemented 31 of these practices.

Specifically, the agency has satisfied all the key practices associated with meeting business needs and capturing investment information and most of those associated with instituting an investment board and selecting an investment. However, the agency has not executed most of the key practices related to providing investment oversight. Moreover, the agency has not developed some policies and procedures required for the critical process areas, including providing investment oversight.

Table 3 summarizes the status of SSA’s Stage 2 critical processes, showing the number of associated practices that have been implemented, as they apply to the agency’s management of its IT work year budget for in-house projects.

The establishment of decision-making bodies or boards is a key component of the IT investment management process. At the Stage 2 level of maturity, organizations define one or more boards, provide resources to support their operations, and appoint members who have expertise in both operational and technical aspects of proposed investments. The board operates according to a written IT investment process guide that is tailored to the organization’s unique characteristics, thus ensuring that consistent and effective management practices are implemented across the organization. Once board members are selected, the organization ensures that they are knowledgeable about policies and procedures for managing investments. Organizations at the Stage 2 level of maturity also take steps to ensure that executives and line managers support and carry out the decisions of the IT investment board. An IT investment management process guide should be an authoritative document that the organization uses to initiate and manage IT investment processes and should provide a comprehensive foundation for the policies and procedures that are developed for all of the other related processes. (The complete list of key practices is provided in table 4.)

SSA has executed seven of the eight key practices for instituting the investment board. In particular, it has established the ITAB as its investment board. As previously discussed, the board is chaired by the CIO, and includes deputy commissioners and other agency senior executives, such as the Deputy Commissioner, Budget, Finance and Management. Further, the agency has a documented investment governance process and provides resources for the board. Management controls have been established for ensuring that the investment board’s decisions are carried out.

However, the agency is not executing one of the key practices associated with this process. The board is not implementing one of the three stages of the IT investment governance process based on the Clinger-Cohen Act. Specifically, it is not evaluating IT investments, including performing postimplementation reviews. Rather, the CIO alone is assigned this responsibility and the investment board does not receive the results of these reviews. Until all relevant IT governance becomes the responsibility of the ITAB, SSA may have insufficient high-level executive involvement in its investment management process and will not benefit from the contributions of those executives who are in the best position to make the full range of decisions needed for the agency to carry out its mission most effectively.

Further, although SSA has established its investment board, the policies and procedures to define and implement the investment governance process are not fully established for all of the key practices. For example, the procedures for elevating underperforming investments to the board are not established. Further, although the CIO and Deputy Commissioner, Systems agree that the CPIC guide and other guidance they provided are official agency documents, these documents had not been officially approved by SSA’s management. Without policy guidance that is agreed to and approved by all the appropriate levels of the organization, consistent and repeatable investment management practices cannot be assured.

Table 4 summarizes our findings relative to SSA’s execution of the eight key practices for instituting the investment board.

Defining business needs for each IT project helps to ensure that projects and systems support the organization’s business needs and meet users’ needs. According to ITIM, effectively meeting business needs requires, among other things, (1) documenting business needs with stated goals and objectives; (2) identifying specific users and other beneficiaries of IT projects and systems; (3) providing adequate resources to ensure that projects and systems support the organization’s business needs and meet users’ needs; and (4) periodically evaluating the alignment of IT projects and systems with the organization’s strategic goals and objectives. (The complete list of key practices is provided in table 5).

SSA has in place all seven key practices for meeting business needs. The agency’s CPIC Guide and IT Planning Training Package require that sponsors identify the current and future business needs for proposed and ongoing projects and systems. Business needs are to be aligned with the SSA Strategic Plan. Resources for ensuring that IT projects and systems support the organization’s business needs and meet users’ needs include the ITAB, project sponsors and reviewers, the Systems Planning and Reporting System (which documents business needs information on proposed and ongoing projects), and the project scope agreement (which documents the business needs that the developer agrees will meet user needs). In reviewing selected agency projects as part of our study, we verified that the new and ongoing projects had these scope agreements.

Table 5 shows the analysis for each key practice of the critical process for meeting business needs and summarizes the supporting evidence.

Selecting new IT proposals and reselecting ongoing investments requires a well-defined and disciplined process to provide the agency’s investment boards, business units, and developers with a common understanding of the process and the cost, benefit, schedule, and risk criteria that will be used both to select new projects and to reselect ongoing projects for continued funding. According to ITIM, this critical process requires, among other things, (1) making funding decisions for new proposals according to an established process; (2) providing adequate resources for investment selection activities; (3) using a defined selection process to select new investments and reselect ongoing investments; (4) establishing criteria for analyzing, prioritizing, and selecting new IT investments and for reselecting ongoing investments; and (5) creating a process for ensuring that the criteria change as organizational objectives change. (The complete list of key practices is provided in table 6.)

SSA has in place 9 of 10 key practices for selecting investments. For example, the agency has established policies and procedures for integrating funding with the process of selecting an investment; the IT Planning Training Package states that the ITAB is to specify the resources available to each portfolio team for its investments. According to SSA officials, resources are provided for selecting investments, including managerial attention and tracking systems. Criteria have been established for selecting and reselecting investments, including return on investment, the business value of the investment, and investment cost.

The agency ensures that selection criteria reflect organizational goals by aligning project selection with the organizational priorities set by the Commissioner each year. The IT Planning Training Package and ITAB meeting notes document the predefined selection criteria and process for selection of new investments. We verified that the three case study projects we reviewed were selected using the predefined selection process and criteria, and that these funding decisions were based on the selection information for the projects.

However, SSA is not fully executing the key practice requiring policies and procedures for selecting new IT investment proposals. While the CPIC Guide has policies for identifying and evaluating new IT proposals, the IT Planning Training Package does not have documented procedures for prioritizing investment proposals. SSA officials said they do not have documented procedures because predefined criteria might result in not selecting a proposal that a portfolio team determines is required for operations. However, without predefined criteria for prioritizing investments consistently in each portfolio, SSA risks having less critical investments selected over investments that are more critical to accomplishing the portfolio’s objective.

Table 6 shows the rating for each key practice required to implement the critical process for selecting investments at the Stage 2 level of maturity and summarizes the evidence that supports these ratings.

An organization should provide effective oversight for its IT projects throughout all phases of their life cycles. Its investment board should maintain adequate oversight and observe each project’s performance and progress toward predefined cost and schedule expectations as well as each project’s anticipated benefits and risk exposure. The investment board should also employ early warning systems that enable it to take corrective action at the first sign of cost, schedule, or performance slippages. This board has ultimate responsibility for the activities within this critical process. According to ITIM, effective project oversight requires, among other things, (1) having written policies and procedures for management oversight; (2) developing and maintaining an approved project management plan for each IT project; (3) providing adequate resources for supporting the investment board; (4) having regular reviews by each investment board of each project’s performance against stated expectations; and (5) ensuring that corrective actions for each underperforming project are documented, agreed to, implemented, and tracked until the desired outcome is achieved.

The agency is executing two of seven key practices for providing oversight. The agency provides resources for oversight, and the board reviews summary reports on projects’ cost and schedule performance. Also, the agency maintains project plans, including cost and schedule milestones for its investments.

However, the agency is not executing the remaining five key practices related to providing oversight of IT projects. Although SSA provides the investment board with summary data on projects’ performance related to cost, schedule, and benefits, the board does not receive information on projects’ risks. Also, the board does not regularly track the implementation of corrective actions for each underperforming project.

The board’s meeting agenda allows individual deputy commissioners to raise concerns about project performance at quarterly meetings, but, based on our analysis of the ITAB meeting minutes, this opportunity is infrequently exercised. Specifically, during 2007, the meeting minutes showed that underperforming investments were discussed at only one of the quarterly meetings. Also, SSA officials have not specified the criteria for terminating projects that are underperforming. The Deputy Commissioner, Systems told us that he takes corrective actions to address underperforming projects but does not document these actions or report them to the ITAB.

Table 7 shows the status of each key practice required to provide investment oversight at the project level and summarizes the supporting evidence.

To make good IT investment decisions, an organization must be able to acquire pertinent information about each investment and store that information in a retrievable format. During this critical process, an organization identifies its IT assets and creates a comprehensive repository of investment information. This repository provides information to investment decision makers to help them evaluate the potential impacts and opportunities created by proposed or continuing investments. The repository can take many forms and need not be centrally located, but the collection method should, at a minimum, identify each IT investment and its associated components. According to ITIM, effectively managing this repository requires, among other things, (1) developing written policies and procedures for identifying and collecting the information; (2) assigning responsibilities for ensuring that the information being collected meets the needs of the investment management process; (3) identifying IT projects and systems and collecting relevant information to support decisions about them; and (4) making the information easily accessible to decision makers and others. (The complete list of key practices is provided in table 8.)

SSA has in place all six key practices associated with capturing investment information. For example, the agency’s Project Resource Guide documents policies and procedures for submitting, updating, and maintaining relevant project information. One policy document, the Office of Systems Project Management Directive, identifies project management activities and work products for all projects approved by the investment board. SSA’s Systems Process Improvement team is responsible for developing and maintaining the monthly health reports on project performance that are provided to the Deputy Commissioner, Systems to track actual project work years. In addition, projects must be recorded in the Systems Planning and Reporting System and each item to be considered by the investment board must be documented, including project dollar and work year estimates. The automated project status reports provide comprehensive status information for all development projects, including activities completed, activities in progress, and activities planned.

We verified that information for three of the agency’s IT projects we examined was collected in the Systems Planning and Reporting System and they all had a project scope agreement, which described the business, user, customer, and systems functions required. Also, project performance was reported in the monthly IT project health reports for all three projects.

Table 8 summarizes the status of the six key practices for capturing investment information.

SSA Has Established Processes for Managing Investments as an Enterprisewide Portfolio, but Key Practices Remain Not Executed

Once an agency has attained Stage 2 maturity, it needs to implement critical processes for managing its investments as an enterprisewide portfolio (Stage 3). An investment portfolio is an integrated, agencywide collection of investments that are assessed and managed collectively based on common criteria. Managing investments as a portfolio is a conscious, continuous, and proactive approach to allocating limited resources among an organization’s competing initiatives in light of the relative benefits expected from these investments. Taking an agencywide perspective enables an organization to consider its investments comprehensively, so that collectively the investments optimally address the organization’s mission, strategic goals, and objectives. Managing IT investments as a portfolio also enables an organization to determine its priorities and make decisions about which projects to fund and continue to fund based on analyses of the relative organizational value and risks of all projects, including projects that are proposed, under development, and in operations. Although investments may initially be organized into separate portfolios—based on, for example, business lines or life-cycle stages—and managed by subordinate investment boards, they should ultimately be aggregated into this enterprise-level portfolio.

According to the ITIM, Stage 3 maturity includes (1) defining the portfolio, (2) creating the portfolio criteria, (3) evaluating the portfolio, and (4) conducting postimplementation reviews. Table 9 summarizes the purpose of each critical process in Stage 3.

Within these 4 critical processes are 27 key practices associated with portfolio-level management. For the work year budget managed by its investment review board, SSA has executed 18 of the 27 key practices. SSA has executed all of the key practices for creating the portfolio and most of those for defining the criteria and conducting postimplementation reviews. However, the agency has not executed nine key practices, including establishing enterprisewide selection criteria and managing all of its investments as an enterprisewide portfolio. SSA has implemented postrelease reviews of its investments, but does not include evaluations of quantitative data and analyses, such as the investments’ contributions toward achieving both the strategy and the objectives of the organization’s IT strategic plan.

Table 10 summarizes the status of SSA’s Stage 3 critical processes and key practices.

Developing an IT investment portfolio involves defining appropriate investment cost, benefit, schedule, and risk criteria to ensure that the organization’s strategic goals, objectives, and mission will be satisfied by the selected investments. Portfolio selection criteria reflect the strategic and enterprisewide focus of the organization and build on the criteria that are used to select individual projects. When IT projects are not considered in the context of a portfolio, criteria based on narrow, lower-level requirements may dominate enterprisewide selection criteria.

SSA is executing five of seven key practices associated with defining the portfolio criteria, including assigning responsibility to the ITAB for developing and modifying portfolio guidance and providing thresholds for selecting investments to the portfolio teams. According to SSA officials, the agency also has adequate resources for portfolio selection activities, including people and tools. Further, project management personnel are aware of the portfolio selection criteria.

However, SSA is not executing two key practices. The agency has not fully documented policies and procedures, such as key procedures for creating and modifying IT portfolio selection criteria. Further, the investment board approved the core criteria for selection, but it has delegated the weighting of core criteria to the portfolio teams. This delegated approach conflicts with the need articulated in the ITIM framework to manage investments in a strategic, enterprisewide manner so that the investments address not only the objectives of individual programs, or lines of business, but also the impact that projects have on one another and the IT portfolio’s overall benefit to the organization. Lacking complete enterprisewide portfolio criteria, SSA risks optimizing individual business processes while producing stovepiped systems, as well as not maximizing overall benefits to the agency.

Table 11 shows the status for each key practice required to implement the critical process for defining the portfolio criteria and summarizes the evidence that supports these ratings.

At ITIM Stage 3, organizations create a portfolio of IT investments to ensure that (1) they are analyzed according to the organization’s portfolio selection criteria and (2) an optimal investment portfolio with manageable risks and returns is selected and funded. According to ITIM, creating the portfolio requires organizations to, among other things, document policies and procedures for analyzing, selecting, and maintaining the portfolio; provide adequate resources, including people, funding, and tools for creating the portfolio; and capture the information used to select, control, and evaluate the portfolio and maintain it for future reference. In creating the portfolio, the investment board should also (1) examine the mix of new and ongoing investments and their respective data and analyses and select investments for funding and (2) approve or modify the performance expectations for the IT investments they have selected. (The complete list of key practices is provided in table 12.)

SSA is executing the seven key practices associated with creating the portfolio. For example, according to SSA officials, the agency has adequate resources for selecting the portfolio, including the ITAB executives, other supporting staff, and a system that tracks proposal information. The ITAB also considers a list of proposed IT investments and assigns IT staffing resources to the investment portfolios.

Table 12 shows the status for each key practice required to implement the critical process for creating the portfolio and summarizes the evidence that supports these ratings.

This critical process builds upon the Stage 2 critical process related to providing investment oversight by adding the elements of portfolio performance to an organization’s investment control capacity. Compared to less mature organizations, Stage 3 organizations will have the foundation they need to control the risks faced by each investment and to deliver benefits that are linked to mission performance. In addition, a Stage 3 organization will have the benefit of good performance data generated by Stage 2 processes. Expanding this focus to the entire portfolio provides the organization with longer-term assurances that the IT investment portfolio will deliver mission value at acceptable cost.

SSA has executed two of the seven key practices associated with this process: ensuring adequate resources, including staff and tools for reviewing the investment portfolio, and ensuring that the ITAB is familiar with the process for evaluating and improving investments. The remaining five key practices were not executed, partly because SSA has delegated portfolio management and partly because it is not executing the Stage 2 prerequisite critical process, providing investment oversight, which collects information on projects. As we have discussed, the ITAB does not receive information on nonperforming projects, because performance monitoring has been delegated to the Deputy Commissioner, Systems. SSA officials agreed that they were not evaluating the portfolio as a whole. Until SSA executes all the key practices associated with this critical process, senior executives will not have the information they need to determine whether the investments they have selected are delivering mission value at the expected cost and risk.

Table 13 shows the status for each key practice required to implement the critical process for evaluating the portfolio and summarizes the evidence that supports these ratings.

The purpose of a postimplementation review is to evaluate an investment after it has completed development in order to validate whether the estimated return on investment was actually achieved. Specifically, the review is conducted to (1) examine differences between estimated and actual investment costs and benefits and possible ramifications for unplanned funding needs in the future and (2) extract “lessons learned” about the investment selection and control processes that can be used as the basis for management improvements. Postimplementation reviews should also be conducted for investment projects that were terminated before completion to readily identify potential management and process improvements.

SSA has executed four of the six key practices associated with this process: policies and procedures are defined, adequate resources are provided, individuals assigned to conduct postimplementation reviews are familiar with the processes, and projects for which reviews will be conducted are identified.

The remaining two key practices were not executed: quantitative investment data are not collected and analyzed and lessons learned are not conducted on investment processes for selection, control, and evaluation. Without analyzing quantitative data on benefits achieved, SSA cannot determine whether the project has delivered anticipated benefits. Further, without knowledge of what benefits are actually achieved from projects, the portfolio cannot be evaluated, and Stage 4 and 5 practices cannot be carried out effectively. Also, without developing lessons learned from postimplementation reviews to improve the CPIC’s select, control, and evaluate phases, the agency will be unable to use the reviews to improve its investment management processes.

Table 14 shows the status for each key practice required to implement the critical process for conducting postimplementation reviews and summarizes the evidence that supports these ratings.

More Than Half of SSA’s IT Budget Is Not Subject to Its Current Investment Management Process

Even though SSA is executing most Stage 2 and Stage 3 key practices for the work year budget managed by its investment board, IT products and services acquired with the acquisition budget ($610 million in acquisitions in fiscal year 2008—58 percent of the IT budget) are not managed as investments under SSA’s CPIC process, and are not reviewed by the ITAB. These products and services include, among other things, engineering support services, network infrastructure, mainframe capacity infrastructure, hardware maintenance, software maintenance, local telecom services, telephone systems maintenance, and an agencywide support service contract.

These acquisition budget expenditures are under the overall direction of the Deputy Commissioner, Systems and are determined by funding requests from the business units and subsequent negotiations. Each deputy commissioner and the associate commissioners who report to the Deputy Commissioner, Systems, submit requests for funds based on the unit’s acquisition needs. These requests are analyzed by the Deputy Commissioner, Systems staff, requests are reconciled with the available resources, a budget is developed, and the CIO reviews and signs it.

Although this process involves a large budget and important assets, it is not subject to the CPIC select, control, and evaluate phases. For example, acquisitions of IT products and services are not selected by a board in a disciplined fashion, such as using the agency’s CPIC select and control procedures, but instead are largely selected by one individual—–the Deputy Commissioner, Systems. While the ITAB is provided a list of proposed projects for the Agency IT Plan, the list does not include the acquisition budget expenses associated with projects. However, the investment board does receive a report summarizing the total amount of the funds expended.

Agency officials gave several reasons why the acquisition budget is not managed by the investment board. Specifically, in SSA’s view, just as the other deputy commissioners have discretion to manage funding allocated to their portfolios, the Deputy Commissioner, Systems should have the same discretion to allocate funding in the infrastructure portfolio. Further, the officials stated that many items included in this budget are very technical and might not be well understood by senior business management; thus, review at this level is not thought to be effective. In addition, officials said that many items in the acquisition budget (such as telephones) are not optional, but necessary to keep the agency running, and thus do not require a decision process.

Given the large amount of funds involved, senior management involvement and oversight are essential to ensure effective management of and full accountability for acquisitions of IT products and services. Further, until the agency manages all of its investments from an enterprisewide perspective, it will be unable to consider its investments comprehensively, and ensure that the investments optimally address the organization’s mission, strategic goals, and objectives.

SSA Is Beginning Initiatives Intended to Address High-Level ITIM Processes

Organizations that achieve the Stage 4 level of maturity evaluate their IT investment processes and portfolios to identify opportunities for improvement. At the same time, these organizations are able to maintain the mature control and selection processes that are characteristic of Stage 3 in the ITIM model. At Stage 4, organizations are capable of systematically planning for and implementing decisions to discontinue or deselect obsolete, high-cost, and low-value IT investments and planning for successor investments that better support strategic goals and business needs.

Organizations acquire Stage 5 capabilities when they create opportunities to shape strategic outcomes by learning from other organizations and continuously improving the manner in which they use IT to support and improve business outcomes. Thus, organizations at Stage 5 benchmark their IT investment processes relative to other best-in-class organizations and conduct proactive monitoring for breakthrough information technologies that will allow them to significantly improve business performance.

Table 15 shows the purpose of each critical process in Stages 4 and 5.

Because the ITIM is cumulative, agencies cannot fully implement Stage 4 and 5 processes without first executing Stage 2 and 3. Nonetheless, SSA officials said they have begun two initiatives related to a Stage 4 objective (improving the investment process) and a Stage 5 objective (leveraging IT for strategic outcomes). The first initiative, Application Portfolio Management, was established to improve the agency’s information technology decision-making process. When fully implemented, the initiative is intended to address the Stage 4 critical process (managing the succession of information systems). The Application Portfolio Management review is used to analyze and quantify the health of existing software applications to determine whether they are eligible to be retired, renovated, or maintained. According to the agency, SSA has released version 1.0 of Application Portfolio Management and has begun identifying software applications that are eligible to be retired, renovated, or maintained.

The second initiative, the Technology Infusion Process, is beginning to address the second Stage 5 critical process—using IT to drive strategic business change. The Technology Infusion Process was established to evaluate and implement new technologies or new uses of existing technologies that will facilitate SSA’s ability to achieve the agency’s strategic goals. SSA has begun to identify various technologies for research and has begun to review technology projects submitted by a component sponsor as candidates for the Technology Infusion Process. However, Application Portfolio Management has not identified hardware or infrastructure projects for retirement, renovation, or maintenance.

Conclusions

Given the importance of IT to SSA’s mission, it is vital that the agency manages its investments effectively. To its credit, SSA has established many of the basic practices needed to build the foundation for managing its projects as investments and for managing its investments as a portfolio. However, weaknesses remain. For example, although the agency has established an investment board as the decision-making body that defines and implements the investment governance process, key policies and procedures for the investment management process are not fully defined, and the investment board does not provide oversight of underperforming investments. Moreover, the agency does not track corrective actions for its underperforming projects. SSA has also taken the important step of creating an investment portfolio. However, it has not fully established the policies and procedures essential to managing the portfolio, such as for reviewing, evaluating, and improving the performance of the portfolio. Further, the agency’s postimplementation reviews do not evaluate whether the expected benefits were achieved or identified lessons learned for improving the investment management processes.

Moreover, the agency’s IT acquisition budget, used to acquire IT-related products and services, is not allocated or overseen by the investment board and is not managed using investment governance processes. Failure to apply these processes to the acquisition budget makes it impossible for SSA executive management tasked with overseeing the agency’s investments to ensure that this portion of the budget is spent in the most efficient and effective manner.

Recommendations for Executive Action

To strengthen SSA’s investment management capability and address weaknesses discussed in this report, we recommend that the Commissioner of Social Security take the following actions: To fully implement the key practices for building the investment foundation (Stage 2) for current and project-level future IT investments’ success, direct the Chief Information Officer to establish comprehensive policies and procedures for defining the investment governance process that specify (1) investment board operating procedures, (2) delegations of authority, and (3) criteria for prioritizing new and ongoing investments; strengthen and expand the board’s oversight responsibilities for underperforming projects and evaluations of projects; and establish a mechanism for tracking corrective actions for underperforming investments.

To fully implement the key practices for developing a complete investment portfolio (Stage 3), direct the Chief Information Officer to establish policies and procedures for defining the portfolio criteria; establish portfolio-level performance evaluation policies and procedures and criteria for assessing portfolio performance; and evaluate quantitative measures during postimplementation reviews, and lessons learned for improving select, control, and evaluate processes.

To ensure senior management involvement and full accountability for the agency’s investments, direct the Chief Information Officer to develop and implement policies and procedures to manage IT acquisitions as investments and manage them using the investment management framework.

Agency Comments and Our Evaluation

The Commissioner of Social Security provided written comments on a draft of this report (comments are reproduced in appendix II). In its comments SSA agreed with six of our recommendations and disagreed with one.

Regarding those recommendations with which it agreed, SSA stated that it had initiated actions to document existing investment management processes and that it plans to strengthen and expand the role of the investment board in the oversight of underperforming projects and in the evaluations of investments. The agency also stated that it plans to establish a mechanism for tracking corrective actions for underperforming investments. Further, to achieve a complete IT investment portfolio, SSA plans to establish procedures for defining the portfolio criteria within the context of the existing delegation of authority to the portfolio sponsors. In addition, regarding postimplementation reviews, the agency stated it plans to evaluate quantitative measures and lessons learned for improving select, control, and evaluate processes.

SSA disagreed with our recommendation that it develop policies and procedures for managing its IT acquisitions as investments and manage them using the investment board and investment management processes. The agency stated that its existing budget development process already treats these acquisitions as investments and maintains them by using an investment management framework, though not the one described in our ITIM framework. However, under SSA’s current process, these acquisitions are not subject to the agency’s investment management select, control, and evaluate processes and are not managed by its investment board. Given that the IT products and services make up the majority of SSA’s IT budget, the investment board’s involvement is essential to helping ensure effective management of and full accountability for acquisitions of IT products and services. As we previously noted, by the agency not applying its investment management process to the acquisition budget, it limits the ability of SSA’s executive management tasked with overseeing the agency’s investments to ensure that this portion of the budget is spent in the most efficient and effective manner.

SSA also provided technical and other comments, which we have incorporated as appropriate. Among the comments, the agency stated that it had pursued the adoption of industry best practices developed by institutions such as the Software Engineering Institute of Carnegie Mellon University and believed it had achieved comprehensive and mature IT management practices. SSA added that our assessment had provided an opportunity for the agency to think carefully about many aspects of its investment management processes, and had enabled it to better understand the strengths and weaknesses of its current approach to managing investments.

As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from the date of this letter. At that time, we will send copies of the report to interested congressional committees, the Director of the Office of Management and Budget, and the Commissioner of Social Security. Copies of this report will be made available to other interested parties on request. This report will also be available at no charge on our Web site at http://www.gao.gov.

Should you or your staff have questions on matters discussed in this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objective, Scope, and Methodology

Our objective was to determine whether Social Security Administration’s (SSA) investment management approach is consistent with leading investment management best practices. Our analysis was based on best practices contained in GAO’s Information Technology Investment Management (ITIM) framework and the framework’s associated evaluation methodology, and focused on the agency’s implementation of critical processes and key practices for managing its business systems investments.

To address our objective, we asked the agency to complete a self- assessment of its investment management process and provide the supporting documentation. We then reviewed the results of the agency’s self-assessment of Stages 2 and 3 practices and compared them against our ITIM framework. We focused on Stages 2 and 3 because these stages represent the processes needed to meet the standards of the Clinger- Cohen Act and they establish the foundation for effective acquisition management. We also validated and updated the results of the self- assessment through document reviews and interviews with officials, such as the CIO, Deputy Commissioner, Systems, and other staff in these offices. In doing so, we reviewed written policies, procedures, and guidance that provided evidence of documented practices, including SSA’s IT Capital Planning and Investment Control (CPIC) Guide and IT Planning Training Package. We also reviewed the fiscal year 2008-2009 Agency IT Plan and the board’s meeting minutes and other documentation providing evidence of executed practices.

We compared the evidence collected from our document reviews and interviews to the key practices in ITIM. We rated the key practices as “executed” on the basis of whether the agency demonstrated (by providing evidence of performance) that it had met the criteria of the key practice. A key practice was rated as “not executed” when we found insufficient evidence of a practice during the review or when we determined that there were significant weaknesses in SSA’s execution of the key practice. In addition, SSA was provided with the opportunity to produce evidence for key practices rated as “not executed.”

We did not assess investments made with SSA’s IT acquisition budget because SSA acknowledged that the acquisition budget is not managed using SSA’s investment management process. This budget includes items that are not projects, but are technology items that support projects, or general infrastructure such as mainframe computers, desktop computers, data storage, or telecommunications services.

As part of our analysis, we selected three IT projects as case studies to verify whether certain critical processes and key practices were being applied. SSA officials participated in the selection of these case studies. We selected projects that (1) supported different SSA functional areas, (2) were in different life-cycle phases, and (3) involved different funding amounts. These three projects are described below.

Ready Retirement is a project that automates the processing of retirement applications. It allows individuals to file for benefits using a Web interface. This investment is expected to increase online claims filing, minimize the number of recontacts required to complete an application, and provide progress indicators to inform applicants of where they are in the application process. Ready Retirement is intended to prepare the agency for the growing retirement workload expected as baby boomers become eligible for retirement by enabling applicants to prepare their own applications. According to the agency, this project is estimated to require about 27 staff years for fiscal year 2008, which corresponds to costs of about $3.1 million.

Appeals Council Case Processing is a software development project that automates the handling of case files in appeals of disability determinations. It is intended to provide the capability to process all disability cases electronically at all adjudicative levels. Further, the system can obtain claims, medical evidence, and supporting documentation over the Internet in a secured environment. The users have the capability to complete all disability case-related actions electronically. This project is expected to eliminate backlogs, reduce reliance on paper folders, and increase decisional and documentation accuracy and decisional consistency. SSA estimates that this project will require about 56 staff years in fiscal year 2008, which corresponds to costs of about $6.4 million.

Mainframe Architecture is a large infrastructure investment that involves both developmental and operations and maintenance components, and includes both software development and hardware.

SSA’s mainframes are the hardware platform for many critical systems. The agency states that its objective is to provide 100 percent reliability and availability to mainframe users. Tasks for the project include enhancements to hardware and software technology, annual upgrades to the operating system, routine additions to mainframe capacity dictated by workload growth, and migration to the current software versions of over 100 vendor products. The agency estimates that this project will require about 54 staff years for developmental projects and about 28 staff years for operations and maintenance work in fiscal year 2008, which corresponds to costs of about $9.5 million. In addition, the project is expected to require about $84 million from the acquisition budget for a total cost of about $94 million.

For these projects, we reviewed project management documentation, such as project proposals, project plans, and performance reports on costs and benefits. We also conducted interviews with the agency’s CIO and Deputy Commissioner, Systems, as well as other managers responsible for the agency’s investment management processes.

We conducted our work at SSA headquarters in Baltimore, Maryland from October 2007 through September 2008 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.

Appendix II: Comments from Social Security Administration

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact person named above, key contributors to this report were Cynthia Scott, Assistant Director; Faiza Baluch; Rebecca LaPaze; Sabine Paul; Tomás Ramirez; Glenn Spiegel; Niti Tandon; and Daniel Wexler.