Information Technology Management: Improvements Needed in Strategic Planning, Performance Measurement, and Investment Management Governmentwide
Released: 2004-03-03T00:00:00

Letter

Background

Advances in the use of IT and the Internet are continuing to change the way that federal agencies communicate, use, and disseminate information; deliver services; and conduct business. For example, electronic government (e-government) has the potential to help build better relationships between government and the public by facilitating timely and efficient interaction with citizens. To help agencies more effectively manage IT, the Congress has established a statutory framework of requirements and roles and responsibilities relating to information and technology management. In particular, the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996 require agency heads, acting through agency CIOs to, among other things, better link their IT planning and investment decisions to program missions develop and maintain a strategic information resources management (IRM) plan that describes how IRM activities help to accomplish agency missions; develop and maintain an ongoing process to establish goals for improving IRM’s contribution to program productivity, efficiency, and effectiveness; methods for measuring progress toward these goals; and clear roles and responsibilities for achieving these goals; develop and implement a sound IT architecture; implement and enforce IT management policies, procedures, standards, and guidelines; establish policies and procedures for ensuring that IT systems provide reliable, consistent, and timely financial or program performance data; and implement and enforce applicable policies, procedures, standards, and guidelines on privacy, security, disclosure, and information sharing.

Nevertheless, the agencies face significant challenges in effectively planning for and managing their IT. Such challenges can be overcome through the use of a systematic and robust management approach that addresses critical elements such as IT strategic planning and investment management.

Agencies Did Not Always Have Strategic Planning/Performance Measurement and Investment Management Practices in Place

Federal agencies did not always have in place important practices associated with IT laws, policies, and guidance related to strategic planning/performance measurement and investment management (see fig. 1). A well-defined strategic planning process helps to ensure that an agency’s IT goals are aligned with its strategic goals. Moreover, establishing performance measures and monitoring actual-versus- expected performance using those measures can help to determine whether IT is making a difference in improving performance. Finally, an IT investment management process is an integrated approach to managing investments that provides for the continuous identification, selection, control, life-cycle management, and evaluation of IT investments.

Agency IT officials could not always identify why practices were not in place, but in those instances in which reasons were identified, a variety of explanations were provided; for example, that the CIO position had been vacant, that not including a requirement in the agency’s guidance was an oversight, or that the process was being revised. Nevertheless, these practices are based on law, executive orders, Office of Management and Budget (OMB) policies, and our guidance, and are also important ingredients in ensuring effective strategic planning, performance measurement, and investment management that, in turn, make it more likely that the billions of dollars in government IT investments will be wisely spent.

Agencies’ Use of IT Strategic Planning/Performance Measurement Practices Was Uneven

Critical aspects of the strategic planning/performance measurement area include documenting the agency’s IT strategic planning processes, developing IRM plans, establishing goals, and measuring performance to evaluate whether goals are being met. Although the agencies often had these practices, or elements of these practices, in place, additional work remains, as demonstrated by the following examples: Strategic planning process. Strategic planning defines what an organization seeks to accomplish and identifies the strategies it will use to achieve desired results. A defined strategic planning process allows an agency to clearly articulate its strategic direction and to establish linkages among planning elements such as goals, objectives, and strategies. About half of the agencies had fully documented their strategic planning processes. Such processes are an essential foundation for ensuring that IT resources are effectively managed.

Strategic IRM plans. The Paperwork Reduction Act requires that agencies indicate in strategic IRM plans how they are applying information resources to improve the productivity, efficiency, and effectiveness of government programs. An important element of a strategic plan is that it presents an integrated system of high-level decisions that are reached through a formal, visible process. The Paperwork Reduction Act also requires agencies to develop IRM plans in accordance with OMB’s guidance. However, OMB does not provide cohesive guidance on the specific contents of IRM plans. Accordingly, although agencies generally provided OMB with a variety of planning documents to meet its requirement that they submit an IRM plan, these plans were generally limited to IT strategic or e-government issues and did not address other elements of IRM, as defined by the Paperwork Reduction Act. In particular, these plans generally include individual IT projects and initiatives, security, and enterprise architecture elements but do not often address other information functions—such as information collection, records management, and privacy—or the coordinated management of all information functions.

OMB IT staff agreed that the agency has not set forth guidance on the contents of agency IRM plans in a single place, stating that its focus has been on looking at agencies’ cumulative results and not on planning documents. These staff also noted that agencies account for their IRM activities through multiple documents (e.g., Information Collection Budgets and Government Paperwork Elimination Act plans). Nevertheless, half the agencies indicated a need for OMB to provide additional guidance on the development and content of IRM plans. Accordingly, we recommended that OMB develop and disseminate to agencies guidance on developing IRM plans.

IT goals. The Paperwork Reduction Act and the Clinger-Cohen Act require agencies to establish goals that address how IT contributes to program productivity, efficiency, effectiveness, and service delivery to the public. We have previously reported that leading organizations define specific goals, objectives, and measures, use a diversity of measure types, and describe how IT outputs and outcomes impact operational customer and agency program delivery requirements. The agencies generally had the types of goals outlined in the Paperwork Reduction Act and the Clinger- Cohen Act. However, five agencies did not have one or more of the goals required by the Paperwork Reduction Act and the Clinger-Cohen Act. It is important that agencies specify clear goals and objectives to set the focus and direction for IT performance.

IT performance measures. The Paperwork Reduction Act, the Clinger- Cohen Act, and an executive order require agencies to establish a variety of IT performance measures—such as those related to how IT contributes to program productivity, efficiency, and effectiveness—and to monitor the actual-versus-expected performance using those measures. Although the agencies largely had one or more of the required performance measures in place, these measures were not always linked to the agencies’ enterprisewide IT goals. Moreover, few agencies monitored actual-versus- expected performance for all of their enterprisewide IT goals. Specifically, although some agencies tracked actual-versus-expected outcomes for the IT performance measures in their performance plans or accountability reports and/or for specific IT projects, they generally did not track the performance measures that were specified in their IRM plans. As we have previously reported, an effective IT performance management system offers a variety of benefits, including serving as an early warning indicator of problems and the effectiveness of corrective actions; providing input to resource allocation and planning; and providing periodic feedback to employees, customers, stakeholders, and the general public about the quality, quantity, cost, and timeliness of products and services. Moreover, without enterprisewide performance measures that are tracked against actual results, agencies lack critical information about whether their overall IT activities are achieving expected goals.

Benchmarking. The Clinger-Cohen Act requires agencies to quantitatively benchmark agency process performance against public- and private-sector organizations, where comparable processes and organizations exist. Benchmarking is used because there may be external organizations that have more innovative or more efficient processes than their own processes. Seven agencies in our review had mechanisms in place—such as policies and strategies—related to benchmarking their IT processes. In general, however, agencies’ benchmarking decisions were ad hoc. Few agencies had developed a mechanism to identify comparable external private- or public-sector organizations and processes and/or had policies related to benchmarking, although all but 10 of the agencies provided examples of benchmarking that they had performed. Our previous study of IT performance measurement at leading organizations found that they had spent considerable time and effort comparing their performance information with that of other organizations.

Agency IT officials could not identify why strategic planning/performance measurement practices were not in place in all cases, but in those instances in which reasons were identified, a variety of explanations were provided. For example, reasons cited by agency IT officials included that they lacked the support from agency leadership, that the agency had not been developing IRM plans until recently and recognized that the plan needed further refinement, that the process was being revised, and that requirements were evolving.

Without strong strategic management practices, it is less likely that IT is being used to maximize improvement in mission performance. Moreover, without enterprisewide performance measures that are being tracked against actual results, agencies lack critical information about whether their overall IT activities, at a governmentwide cost of billions of dollars annually, are achieving expected goals.

Agencies’ Use of IT Investment Management Practices Was Mixed

Critical aspects of IT investment management include developing well- supported proposals, establishing investment management boards, and selecting and controlling IT investments. The agencies’ use of practices associated with these aspects of investment management was wide- ranging, as follows: IT investment proposals. Various legislative requirements, an executive order, and OMB policies provide minimum standards that govern agencies’ consideration of IT investments. In addition, we have issued guidance to agencies for selecting, controlling, and evaluating IT investments. Such processes help ensure, for example, that investments are cost-beneficial and meet mission needs and that the most appropriate development or acquisition approach is chosen. The agencies in our review had mixed results when evaluated against these various criteria. For example, the agencies almost always required that proposed investments demonstrate that they support the agency’s business needs, are cost-beneficial, address security issues, and consider alternatives. However, they were not as likely to have fully in place the Clinger-Cohen Act requirement that agencies follow, to the maximum extent practicable, a modular, or incremental, approach when investing in IT projects. Incremental investment helps to mitigate the risks inherent in large IT acquisitions/developments by breaking apart a single large project into smaller, independently useful components with known and defined relationships and dependencies.

Investment management boards. Our investment management guide states that establishing one or more IT investment board(s) is a key component of the investment management process. Such executive-level boards, made up of business-unit executives, concentrate management’s attention on assessing and managing risks and regulating the trade-offs between continuing to fund existing operations and developing new performance capabilities. Almost all of the agencies in our review had one or more enterprise-level investment management board. However, the investment management boards for six agencies were not involved, or the agency did not document the boards’ involvement, in the control phase. Maintaining responsibility for oversight with the same body that selected the investment is crucial to fostering a culture of accountability by holding the investment board that initially selected an investment responsible for its ongoing success.

Selection of IT investments. During the selection phase of an IT investment management process, the organization (1) selects projects that will best support its mission needs and (2) identifies and analyzes each project’s risks and returns before committing significant funds. To achieve desired results, it is important that agencies have a selection process that, for example, uses selection criteria to choose the IT investments that best support the organization’s mission and that prioritizes proposals. Twenty- two agencies used selection criteria in choosing their IT investments. In addition, about half the agencies used scoring models to help choose their investments.

Control over IT investments. During the control phase of the IT investment management process, the organization ensures that, as projects develop and as funds are spent, the project is continuing to meet mission needs at the expected levels of cost and risk. If the project is not meeting expectations or if problems have arisen, steps are quickly taken to address the deficiencies. In general, the agencies were weaker in the practices pertaining to the control phase of the investment management process than to the selection phase and no agency had the practices associated with the control phase fully in place. In particular, the agencies did not always have important mechanisms in place for agencywide investment management boards to effectively control investments, including decision-making rules for project oversight, early warning mechanisms, and/or requirements that corrective actions for under- performing projects be agreed upon and tracked. Executive level oversight of project-level management activities provides an organization with increased assurance that each investment will achieve the desired cost, benefit, and schedule results.

Among the variety of reasons that agencies cited for not having IT investment management practices fully in place were that the CIO position had been vacant, that not including a requirement in the IT investment management guide was an oversight, and that the process was being revised. However, in some cases agencies could not identify why certain practices were not in place. It is important that agencies address their shortcomings, because only by effectively and efficiently managing their IT resources through a robust investment management process can they gain opportunities to make better allocation decisions among many investment alternatives and to further leverage their IT investments.

Improving Agencies’ IT Strategic Planning/Performance Measurement and Investment Management

To help agencies improve their IT strategic planning/performance measurement and investment management, we have made numerous recommendations to agencies and issued guidance. Specifically, in our January 2004 report we made recommendations to the 26 agencies in our review regarding practices that were not fully in place. These recommendations addressed issues such as IT strategic planning; establishing and linking enterprisewide goals and performance measures and tracking progress against these measures; and selecting, controlling, and evaluating investments. By implementing these recommendations, agencies can better ensure that they are using strategic planning, performance measurement, and investment management practices that are consistent with IT legislation, executive orders, OMB policies, and our guidance.

Another mechanism that agencies can use to improve their IT management is to apply the management frameworks and guides that we have issued, which are based on our research into IT management best practices and our evaluations of agency IT management performance. In this vein, today we are releasing the latest version of our ITIM framework. This framework identifies and organizes critical processes for selecting, controlling, and evaluating IT investments into a framework of increasingly mature stages (see fig. 2).

First issued as an exposure draft in May 2000, this new version of the ITIM includes lessons learned from our use of the framework in our agency reviews and from lessons conveyed to us by users of the framework. In addition, in order to validate the appropriateness of our changes and to gain the advantage of their experience, we had the new version reviewed by several outside experts who are familiar with the ITIM exposure draft and with investment management in a broad array of public and private organizations.

ITIM can be used to analyze an organization’s investment management processes and to determine its level of maturity. The framework is useful to many federal agencies because it provides: (1) a rigorous, standardized tool for internal and external evaluations of an agency’s IT investment management process; (2) a consistent and understandable mechanism for reporting the results of these assessments to agency executives, Congress, and other interested parties; and (3) a road map that agencies can use for improving their investment management processes. Regarding the first two points, we and selected agency Inspectors General have used the ITIM to evaluate and report on the investment management processes of several agencies. Concerning the third point, a number of agencies have recognized the usefulness of the ITIM framework and have used it to develop and enhance their investment management strategies. For example, one agency uses the framework to periodically review its IT investment management capabilities and has developed an action plan to move through the stages of maturity.

In summary, our January 2004 report indicates that the federal government can significantly improve its IT strategic planning, performance measurement, and investment management. Such improvement would better ensure that agencies are being responsible stewards of the billions of dollars for IT with which they have been entrusted, by helping them to invest these monies wisely. This can be accomplished, in part, through the expeditious implementation of our recommendations and the adoption of best practices, which we have incorporated into our IT management frameworks and guides such as the ITIM.

Mr. Chairman, this completes my prepared statement. I would be happy to respond to any questions that you or other Members of the Subcommittee may have at this time.

Contacts

If you have any questions regarding this statement, please contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Specific questions related to our January 2004 report may also be directed to Linda Lambert at (202) 512-9556 or via e-mail at lambertl@gao.gov or Mark Shaw at (202) 512-6251 or via e-mail at shawm@gao.gov. Questions related to the ITIM framework can be directed to Lester Diamond at (202) 512-7957 or via e- mail at diamondl@gao.gov.

Appendix I: Information Technology (IT) Strategic Planning/Performance Measurement and Investment Management Practices

Table 1 describes the 12 IT strategic planning/performance measurement and the 18 IT investment management practices that we used in our January 2004 report on the government’s performance in these areas. We identified these 30 practices after reviewing major legislative requirements (e.g., the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996), executive orders, Office of Management and Budget policies, and our own guidance.