Information Technology Management: Improvements Needed in Strategic Planning, Performance Measurement, and Investment Management Governmentwide
Published: 2004-03-03T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The federal government spends billions of dollars annually on information technology (IT) investments that are critical to the effective implementation of major government programs. To help agencies effectively manage their substantial IT investments, the Congress has established a statutory framework of requirements and roles and responsibilities relating to information and technology management, that addresses, for example, (1) IT strategic planning/performance measurement (which defines what an organization seeks to accomplish, identifies the strategies it will use to achieve desired results, and then determines how well it is succeeding in reaching resultsoriented goals and achieving objectives) and (2) IT investment management (which involves selecting, controlling, and evaluating investments). GAO was asked to summarize its January 2004 report on IT strategic planning/performance measurement and investment management (Information Technology Management: Governmentwide Strategic Planning, Performance Measurement, and Investment Management Can Be Further Improved, GAO-04-49 , January 12, 2004) and to discuss how agencies can improve their performance in these areas.
section_title: What GAO Found
paragraphs: GAO recently reported that the use of important IT strategic planning/performance measurement and investment management practices by 26 major federal agencies was mixed. For example, agencies generally had IT strategic plans and goals, but these goals were not always linked to specific performance measures that were tracked. Agencies also largely had IT investment management boards, but no agency had the practices associated with the oversight of IT investments fully in place. Although they could not always provide an explanation, agencies cited a variety of reasons for not having practices fully in place, including that the chief information officer position had been vacant and that the process was being revised. By improving their IT strategic planning, performance measurement, and investment management, agencies can better ensure that they are being responsible stewards of the billions of dollars for IT that they have been entrusted with through the wise investment of these monies. To help agencies improve in these areas, GAO has made numerous recommendations to agencies and issued guidance. For example, in the January 2004 report, GAO made recommendations to the 26 agencies regarding practices that were not fully in place. In addition, today GAO is releasing the latest version of its Information Technology Investment Management (ITIM) framework, which identifies critical processes for selecting, controlling, and evaluating IT investments and organizes them into a framework of increasingly mature stages; thereby providing agencies a road map for improving IT investment management processes in a systematic and organized manner.

section_title: Letter
paragraphs: Thank you for the opportunity to join in today’s hearing on the  government’s information technology (IT) management. This is a critical  topic because, according to the President’s most recent budget, the federal  government spends billions of dollars annually on IT—reportedly investing  about $57 billion in fiscal year 2003. Yet these dollars are not always  managed wisely. For example, the Administration reported that of the $60  billion in IT investments requested for fiscal year 2005, $22 billion— representing 621 major projects—are currently on its “Management Watch  List.” This list includes mission-critical projects that need improvement in  the areas of performance measures, earned value management, and/or IT  security.
To help agencies effectively manage their substantial IT investments, the  Congress has established a statutory framework of requirements and roles  and responsibilities relating to information and technology management  through laws such as the Paperwork Reduction Act of 1995 and the  Clinger-Cohen Act of 1996. This framework addresses, for example, IT  strategic planning/performance measurement (which defines what an  organization seeks to accomplish, identifies the strategies it will use to  achieve desired results, and then determines how well it is succeeding in  reaching results-oriented goals and achieving objectives), and investment   management (which involves selecting,  controlling, and evaluating  investments).
At your request, today I will summarize our recently issued report on the  extent to which 26 agencies had in place 30 important practices  associated with key legislative and other requirements for IT strategic  planning/performance measurement and IT investment management (app.  I lists the 30 practices). I will also discuss how agencies can improve their  performance in these areas.
subsections: section_title: Background
paragraphs: Advances in the use of IT and the Internet are continuing to change the  way that federal agencies communicate, use, and disseminate information;  deliver services; and conduct business. For example, electronic  government (e-government) has the potential to help build better  relationships between government and the public by facilitating timely and  efficient interaction with citizens. To help agencies more effectively  manage IT, the Congress has established a statutory framework of  requirements and roles and responsibilities relating to information and  technology management. In particular, the Paperwork Reduction Act of  1995 and the Clinger-Cohen Act of 1996 require agency heads, acting  through agency CIOs to, among other things,    better link their IT planning and investment decisions to program missions    develop and maintain a strategic information resources management  (IRM) plan that describes how IRM activities help to accomplish agency  missions;    develop and maintain an ongoing process to establish goals for improving  IRM’s contribution to program productivity, efficiency, and effectiveness;  methods for measuring progress toward these goals; and clear roles and  responsibilities for achieving these goals;   develop and implement a sound IT architecture;  implement and enforce IT management policies, procedures, standards,  and guidelines;   establish policies and procedures for ensuring that IT systems provide  reliable, consistent, and timely financial or program performance data; and implement and enforce applicable policies, procedures, standards, and  guidelines on privacy, security, disclosure, and information sharing.
Nevertheless, the agencies face significant challenges in effectively  planning for and managing their IT. Such challenges can be overcome  through the use of a systematic and robust management approach that  addresses critical elements such as IT strategic planning and investment  management.
section_title: Agencies Did Not Always Have Strategic Planning/Performance Measurement and Investment Management Practices in Place
paragraphs: Federal agencies did not always have in place important practices  associated with IT laws, policies, and guidance related to strategic  planning/performance measurement and investment management (see fig.  1). A well-defined strategic planning process helps to ensure that an  agency’s IT goals are aligned with its strategic goals. Moreover,  establishing performance measures and monitoring actual-versus- expected performance using those measures can help to determine  whether IT is making a difference in improving performance. Finally, an IT  investment management process is an integrated approach to managing  investments that provides for the continuous identification, selection,  control, life-cycle management, and evaluation of IT investments.
Agency IT officials could not always identify why practices were not in  place, but in those instances in which reasons were identified, a variety of  explanations were provided; for example, that the CIO position had been  vacant, that not including a requirement in the agency’s guidance was an  oversight, or that the process was being revised. Nevertheless, these  practices are based on law, executive orders, Office of Management and  Budget (OMB) policies, and our guidance, and are also important  ingredients in ensuring effective strategic planning, performance  measurement, and investment management that, in turn, make it more  likely that the billions of dollars in government IT investments will be  wisely spent.
subsections: section_title: Agencies’ Use of IT Strategic Planning/Performance Measurement Practices Was Uneven
paragraphs: Critical aspects of the strategic planning/performance measurement area  include documenting the agency’s IT strategic planning processes,  developing IRM plans, establishing goals, and measuring performance to  evaluate whether goals are being met. Although the agencies often had  these practices, or elements of these practices, in place, additional work  remains, as demonstrated by the following examples:    Strategic planning process. Strategic planning defines what an  organization seeks to accomplish and identifies the strategies it will use to  achieve desired results. A defined strategic planning process allows an  agency to clearly articulate its strategic direction and to establish linkages  among planning elements such as goals, objectives, and strategies. About  half of the agencies had fully documented their strategic planning  processes. Such processes are an essential foundation for ensuring that IT  resources are effectively managed.
Strategic IRM plans. The Paperwork Reduction Act requires that agencies  indicate in strategic IRM plans how they are applying information  resources to improve the productivity, efficiency, and effectiveness of  government programs. An important element of a strategic plan is that it  presents an integrated system of high-level decisions that are reached  through a formal, visible process. The Paperwork Reduction Act also  requires agencies to develop IRM plans in accordance with OMB’s  guidance. However, OMB does not provide cohesive guidance on the  specific contents of IRM plans. Accordingly, although agencies generally  provided OMB with a variety of planning documents to meet its  requirement that they submit an IRM plan, these plans were generally  limited to IT strategic or e-government issues and did not address other  elements of IRM, as defined by the Paperwork Reduction Act. In  particular, these plans generally include individual IT projects and  initiatives, security, and enterprise architecture elements but do not often  address other information functions—such as information collection,  records management, and privacy—or the coordinated management of all  information functions.
OMB IT staff agreed that the agency has not set forth guidance on the  contents of agency IRM plans in a single place, stating that its focus has  been on looking at agencies’ cumulative results and not on planning  documents. These staff also noted that agencies account for their IRM  activities through multiple documents (e.g., Information Collection  Budgets and Government Paperwork Elimination Act plans).  Nevertheless, half the agencies indicated a need for OMB to provide  additional guidance on the development and content of IRM plans.  Accordingly, we recommended that OMB develop and disseminate to  agencies guidance on developing IRM plans.
IT goals. The Paperwork Reduction Act and the Clinger-Cohen Act require  agencies to establish goals that address how IT contributes to program  productivity, efficiency, effectiveness, and service delivery to the public.  We have previously reported that leading organizations define specific  goals, objectives, and measures, use a diversity of measure types, and  describe how IT outputs and outcomes impact operational customer and  agency program delivery requirements. The agencies generally had the  types of goals outlined in the Paperwork Reduction Act and the Clinger- Cohen Act. However, five agencies did not have one or more of the goals  required by the Paperwork Reduction Act and the Clinger-Cohen Act. It is  important that agencies specify clear goals and objectives to set the focus  and direction for IT performance.
IT performance measures. The Paperwork Reduction Act, the Clinger- Cohen Act, and an executive order require agencies to establish a variety  of IT performance measures—such as those related to how IT contributes  to program productivity, efficiency, and effectiveness—and to monitor the  actual-versus-expected performance using those measures. Although the  agencies largely had one or more of the required performance measures in  place, these measures were not always linked to the agencies’  enterprisewide IT goals. Moreover, few agencies monitored actual-versus- expected performance for all of their enterprisewide IT goals. Specifically,  although some agencies tracked actual-versus-expected outcomes for the  IT performance measures in their performance plans or accountability  reports and/or for specific IT projects, they generally did not track the  performance measures that were specified in their IRM plans. As we have  previously reported, an effective IT performance management system  offers a variety of benefits, including serving as an early warning indicator  of problems and the effectiveness of corrective actions; providing input to  resource allocation and planning; and providing periodic feedback to  employees, customers, stakeholders, and the general public about the  quality, quantity, cost, and timeliness of products and services. Moreover,  without enterprisewide performance measures that are tracked against  actual results, agencies lack critical information about whether their  overall IT activities are achieving expected goals.
Benchmarking. The Clinger-Cohen Act requires agencies to quantitatively  benchmark agency process performance against public- and private-sector  organizations, where comparable processes and organizations exist.  Benchmarking is used because there may be external organizations that  have more innovative or more efficient processes than their own  processes. Seven agencies in our review had mechanisms in place—such  as policies and strategies—related to benchmarking their IT processes. In  general, however, agencies’ benchmarking decisions were ad hoc. Few  agencies had developed a mechanism to identify comparable external  private- or public-sector organizations and processes and/or had policies  related to benchmarking, although all but 10 of the agencies provided  examples of benchmarking that they had performed. Our previous study of  IT performance measurement at leading organizations found that they had  spent considerable time and effort comparing their performance  information with that of other organizations.
Agency IT officials could not identify why strategic planning/performance  measurement practices were not in place in all cases, but in those  instances in which reasons were identified, a variety of explanations were  provided. For example, reasons cited by agency IT officials included that  they lacked the support from agency leadership, that the agency had not  been developing IRM plans until recently and recognized that the plan  needed further refinement, that the process was being revised, and that  requirements were evolving.
Without strong strategic management practices, it is less likely that IT is  being used to maximize improvement in mission performance. Moreover,  without enterprisewide performance measures that are being tracked  against actual results, agencies lack critical information about whether  their overall IT activities, at a governmentwide cost of billions of dollars  annually, are achieving expected goals.
section_title: Agencies’ Use of IT Investment Management Practices Was Mixed
paragraphs: Critical aspects of IT investment management include developing well- supported proposals, establishing investment management boards, and  selecting and controlling IT investments. The agencies’ use of practices  associated with these aspects of investment management was wide- ranging, as follows:  IT investment proposals. Various legislative requirements, an executive  order, and OMB policies provide minimum standards that govern agencies’  consideration of IT investments. In addition, we have issued guidance to  agencies for selecting, controlling, and evaluating IT investments. Such  processes help ensure, for example, that investments are cost-beneficial  and meet mission needs and that the most appropriate development or  acquisition approach is chosen. The agencies in our review had mixed  results when evaluated against these various criteria. For example, the  agencies almost always required that proposed investments demonstrate  that they support the agency’s business needs, are cost-beneficial, address  security issues, and consider alternatives. However, they were not as likely  to have fully in place the Clinger-Cohen Act requirement that agencies  follow, to the maximum extent practicable, a modular, or incremental,  approach when investing in IT projects. Incremental investment helps to  mitigate the risks inherent in large IT acquisitions/developments by  breaking apart a single large project into smaller, independently useful  components with known and defined relationships and dependencies.
Investment management boards. Our investment management guide  states that establishing one or more IT investment board(s) is a key  component of the investment management process. Such executive-level  boards, made up of business-unit executives, concentrate management’s  attention on assessing and managing risks and regulating the trade-offs  between continuing to fund existing operations and developing new  performance capabilities. Almost all of the agencies in our review had one  or more enterprise-level investment management board. However, the  investment management boards for six agencies were not involved, or the  agency did not document the boards’ involvement, in the control phase.  Maintaining responsibility for oversight with the same body that selected  the investment is crucial to fostering a culture of accountability by holding  the investment board that initially selected an investment responsible for  its ongoing success.
Selection of IT investments. During the selection phase of an IT  investment management process, the organization (1) selects projects that  will best support its mission needs and (2) identifies and analyzes each  project’s risks and returns before committing significant funds. To achieve  desired results, it is important that agencies have a selection process that,  for example, uses selection criteria to choose the IT investments that best  support the organization’s mission and that prioritizes proposals. Twenty- two agencies used selection criteria in choosing their IT investments. In  addition, about half the agencies used scoring models to help choose  their investments.
Control over IT investments. During the control phase of the IT  investment management process, the organization ensures that, as  projects develop and as funds are spent, the project is continuing to meet  mission needs at the expected levels of cost and risk. If the project is not  meeting expectations or if problems have arisen, steps are quickly taken to  address the deficiencies. In general, the agencies were weaker in the  practices pertaining to the control phase of the investment management  process than to the selection phase and no agency had the practices  associated with the control phase fully in place. In particular, the agencies  did not always have important mechanisms in place for agencywide  investment management boards to effectively control investments,  including decision-making rules for project oversight, early warning  mechanisms, and/or requirements that corrective actions for under- performing projects be agreed upon and tracked. Executive level oversight  of project-level management activities provides an organization with  increased assurance that each investment will achieve the desired cost,  benefit, and schedule results.
Among the variety of reasons that agencies cited for not having IT  investment management practices fully in place were that the CIO position  had been vacant, that not including a requirement in the IT investment  management guide was an oversight, and that the process was being  revised. However, in some cases agencies could not identify why certain  practices were not in place. It is important that agencies address their  shortcomings, because only by effectively and efficiently managing their IT  resources through a robust investment management process can they gain  opportunities to make better allocation decisions among many investment  alternatives and to further leverage their IT investments.
section_title: Improving Agencies’ IT Strategic Planning/Performance Measurement and Investment Management
paragraphs: To help agencies improve their IT strategic planning/performance  measurement and investment management, we have made numerous  recommendations to agencies and issued guidance. Specifically, in our  January 2004 report we made recommendations to the 26 agencies in our  review regarding practices that were not fully in place. These  recommendations addressed issues such as IT strategic planning;  establishing and linking enterprisewide goals and performance measures  and tracking progress against these measures; and selecting, controlling,  and evaluating investments. By implementing these recommendations,  agencies can better ensure that they are using strategic planning,  performance measurement, and investment management practices that are  consistent with IT legislation, executive orders, OMB policies, and our  guidance.
Another mechanism that agencies can use to improve their IT management  is to apply the management frameworks and guides that we have issued,  which are based on our research into IT management best practices and  our evaluations of agency IT management performance. In this vein,  today we are releasing the latest version of our ITIM framework. This  framework identifies and organizes critical processes for selecting,  controlling, and evaluating IT investments into a framework of  increasingly mature stages (see fig. 2).
First issued as an exposure draft in May 2000, this new version of the ITIM  includes lessons learned from our use of the framework in our agency  reviews and from lessons conveyed to us by users of the framework. In  addition, in order to validate the appropriateness of our changes and to  gain the advantage of their experience, we had the new version reviewed  by several outside experts who are familiar with the ITIM exposure draft  and with investment management in a broad array of public and private  organizations.
ITIM can be used to analyze an organization’s investment management  processes and to determine its level of maturity. The framework is useful  to many federal agencies because it provides: (1) a rigorous, standardized  tool for internal and external evaluations of an agency’s IT investment  management process; (2) a consistent and understandable mechanism for  reporting the results of these assessments to agency executives, Congress,  and other interested parties; and (3) a road map that agencies can use for  improving their investment management processes. Regarding the first  two points, we and selected agency Inspectors General have used the ITIM  to evaluate and report on the investment management processes of several  agencies. Concerning the third point, a number of agencies have  recognized the usefulness of the ITIM framework and have used it to  develop and enhance their investment management strategies. For  example, one agency uses the framework to periodically review its IT  investment management capabilities and has developed an action plan to  move through the stages of maturity.
In summary, our January 2004 report indicates that the federal government  can significantly improve its IT strategic planning, performance  measurement, and investment management. Such improvement would  better ensure that agencies are being responsible stewards of the billions  of dollars for IT with which they have been entrusted, by helping them to  invest these monies wisely. This can be accomplished, in part, through the  expeditious implementation of our recommendations and the adoption of  best practices, which we have incorporated into our IT management  frameworks and guides such as the ITIM.
Mr. Chairman, this completes my prepared statement. I would be happy to  respond to any questions that you or other Members of the Subcommittee  may have at this time.
section_title: Contacts
paragraphs: If you have any questions regarding this statement, please contact me at  (202) 512-9286 or by e-mail at pownerd@gao.gov. Specific questions  related to our January 2004 report may also be directed to Linda Lambert  at (202) 512-9556 or via e-mail at lambertl@gao.gov or Mark Shaw at (202)  512-6251 or via e-mail at shawm@gao.gov. Questions related to the ITIM  framework can be directed to Lester Diamond at (202) 512-7957 or via e- mail at diamondl@gao.gov.
section_title: Appendix I: Information Technology (IT) Strategic Planning/Performance Measurement and Investment Management Practices
paragraphs: Table 1 describes the 12 IT strategic planning/performance measurement  and the 18 IT investment management practices that we used in our  January 2004 report on the government’s performance in these areas. We  identified these 30 practices after reviewing major legislative requirements  (e.g., the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of  1996), executive orders, Office of Management and Budget policies, and  our own guidance.