Information Technology Management: Governmentwide Strategic Planning, Performance Measurement, and Investment Management Can Be Further Improved
Published: 2004-01-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Over the years, the Congress has promulgated laws and the Office of Management and Budget and GAO have issued policies and guidance, respectively, on (1) information technology (IT) strategic planning/performance measurement (which defines what an organization seeks to accomplish, identifies the strategies it will use to achieve desired results, and then determines how well it is succeeding in reaching resultsoriented goals and achieving objectives) and (2) investment management (which involves selecting, controlling, and evaluating investments). To obtain an understanding of the government's implementation of these key IT management policies, congressional requesters asked GAO to determine the extent to which 26 major agencies have in place practices associated with key legislative and other requirements for (1) IT strategic planning/ performance measurement and (2) IT investment management.
section_title: What GAO Found
paragraphs: Agencies' use of 12 IT strategic planning/performance measurement practices--identified based on legislation, policy, and guidance--is uneven. For example, agencies generally have IT strategic plans and goals, but these goals are not always linked to specific performance measures that are tracked. Without enterprisewide performance measures that are tracked against actual results, agencies lack critical information about whether their overall IT activities are achieving expected goals. Agencies' use of 18 IT investment management practices that GAO identified is also mixed. For example, the agencies largely have IT investment management boards, but no agency had the practices associated with the control phase fully in place. Executive-level oversight of project-level management activities provides organizations with increased assurance that each investment will achieve the desired cost, benefit, and schedule results. Agencies cited a variety of reasons for not having practices fully in place, such as that the chief information officer position had been vacant, that not including a requirement in guidance was an oversight, and that the process was being revised, although they could not always provide an explanation. Regardless of the reason, these practices are important ingredients for ensuring effective strategic planning, performance measurement, and investment management, which, in turn, make it more likely that the billions of dollars in government IT investments are wisely spent.

section_title: Letter
paragraphs: According to the President’s most recent budget, the federal government  spends billions of dollars annually on information technology (IT)— reportedly investing about $50 billion in fiscal year 2002 and expecting to  invest about $60 billion in fiscal year 2004. Despite this substantial  investment, the government’s management of information resources has  produced mixed results. Although agencies have taken constructive steps  to implement modern strategies, systems, and management policies and  practices, our most recent high-risk and performance and accountability  series identified continuing high-risk system modernization efforts and  governmentwide information and technology management challenges.
For years, the Congress has been working to increase the effectiveness of  information and technology management in the federal government by  passing legislation and providing oversight. For example, the Paperwork  Reduction Act of 1995 applied life-cycle management principles to  information and technology management and required that agencies  indicate in strategic information resources management (IRM) plans how  they are applying information resources to improve the productivity,  efficiency, and effectiveness of government programs. The Clinger-Cohen  Act of 1996 amended the Paperwork Reduction Act, establishing agency  chief information officers (CIO) who report directly to the agency head and  are responsible for information resources management activities. Among  other things, the Clinger-Cohen Act also (1) required senior executive  involvement in IT decision making and (2) imposed much-needed  discipline in acquiring and managing technology resources.
To obtain a broad view of the government’s implementation of key IT  management, you requested that we determine the extent to which  agencies have in place practices associated with key legislative and other  requirements for (1) IT strategic planning/performance measurement and  (2) IT investment management. To address these objectives, we identified  and reviewed major legislative requirements and executive orders  pertaining to IT strategic planning/performance measurement, which  defines what an organization seeks to accomplish, identifies the strategies  it will use to achieve desired results, and then determines—through  measurement—how well it is succeeding in reaching results-oriented goals  and achieving objectives; and IT investment management, which involves  selecting, controlling, and evaluating investments. Specifically, we  identified 30 important IT management practices in these areas using  legislative requirements, such as the Paperwork Reduction Act and the  Clinger-Cohen Act, and policy and guidance issued by the Office of  Management and Budget (OMB) and GAO. We selected 26 organizations  for our review (23 major departments and agencies identified in 31 U.S.C.  901 and the 3 military services).
subsections: section_title: Background
paragraphs: Advances in the use of IT and the Internet are continuing to change the way  that federal agencies communicate, use, and disseminate information;  deliver services; and conduct business. For example, electronic  government (e-government) has the potential to help build better  relationships between government and the public by facilitating timely and  efficient interaction with citizens. To help the agencies more effectively  manage IT, the Congress has established a statutory framework of  requirements and roles and responsibilities relating to information and  technology management. Nevertheless, the agencies face significant  challenges in effectively planning for and managing their IT. Such  challenges can be overcome through the use of a systematic and robust  management approach that addresses critical elements, such as IT  strategic planning and investment management.
subsections: section_title: Federal Government’s Statutory Framework for Information and Technology Management
paragraphs: The Congress established a statutory framework to help address the  information and technology management challenges that agencies face.  Under this framework, agencies are accountable for effectively and  efficiently developing, acquiring, and using IT in their organizations. In  particular, the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act  of 1996 require agency heads, acting through agency CIOs, to, among other  things,  better link their IT planning and investment decisions to program   develop and maintain a strategic IRM plan that describes how IRM  activities help accomplish agency missions;  develop and maintain an ongoing process to establish goals for  improving IRM’s contribution to program productivity, efficiency, and  effectiveness; methods for measuring progress toward these goals; and  clear roles and responsibilities for achieving these goals;  develop and implement a sound IT architecture;  implement and enforce IT management policies, procedures, standards,   establish policies and procedures for ensuring that IT systems provide  reliable, consistent, and timely financial or program performance data;  and  implement and enforce applicable policies, procedures, standards, and  guidelines on privacy, security, disclosure, and information sharing.
Moreover, under the government’s current legislative framework, OMB has  important responsibilities for providing direction on governmentwide  information and technology management and overseeing agency activities  in these areas. Among OMB’s responsibilities are  ensuring agency integration of IRM plans, program plans, and budgets  for the acquisition and use of IT and the efficiency and effectiveness of  interagency IT initiatives;  developing and maintaining a governmentwide strategic IRM plan;  developing, as part of the budget process, a mechanism for analyzing,  tracking, and evaluating the risks and results of all major capital  investments made by an executive agency for information systems; directing and overseeing the implementation of policy, principles,  standards, and guidelines for the dissemination of and access to public  information;  encouraging agency heads to develop and use best practices in IT   developing and overseeing the implementation of privacy and security  policies, principles, standards, and guidelines.
Further, in 2002, the Congress passed, and the President signed, legislation  intended to improve the collection, use, and dissemination of government  information and to strengthen information security. Specifically, Public  Law 107-347, the E-Government Act of 2002, which was enacted in  December 2002, includes provisions to promote the use of the Internet and  other information technologies to provide government services  electronically. The E-Government Act also contains the Federal  Information Security Management Act (FISMA) of 2002, which replaced  and strengthened the Government Information Security Reform legislative  provisions (commonly referred to as “GISRA”). Among other provisions,  FISMA requires each agency, including national security agencies, to (1)  establish an agencywide risk-based information security program to be  overseen by the agency CIO and ensure that information security is  practiced throughout the life cycle of each agency system; and (2) develop,  maintain, and annually update an inventory of major information systems  (including major national security systems) operated by the agency or  under its control.
section_title: Federal IT Challenges
paragraphs: Even with the framework laid out by the Congress, the federal government  faces enduring IT challenges. Specifically, in January 2003, we reported on  a variety of challenges facing federal agencies in continuing to take  advantage of the opportunities presented by IT. Unless and until the  challenges outlined below are overcome, federal agencies are unlikely to  optimize their use of IT, which can affect an organization’s ability to  effectively and efficiently implement its programs and missions.
Pursuing opportunities for e-government. E-government offers many  opportunities to better serve the public, make government more  efficient and effective, and reduce costs. Federal agencies have  implemented a wide array of e-government applications, including using  the Internet to collect and disseminate information and forms; buy and  pay for goods and services; submit bids and proposals; and apply for  licenses, grants, and benefits. Although substantial progress has been  made, the government has not yet fully reached its potential in this area.  Recognizing this, a key element of the President’s Management Agenda  is the expansion of e-government to enhance access to information and  services, particularly through the Internet. In response, OMB  established a task force that selected a strategic set of initiatives to lead  this expansion. Our review of the initial planning projects associated  with these initiatives found that important aspects—such as  collaboration and customer focus—had not been thought out for all of  the projects and that major uncertainties in funding and milestones  were not uncommon. Accordingly, we recommended that OMB take  steps as overseer of the e-government initiatives to reduce the risk that  the projects would not meet their objectives.
Improving the collection, use, and dissemination of government  information. The rapid evolution of IT is creating challenges in  managing and preserving electronic records. Complex electronic  records are increasingly being created in a decentralized environment  and in volumes that make it difficult to organize them and make them  accessible. Further, storage media themselves are affected by the dual  problems of obsolescence and deterioration. These problems are  compounded as computer hardware and application software become  obsolete, since they may leave behind electronic records that can no  longer be read. Overall responsibility for the government’s electronic  records lies with the National Archives and Records Administration  (NARA). Our past work has shown that while NARA has taken some  action to respond to the challenges associated with managing and  preserving electronic records, most electronic records remain  unscheduled; that is, their value had not been assessed and their  disposition had not been determined. In addition, records of historical  value were not being identified and provided to NARA; as a result, they  were at risk of being lost. We recommended that NARA develop  strategies for raising agency management’s awareness of the importance  of records management and for performing systematic inspections. In  July 2003 we testified that although NARA has made progress in  addressing these issues, more work remains to be done.
The growth of electronic information—as well as the security threats  facing our nation—are also highlighting privacy issues. For example,  online privacy has emerged as one of the key—and most contentious— issues surrounding the continued evolution of the Internet. In addition,  our survey of 25 departments and agencies about their implementation  of the Privacy Act—which regulates how federal agencies may use the  personal information that individuals supply when obtaining  government services or fulfilling obligations—found that a key  characteristic of the agencies’ 2,400 systems of records is that an  estimated 70 percent contained electronic records. Our survey also  found that although compliance with Privacy Act provisions and  related OMB guidance was generally high in many areas, according to  agency reports, it was uneven across the federal government. To  improve agency compliance and address issues reported by the  agencies, we made recommendations to OMB, such as to direct  agencies to correct compliance deficiencies, to monitor agency  compliance, and to reassess its guidance.
Strengthening information security. Since September 1996, we have  reported that poor information security is a high-risk area across the  federal government with potentially devastating consequences.  Although agencies have taken steps to redesign and strengthen their  information system security programs, our analyses of information  security at major federal agencies have shown that federal systems were  not being adequately protected from computer-based threats. Our latest  analyses of audit reports published from October 2001 through October  2002 continue to show significant weaknesses in federal computer  systems that put critical operations and assets at risk. In addition, in  June 2003 we testified that agencies’ fiscal year 2002 reports and  evaluations required by GISRA found that many agencies have not  implemented security requirements for most of their systems, such as  performing risk assessments and testing controls. In addition, the  usefulness of agency corrective action plans may be limited when they  do not identify all weaknesses or contain realistic completion dates.
One of the most serious problems currently facing the government is  cyber critical infrastructure protection, which is protecting the information systems that support the nation’s critical infrastructures,  such as national defense and power distribution. Since the September  11 attacks, warnings of the potential for terrorist cyber attacks against  our critical infrastructures have increased. In addition, as greater  amounts of money are transferred through computer systems, as more  sensitive economic and commercial information is exchanged  electronically, and as the nation’s defense and intelligence communities  increasingly rely on commercially available information technology, the  likelihood increases that information attacks will threaten vital national  interests. Among the critical infrastructure protection challenges the  government faces are (1) developing a national critical infrastructure  protection strategy, (2) improving analysis and warning capabilities,  and (3) improving information sharing on threats and vulnerabilities.  For each of the challenges, improvements have been made and  continuing efforts are in progress, but much more is needed to address  them. In particular, we have identified and made numerous  recommendations over the last several years concerning critical  infrastructure challenges that still need to be addressed. As a result of  our concerns in this area, we have expanded our information security  high-risk area to include cyber critical infrastructure protection.
Constructing and enforcing sound enterprise architectures. Our  experience with federal agencies has shown that attempts to modernize  IT environments without blueprints—models simplifying the  complexities of how agencies operate today, how they want to operate  in the future, and how they will get there—often result in unconstrained  investment and systems that are duplicative and ineffective. Enterprise  architectures offer such blueprints. Our reports on the federal  government’s use of enterprise architectures in both February 2002 and  November 2003 found that agencies’ use of enterprise architectures was  a work in progress, with much to be accomplished. Nevertheless,  opportunities exist to significantly improve this outlook if OMB were to  adopt a governmentwide, structured, and systematic approach to  promoting enterprise architecture use, measuring agency progress, and  identifying and pursuing governmentwide solutions to common  enterprise architecture challenges that agencies face. Accordingly, we  made recommendations to OMB to address these areas.
Employing IT system and service management practices. Our work  and other best-practice research have shown that applying rigorous  practices to the acquisition or development of IT systems or the  acquisition of IT services improves the likelihood of success. In other  words, the quality of IT systems and services is governed largely by the  quality of the processes involved in developing or acquiring each. For  example, using models and methods that define and determine  organizations’ software-intensive systems process maturity that were  developed by Carnegie Mellon University’s Software Engineering  Institute, which is recognized for its expertise in software processes, we  evaluated several agencies’ software development or acquisition  processes. We found that agencies are not consistently using rigorous or  disciplined system management practices. We have made numerous  recommendations to agencies to improve their management processes,  and they have taken, or plan to take, actions to improve. Regarding IT  services acquisition, we identified leading commercial practices for  outsourcing IT services that government entities could use to enhance  their acquisition of IT services.
Using effective agency IT investment management practices.
Investments in IT can have a dramatic impact on an organization’s  performance. If managed effectively, these investments can vastly  improve government performance and accountability. If not, however,  they can result in wasteful spending and lost opportunities for  improving delivery of services to the public. Using our information  technology investment management maturity framework, we  evaluated selected agencies and found that while some processes have  been put in place to help them effectively manage their planned and  ongoing IT investments, more work remains.
section_title: IT Challenges Are Interdependent
paragraphs: Complicating the government’s ability to overcome these IT management  challenges are these challenges’ interdependencies. As a result, the  inability of an organization to successfully address one IT management  area can reduce the effectiveness of its success in addressing another  management function. For example, a critical aspect of implementing  effective e-government solutions and developing and deploying major  systems development projects is ensuring that robust information security  is built into these endeavors early and is periodically revisited.
The government’s many IT challenges can be addressed by the use of  effective planning and execution, which can be achieved, in part, through  strategic planning/performance measurement, and investment  management. For example, strong strategic planning is focused on using IT  to help accomplish the highest priority customer needs and mission goals,  while effective performance measurement helps determine the success or  failure of IT activities. Finally, IT investment management provides a  systematic method for minimizing risks while maximizing the return on  investments and involves a process for selecting, controlling, and  evaluating investments. These processes, too, are interdependent. For  example, the investment management process is a principal mechanism to  ensure the effective execution of an agency’s IT strategic plan.
section_title: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine the extent to which federal agencies are  following practices associated with key legislative and other requirements  for (1) IT strategic planning/performance measurement and (2) IT  investment management.
To address these objectives, we identified and reviewed major legislative  requirements and executive orders pertaining to IT strategic planning,  performance measurement, and investment management. Specifically, we  reviewed  the Paperwork Reduction Act of 1995;  the Clinger-Cohen Act of 1996;  the E-Government Act of 2002;  the Federal Information Security Management Act of 2002;  Executive Order 13011, Federal Information Technology; and  Executive Order 13103, Computer Software Piracy.
Using these requirements and policy and guidance issued by OMB and  GAO, we identified 30 IT management practices that (1) can be applied at  the enterprise level and (2) were verifiable through documentation and  interviews. These 30 practices focused on various critical aspects of IT  strategic management, performance measurement, and investment  management, including the development of IRM plans, the identification of  goals and related measures, and the selection and control of IT  investments, respectively.
We selected 26 major departments and agencies for our review (23 entities  identified in 31 U.S.C. 901 and the 3 military services). At our request,  each agency completed a self-assessment on whether and how it had  implemented the 30 IT management practices. We reviewed the completed  agency self-assessments and accompanying documentation, including  agency and IT strategic plans, agency performance plans and reports  required by the Government Performance and Results Act, and IT  investment management policy and guidance, and interviewed applicable  agency IT officials to corroborate whether the practices were in place. We  did not evaluate the effectiveness of agencies’ implementation of the  practices. For example, we did not review specific IT investments to  determine whether they were selected, controlled, and reviewed in  accordance with agency policy and guidance. However, we reviewed  applicable prior GAO and agency inspector general reports and discussed  whether agency policies had been fully implemented with applicable  agency IT officials.
On the basis of the above information, we assessed whether the practices  were in place, using the following definitions:  Yes—the practice was in place.
Partially—the agency has some, but not all, aspects of the practice in  place. Examples of circumstances in which the agency would receive  this designation include when (1) some, but not all, of the elements of  the practice were in place; (2) the agency documented that it has the  information or process in place but it was not in the prescribed form  (e.g., in a specific document as required by law or OMB); (3) the  agency’s documentation was in draft form; or (4) the agency had a policy  related to the practice but evidence supported that it had not been  completely or consistently implemented.
No—the practice was not in place.
Not applicable—the practice was not relevant to the agency’s particular  circumstances.
We also collected information from the Department of Homeland Security  (DHS) but found that since it had been established so recently, it was too  early to judge its IT strategic planning, performance measurement, and  investment management. As a result, although we provided information on  what DHS was doing with respect to these areas, we did not include it in  our assessment.
We also interviewed officials from OMB’s Office of Information and  Regulatory Affairs regarding OMB’s role in establishing policies and  overseeing agencies’ implementation of the identified practices.
We performed our work at the agencies’ offices in greater Washington, D.C.  We conducted our review between April and mid-December 2003 in  accordance with generally accepted government auditing standards.
section_title: Agencies’ Use of IT Strategic Planning/Performance Measurement Practices Is Uneven
paragraphs: The use of IT strategic planning/performance measurement practices is  uneven (see fig. 1), which is of concern because a well-defined strategic  planning process helps ensure that an agency’s IT goals are aligned with  that agency’s strategic goals. Moreover, establishing performance measures  and monitoring actual-versus-expected performance of those measures can  help determine whether IT is making a difference in improving  performance. Among the practices or elements of practices that agencies  largely have in place were those pertaining to establishing goals and  performance measures. On the other hand, agencies are less likely to have  fully documented their IT strategic planning processes, developed  comprehensive IRM plans, linked performance measures to their  enterprisewide IT goals, or monitored actual-versus-expected performance  for these enterprisewide goals. Agencies cited various reasons, such as the  lack of support from agency leadership, for not having strategic  practices/performance measurement practices in place. Without strong  strategic management practices, it is less likely that IT is being used to  maximize improvement in mission performance. Moreover, without  enterprisewide performance measures that are being tracked against actual  results, agencies lack critical information about whether their overall IT  activities, at a governmentwide cost of billions of dollars annually, are  achieving expected goals.
subsections: section_title: Governmentwide Progress Demonstrated, but More Work Remains
paragraphs: Critical aspects of the strategic planning/performance measurement area  include documenting the agency’s IT strategic planning processes,  developing IRM plans, establishing goals, and measuring performance to  evaluate whether goals are being met. Although the agencies often have  these practices, or elements of these practices, in place, additional work  remains, as demonstrated by the following examples:  Strategic planning process. Strategic planning defines what an  organization seeks to accomplish and identifies the strategies it will use  to achieve desired results. A defined strategic planning process allows  an agency to clearly articulate its strategic direction and to establish  linkages among planning elements such as goals, objectives, and  strategies.
About half of the agencies fully documented their strategic planning  processes. For example, the General Services Administration (GSA)  documented an IT governance structure that addresses the roles and  responsibilities of various organizations in strategic planning and  investment management. In addition, in its IT strategic plan, GSA  describes how it developed the plan, including its vision, business- related priorities, and goals. In contrast, the Department of Agriculture  has not completely documented its IT strategic planning process or  integrated its IT management operations and decisions with other  agency processes. According to Agriculture IT officials, the  department’s ongoing budget and performance integration initiative is  expected to result in a more clearly defined and integrated IT strategic  management planning process. Such a process provides the essential  foundation for ensuring that IT resources are effectively managed.
Strategic IRM plans. The Paperwork Reduction Act requires that  agencies indicate in strategic IRM plans how they are applying  information resources to improve the productivity, efficiency, and  effectiveness of government programs. An important element of a  strategic plan is that it presents an integrated system of high-level  decisions that are reached through a formal, visible process. The plan is  thus an effective tool with which to communicate the mission and  direction to stakeholders. In addition, a strategic IRM plan that  communicates a clear and comprehensive vision for how the agency will  use information resources to improve agency performance is important  because IRM encompasses virtually all aspects of an agency’s  information activities.
Although the Paperwork Reduction Act also requires agencies to  develop IRM plans in accordance with OMB’s guidance, OMB does not  provide cohesive guidance on the specific contents of IRM plans. OMB  Circular A-130 directs that agencies have IRM plans that support  agency strategic plans, provide a description of how IRM helps  accomplish agency missions, and ensure that IRM decisions are  integrated with organizational planning, budgets, procurement,  financial management, human resources management, and program  decisions. However, Circular A-130 does not provide overall guidance  on the plan’s contents. As a result, although agencies generally  provided OMB with a variety of planning documents to meet its  requirement that they submit an IRM plan, these plans were generally  limited to IT strategic or e-government issues and did not address other  elements of IRM, as defined by the Paperwork Reduction Act.  Specifically, these plans generally include individual IT projects and  initiatives, security, and enterprise architecture elements but do not  often address other information functions, such as information  collection, records management, and privacy, or the coordinated  management of all information functions.
OMB IT staff agreed that the agency has not set forth guidance on the  contents of agency IRM plans in a single place, stating that its focus has  been on looking at agencies’ cumulative results and not on planning  documents. In addition, these staff also noted that agencies account for  their IRM activities through multiple documents (e.g., Information  Collection Budgets and Government Paperwork Elimination Act  plans). However, the OMB IT staff stated that they would look at  whether more guidance is needed to help agencies in their development  of IRM plans, but have not yet made a commitment to provide such  guidance. Half the agencies indicated a need for OMB to provide  additional guidance on the development and content of IRM plans.
Strong agency strategic IRM plans could also provide valuable input to  a governmentwide IRM plan, which is also required by the Paperwork  Reduction Act. As we reported last year, although OMB designated the  CIO Council’s strategic plan for fiscal years 2001-2002 as the  governmentwide strategic IRM plan, it does not constitute an effective  and comprehensive strategic vision. Accordingly, we recommended  that OMB develop and implement a governmentwide strategic IRM plan  that articulates a comprehensive federal vision and plan for all aspects  of government information. In April 2003, we testified that OMB had  taken a number of actions that demonstrate progress in fulfilling the  Paperwork Reduction Act’s requirement of providing a unifying IRM  vision. However, more remains to be done. In particular, we reported  that although OMB’s strategies and models are promising, their ability  to reduce paperwork burden and accomplish other objectives depends  on how OMB implements them.
One element required by the Clinger-Cohen Act to be included in  agency IRM plans is the identification of a major IT acquisition  program(s), or any phase or increment of that program, that  significantly deviated from cost, performance, or schedule goals  established by the program. However, few agencies met this  requirement. In these cases, a common reason cited for not including  this information was that it was not appropriate to have such detailed  information in a strategic plan because such plans should be forward  thinking and may not be developed every year. Agencies also identified  other mechanisms that they use to track and report cost, schedule, and  performance deviations. Because agencies generally do not address  this Clinger-Cohen Act requirement in their IRM plans, they may benefit  from additional guidance from OMB on how to address this  requirement.
IT goals. The Paperwork Reduction Act and the Clinger-Cohen Act  require agencies to establish goals that address how IT contributes to  program productivity, efficiency, effectiveness, and service delivery to  the public. We have previously reported that leading organizations  define specific goals, objectives, and measures, use a diversity of  measure types, and describe how IT outputs and outcomes impact  operational customer and agency program delivery requirements.
The agencies generally have the types of goals outlined in the  Paperwork Reduction Act and the Clinger-Cohen Act. For example, the  Social Security Administration (SSA) set a goal of achieving an average  of at least a 2 percent per year improvement in productivity, and it  expects that advances in automation will be a key to achieving this goal  along with process and regulation changes. In addition, the Department  of Veterans Affairs’ (VA) latest departmental strategic plan has a goal  that includes using business process reengineering and technology  integration to speed up delivery of benefit payments, improve the  quality of health care provided in its medical centers, and administer  programs more efficiently. The VA goal includes strategies such as  using its enterprise architecture as a continuous improvement process,  implementing e-government solutions to transform paper-based  electronic collections to electronic-based mechanisms, and  establishing a single, high-performance wide area data network. Five  agencies do not have one or more of the goals required by the  Paperwork Reduction Act and the Clinger-Cohen Act. For example, the  Department of Labor’s single IT strategic goal—to provide better and  more secure service to citizens, businesses, government, and Labor  employees to improve mission performance—which it included in its  fiscal year 2004 performance plan, does not address all required goals.  Further, in contrast to other agencies, Labor does not have goals in its  IRM plan. It is important that agencies specify clear goals and  objectives to set the focus and direction of IT performance.
IT performance measures. The Paperwork Reduction Act, the Clinger- Cohen Act, and Executive Order 13103 require agencies to establish a  variety of IT performance measures, such as those related to how IT  contributes to program productivity, efficiency, and effectiveness, and  to monitor the actual-versus-expected performance of those measures.  As we have previously reported, an effective performance management  system offers a variety of benefits, including serving as an early warning  indicator of problems and the effectiveness of corrective actions,  providing input to resource allocation and planning, and providing  periodic feedback to employees, customers, stakeholders, and the  general public about the quality, quantity, cost, and timeliness of  products and services.
Although the agencies largely have one or more of the required  performance measures, these measures are not always linked to the  agencies’ enterprisewide IT goals. For example, the Department of  Defense (DOD), Air Force, and Navy have a variety of enterprisewide  IT goals but do not have performance measures associated with these  goals. Each of these organizations are in the process of developing such  measures. To illustrate, the Air Force’s August 2002 information  strategy includes nine goals, such as providing decision makers and all  Air Force personnel with on-demand access to authoritative, relevant,  and sufficient information to perform their duties efficiently and  effectively, but does not have performance measures for these goals.  The Air Force recognizes the importance of linking performance  measures to its goals and is developing such measures, which it expects  to complete by the fourth quarter of fiscal year 2004.
Leading organizations use performance measures to objectively  evaluate mission, business, and project outcomes. Such organizations  also focus on performance measures for gauging service to key  management processes and tailoring performance measures to  determine whether IT is making a difference in improving performance.  Few agencies monitored actual-versus-expected performance for all of  their enterprisewide IT goals. Specifically, although some agencies  tracked actual-versus-expected outcomes for the IT performance  measures in their performance plans or accountability reports and/or  for specific IT projects, they generally did not track the performance  measures specified in their IRM plans. For example, although the  Department of Health and Human Services’ (HHS) IT strategic plan  identifies enterprisewide goals and performance measures, these  measures generally do not identify quantified outcomes (e.g., the  measures indicate that the outcome will be a percentage transaction  increase or cost decrease in certain areas but do not provide a baseline  or target). In addition, the HHS plan does not describe how the  department will monitor actual-versus-expected performance for these  measures. HHS’s Director of Business Operations in its IRM office  reported that the department recognizes the need to develop an  integrated program for monitoring performance against the  enterprisewide measures in the IT strategic plan. He stated that HHS  has recently begun an initiative to establish such a process. By not  measuring actual-versus-expected performance, agencies lack the  information to determine where to target agency resources to improve  overall mission accomplishment.
Benchmarking. The Clinger-Cohen Act requires agencies to  quantitatively benchmark agency process performance against public-  and private-sector organizations, where comparable processes and  organizations exist. Benchmarking is used by entities because there may  be external organizations that have more innovative or more efficient  processes than their own processes. Our previous study of IT  performance measurement at leading organizations found that they had  spent considerable time and effort comparing their performance  information with that of other organizations.
Seven agencies have mechanisms—such as policies and strategies—in  place related to benchmarking their IT processes. For example, DOD’s  information resources and IT directive states that DOD components  shall routinely and systematically benchmark their functional  processes against models of excellence in the public and private sector  and use these and other analyses to develop, simplify, or refine the  processes before IT solutions are applied. In general, however,  agencies’ benchmarking decisions are ad hoc. Few agencies have  developed a mechanism to identify comparable external private- or  public-sector organizations and processes and/or have policies related  to benchmarking; however, all but 10 of the agencies provided  examples of benchmarking that had been performed. For example, the  Small Business Administration (SBA) does not have benchmarking  policies in place, but the agency provided an example of a  benchmarking study performed by a contractor that compared SBA’s IT  operations and processes against industry cost and performance  benchmarks and best practices and resulted in recommendations for  improvement.
section_title: Practice-Specific Analysis
paragraphs: Table 1 provides additional detail on each strategic planning/performance  measurement practice and our evaluation of whether each agency had the  practice in place. The table indicates that work remains for the agencies to  have each of the practices fully in place as well as that several agencies  reported that they were taking, or planned to take, actions to address the  practices or elements of practices.
Agency IT officials could not identify why practices were not in place in all  cases, but in those instances in which reasons were identified, a variety of  explanations were provided. For example, reasons cited by agency IT  officials included that they lacked the support from agency leadership, that  the agency had not been developing IRM plans until recently and  recognized that the plan needed further refinement, that the process was  being revised (in at least one case because of changes that are needed to  reflect a loss of component organizations to the new DHS), and that  requirements were evolving. In other cases, the agency reported that it had  the information but it was not in the format required by legislation. For  instance, FISMA requires agencies to include in the performance plans  required by the Government Performance and Results Act the resources,  including budget, staffing, and training, and time periods to implement its  information security program. None of the agencies included this  information in their performance plans. However, the agencies commonly  reported that they had this information but that it was in another  document. Nevertheless, this does not negate the need for having the  agency report to the Congress in the required form. This is particularly  important since, as in the example of the FISMA requirement, the reporting  requirement involves a public document, whereas other reports may not be  publicly available.
In the case of DHS, while we did not include the department in our  assessment and in table 1, the department is in the process of developing  its first IT strategic plan. According to DHS, it expects to complete this plan  by mid-February 2004.
section_title: Agencies’ Use of IT Investment Management Practices Is Mixed
paragraphs: The use of IT investment management practices is mixed (as shown in fig.  2), which demonstrates that agencies do not have all the processes in place  to effectively select, control, and evaluate investments. An IT investment  management process is an integrated approach to managing investments  that provides for the continuous identification, selection, control, life-cycle  management, and evaluation of IT investments. Among the investment  management practices that are most frequently in place are having  investment management boards and requiring that projects demonstrate  that they are economically beneficial. Practices less commonly in place are  those requiring that IT investments be performed in a modular, or  incremental, manner and that they be effectively controlled. Only by  effectively and efficiently managing their IT resources through a robust  investment management process can agencies gain opportunities to make  better allocation decisions among many investment alternatives and  further leverage their IT investments.
subsections: section_title: Governmentwide Progress Demonstrated, but More Work Remains
paragraphs: Critical aspects of IT investment management include developing well- supported proposals, establishing investment management boards, and  selecting and controlling IT investments. The agencies’ use of practices  associated with these aspects of investment management is wide-ranging,  as follows:   IT investment proposals. Various legislative requirements, an executive  order, and OMB policies provide minimum standards that govern  agencies’ consideration of IT investments. In addition, we have issued  guidance to agencies for selecting, controlling, and evaluating IT  investments. Such processes help ensure, for example, that  investments are cost-beneficial and meet mission needs and that the  most appropriate development or acquisition approach is chosen.
The agencies in our review have mixed results when evaluated against  these various criteria. For example, the agencies almost always require  that proposed investments demonstrate that they support the agency’s  business needs, are cost-beneficial, address security issues, and  consider alternatives. To demonstrate, the Department of  Transportation requires that proposed projects complete a business  case to indicate that the project (1) will meet basic requirements in  areas such as mission need, affordability, technical standards, and  disabled access requirements, (2) is economically beneficial, and (3)  has considered alternatives.
One element in this area that agencies were not as likely to have fully in  place was the Clinger-Cohen Act requirement that agencies follow, to  the maximum extent practicable, a modular, or incremental, approach  when investing in IT projects. Incremental investment helps to mitigate  the risks inherent in large IT acquisitions/developments by breaking  apart a single large project into smaller, independently useful  components with known and defined relationships and dependencies.  An example of such an approach is DOD’s policy stating that IT  acquisition decisions should be based on phased, evolutionary  segments that are as brief and narrow in scope as possible and that  each segment should solve a specific part of an overall mission problem  and deliver a measurable net benefit independent of future segments.  However, 14 agencies do not have a policy that calls for investments to  be done in a modular manner. For example, although the  Environmental Protection Agency (EPA) reported that it worked with  program offices to try to segment work so that the scope and size of  each project is manageable, it does not have a policy that calls for  investments to be done in a modular manner. The absence of a policy  calls into question whether EPA is implementing incremental  investment in a consistent and effective manner.
Investment management boards. Our investment management guide  states that establishing one or more IT investment boards is a key  component of the investment management process. According to our  guide, the membership of this board should include key business  executives and should be responsible for final project funding decisions  or should provide recommendations for the projects under its scope of  authority. Such executive-level boards, made up of business-unit  executives, concentrate management’s attention on assessing and  managing risks and regulating the trade-offs between continued funding  of existing operations and developing new performance capabilities.
Almost all of the agencies in our review have one or more enterprise- level investment management boards. For example, HUD’s Technology  Investment Board Executive Committee and supporting boards have  responsibility for selecting, controlling, and evaluating the  department’s IT investments. HUD’s contractor-performed maturity  audits also have helped the department validate its board structure and  its related investment management processes. However, the investment  management boards for six agencies are not involved, or the agency did  not document the board’s involvement, in the control phase. For  example, the National Science Foundation (NSF) has a CIO advisory  group that addresses only the select phase of the IT investment  management process. NSF’s CIO explained that the agency reviews the  progress of its major information system projects through other means,  such as meetings with management. In providing comments on a draft  of this report, the CIO stated that he believes that NSF has a  comprehensive set of management processes and review structures to  select, control, and evaluate IT investments and cited various groups  and committees used as part of this process. However, NSF’s summary  of its investment management process and memo establishing the CIO  advisory group include only general statements related to the oversight  of IT investments, and NSF provided no additional documentation  demonstrating that its investment management board plays a role in the  control and evaluation phases. Our investment management guidance  identifies having an IT investment management board(s) be responsible  for project oversight as a critical process. Maintaining responsibility for  oversight with the same body that selected the investment is crucial to  fostering a culture of accountability by holding the investment board  that initially selected an investment responsible for its ongoing success.
In addition, 17 agencies do not fully address the practice that calls for  processes to be in place that address the coordination and alignment of  multiple investment review boards. For example, we recently reported  that the Department of the Interior has established three department- level IT investment boards and begun to take steps to ensure that  investment boards are established at the bureau level. However, at the  time of our review, the department (1) could not assert that  department-level board members exhibited core competencies in using  Interior’s IT investment approach and (2) had limited ability to oversee  investments in its bureaus. We made recommendations to Interior to  strengthen both the activities of the department-level boards and the  department’s ability to oversee investment management activities at the  bureaus.
Selection of IT investments. During the selection phase of an IT  investment management process, the organization (1) selects projects  that will best support its mission needs and (2) identifies and analyzes  each project’s risks and returns before committing significant funds. To  achieve desired results, it is important that agencies have a selection  process that, for example, uses selection criteria to choose the IT  investments that best support the organization’s mission and prioritizes  proposals.
Twenty-two agencies use selection criteria in choosing their IT  investments. In addition, about half the agencies use scoring models  to help choose their investments. For example, the working group and  CIO office officials that support the Department of Education’s  investment review board used a scoring model as part of deciding  which IT investments to recommend for the board’s consideration and  approval. This model contained two main categories of criteria: (1)  value criteria that measured the impact and significance of the  initiative, given project goals and the strategic objectives of the  department; and (2) health criteria that measured the potential for the  success of the initiative and helped to assess both the performance and  the associated risks that are involved in project and contract  management. In the case of DOD, in February 2003 we reported that it  had established some, and was establishing other IT investment  criteria, but these criteria had not been finalized. Accordingly, we  recommended, and DOD concurred, that DOD establish a standard set  of criteria. In September we reported that this recommendation had not  been implemented. DOD officials stated that the department was  developing the criteria but that the proposed governance structure had  not yet been adopted.
Control over IT investments. During the control phase of the IT  investment management process, the organization ensures that, as  projects develop and as funds are spent, the project is continuing to  meet mission needs at the expected levels of cost and risk. If the project  is not meeting expectations or if problems have arisen, steps are quickly  taken to address the deficiencies. Executive level oversight of project- level management activities provides the organization with increased  assurance that each investment will achieve the desired cost, benefit,  and schedule results.
Although no agencies had the practices associated with the control  phase fully in place, some have implemented important aspects of this  phase. For example, Labor requires project managers to prepare a  control status report based on a review schedule established during the  selection phase, which is reviewed by the Office of the CIO and its  technical review board as part of determining whether to continue,  modify, or cancel the initiative. For initiatives meeting certain criteria,  the technical review board makes recommendations to the  management council, which serves as the department’s top tier  executive investment review council, is chaired by the Assistant  Secretary of Administration and Management, and consists of  component agency heads.
Nevertheless, in general, the agencies are weaker in the practices  pertaining to the control phase of the investment management process  than in the selection phase. In particular, the agencies did not always  have important mechanisms in place for agencywide investment  management boards to effectively control investments, including  decision-making rules for project oversight, early warning mechanisms,  and/or requirements that corrective actions for under-performing  projects be agreed upon and tracked. For example, the Department of  the Treasury does not have a department-level control process; instead,  each bureau may conduct its own reviews that address the  performance of its IT investments and corrective actions for under- performing projects. In a multitiered organization like Treasury, the  department is responsible for providing leadership and oversight for  foundational critical processes by ensuring that written policies and  procedures are established, repositories of information are created that  support IT investment decision making, resources are allocated,  responsibilities are assigned, and all of the activities are properly  carried out where they may be most effectively executed. In such an  organization, the CIO is specifically responsible for ensuring that the  organization is effectively managing its IT investments at every level.  Treasury IT officials recognize the department’s weaknesses in this  area and informed us that they are working on developing a new capital  planning and investment control process that is expected to address  these weaknesses. Similarly, the Department of Energy is planning on  implementing the investment control process outlined in its September  2003 capital planning and investment control guide in fiscal year 2004,  which addresses important elements such as corrective action plans.  However, this guide does not document the role of Energy’s investment  management boards in this process.
section_title: Practice-Specific Analysis
paragraphs: Table 2 provides additional detail on each investment management practice  and our evaluation of whether each agency had the practice in place. The  table indicates those practices in which improvement is needed as well as  which agencies reported that they were taking, or planned to take, actions  to address the practices or elements of practices.
Among the variety of reasons cited for practices not being fully in place  were that the CIO position had been vacant, that not including a  requirement in the IT investment management guide was an oversight, and  that the process was being revised. However, in some cases the agencies  could not identify why certain practices were not in place.
Regarding DHS, although we did not include the department in our  assessment or table 2, the department has investment management  processes that it has put in place or is in the process of putting in place.
section_title: Conclusions
paragraphs: Federal agencies did not always have in place important practices  associated with IT laws, policies, and guidance. At the governmentwide  level, agencies generally have IT strategic plans or information resources  management (IRM) plans that address IT elements, such as security and  enterprise architecture, but do not cover other aspects of IRM that are part  of the Paperwork Reduction Act, such as information collection, records  management, and privacy. This may be attributed, in part, to OMB not  establishing comprehensive guidance for the agencies detailing the  elements that should be included in such a plan. There were also numerous  instances of individual agencies that do not have specific IT strategic  planning, performance measurement, or investment management practices  fully in place. Agencies cited a variety of reasons for not having these  practices in place, such as that the CIO position had been vacant, not  including a requirement in guidance was an oversight, or that the process  was being revised. Nevertheless, not only are these practices based on law,  executive orders, OMB policies, and our guidance, but they are also  important ingredients for ensuring effective strategic planning,  performance measurement, and investment management, which, in turn,  make it more likely that the billions of dollars in government IT  investments will be wisely spent. Accordingly, we believe that it is  important that they be expeditiously implemented by individual agencies.
section_title: Recommendations
paragraphs: To help agencies in developing strategic IRM plans that fully comply with  the Paperwork Reduction Act of 1995, we recommend that the Director,  OMB, develop and disseminate to agencies guidance on developing such  plans. At a minimum, such guidance should address all elements of IRM, as  defined by the Paperwork Reduction Act. As part of this guidance, OMB  should also consider the most effective means for agencies to  communicate information about any major IT acquisition program(s) or  phase or increment of that program that significantly deviated from cost,  performance, or schedule goals established by the program. One option for  communicating this information, for example, could be through the annual  agency performance reports that are required by the Government  Performance and Results Act.
We are also generally making recommendations to the agencies in our  review regarding those practices that are not fully in place unless, for  example, (1) we have outstanding recommendations related to the  practice, (2) the agency has a draft document addressing the practice, or  (3) implementation of the practice was ongoing. Appendix I contains these  recommendations.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written or oral comments on a draft of this report from OMB  and 25 of the agencies in our review.  We also requested comments from  the Department of Homeland Security and the Office of Personnel  Management, but none were provided.
Regarding OMB, in oral comments on a draft of this report, representatives  from OMB’s Office of Information and Regulatory Affairs and Office of the  General Counsel questioned the need for additional IRM plan guidance  because they do not want to be prescriptive in terms of what agencies  include in their plans. We continue to believe that agencies need additional  guidance from OMB on the development and content of their IRM plans  because OMB Circular A-130 does not provide overall guidance on the  contents of agency IRM plans and half the agencies indicated a need for  OMB to provide additional guidance on the development and content of  IRM plans. Further, additional guidance would help to ensure that agency  plans address all elements of IRM, as defined by the Paperwork Reduction  Act. A strategic IRM plan that communicates a clear and comprehensive  vision for how the agency will use information resources to improve  agency performance is important because IRM encompasses virtually all  aspects of an agency’s information activities.
In commenting on a draft of the report, most of the agencies in our review  generally agreed with our findings and recommendations. The agencies’  specific comments are as follows:  Agriculture’s CIO stated that the department concurred with the  findings in this report and provided information on action it was taking,  or planned to take, to implement the recommendations. Agriculture’s  written comments are reproduced in appendix II.
The Secretary of Commerce concurred with the recommendations in  this report and stated that, in response, the department is updating its  policies and procedures. Commerce’s written comments are reproduced  in appendix III.
DOD submitted a single letter that included comments from the Departments of the Air  Force, Army, and Navy. recommendations in this report. DOD also provided additional  documentation and information on actions that it is taking, or planned  to take, to address these recommendations. We modified our report  based on these comments and documentation, as appropriate. DOD’s  written comments, along with our responses, are reproduced in  appendix IV.
Education’s Assistant Secretary for Management/CIO stated that the  agency generally agreed with our assessment of the department’s use of  IT strategic planning/performance measurement and investment  management practices. Education provided additional comments and  documentation related to two of our practices. We modified our report  on the basis of these comments and documentation, as appropriate.  Education’s written comments, along with our responses, are  reproduced in appendix V.
Energy’s Director of Architecture and Standards provided e-mail  comments stating that the department believes that GAO fairly depicted  where the department currently stands in the IT investment  management process. The director also provided other comments that  were technical in nature and that we addressed, as appropriate.
EPA’s Assistant Administrator/CIO generally agreed with our findings  and recommendations on the need to complete work currently under  way to formalize the documentation of IT management practices.  However, EPA questioned our characterization of the agency’s IT  management and strategic planning and provided other comments,  which we addressed, as appropriate. EPA’s written comments, along  with our responses, are reproduced in appendix VI.
GSA’s CIO stated that the agency generally agreed with the findings and  recommendations in the report. GSA provided suggested changes and  additional information and documentation related to nine of our  practices and two recommendations. We modified our report on the  basis of these comments and documentation, as appropriate. GSA’s  written comments, along with our responses, are reproduced in  appendix VII.
HHS’s Acting Principal Deputy Inspector General stated that the  department concurred with the findings and recommendations of the  report. HHS’s written comments are reproduced in appendix VIII.
HUD’s Assistant Secretary for Administration/CIO stated that the  department was in agreement with the recommendations in this report.  HUD’s written comments are reproduced in appendix IX.
Interior’s Acting Assistant Secretary for Policy, Management and Budget  stated that the recommendations in our report would further improve  the department’s IT investment management. Interior’s written  comments are reproduced in appendix X.
Justice’s CIO stated that, overall, the department concurred with the  findings and recommendations in this report, noting that our  recommendations will assist in further defining IT strategic planning,  performance measurement, and investment management practices.  Justice’s written comments, along with our response, are reproduced in  appendix XI.
Labor’s Assistant Secretary for Administration and Management/CIO  reported that the department generally concurred with this report and  provided suggested changes in two areas, which we addressed, as  appropriate. Labor’s written comments, along with our responses, are  reproduced in appendix XII.
NASA’s Deputy Administrator reported that the agency generally  concurred with the recommendations in this report and provided  additional information on actions that it is taking, or planned to take, to  address these recommendations. NASA’s written comments, along with  our response, are reproduced in appendix XIII.
NSF’s CIO provided e-mail comments disagreeing with three areas of  this report. First, NSF did not agree with our assessment of practice 1.1,  stating that the agency has a comprehensive agency-level planning  framework that includes a suite of planning documents and internal and  external oversight activities that it believes addresses IT planning  requirements. However, our review of the planning documents cited by  NSF in its self-assessment found that it did not address the elements of  the practice. In particular, the agency did not describe the responsibility  and accountability for IT resources or the method that it uses to define  program information needs and how such needs will be met. Moreover,  in our exit conference with NSF officials, the CIO indicated agreement  with our assessment. Since NSF provided no additional documentation,  we did not modify the report. Second, the CIO disagreed with our  characterization of the agency’s enterprisewide investment management  board. We modified the report to reflect the CIO’s comments; however,  we did not change our overall assessment of the role of the board  because NSF’s summary of its investment management process and  memo establishing the CIO advisory group include only general  statements related to the oversight of IT investments, and NSF provided  no additional documentation demonstrating that its investment  management board plays a role in the control and evaluation phases.  Third, the CIO stated that NSF has established processes, management,  and oversight controls over IT investments. However, NSF provided  limited documentation on the control phase of its investment  management process. In particular, NSF’s summary of its investment  management process and memo establishing the CIO advisory group  include only general statements related to the oversight of IT  investments, and NSF provided no additional documentation  demonstrating that its investment management board plays a role in the  control and evaluation phases. Accordingly, we did not modify the  report.
NRC’s Executive Director for Operations stated that this report provides  useful information and agreed that the practices are important for  ensuring effective use of government IT investments but had no specific  comments. NRC’s written comments are reproduced in appendix XIV.
SBA’s GAO liaison provided e-mail comments questioning the need to  have its enterprise investment management board have final decision- making authority over IT investments. Our IT investment management  guidance states that enterprise-level IT investment boards be capable of  reviewing lower-level board actions and invoking final decision-making  authority over all IT investments. In particular, if disputes or  disagreements arise over decision-making jurisdiction about a specific  IT investment project, the enterprise board must be able to resolve the  issue. Accordingly, we did not modify the report. SBA also provided  technical comments that we incorporated, as appropriate.
SSA’s Commissioner generally agreed with the recommendations in the  report and provided comments on each recommendation that we  addressed, as appropriate. SSA’s written comments, along with our  responses, are reproduced in appendix XV.
State’s Assistant Secretary/Chief Financial Officer stated that the  findings in the report are consistent with discussions held with its IT  staff and provided additional information on four practices. On the basis  of this additional information, we modified our report, as appropriate.  State’s written comments, along with our response, are reproduced in  appendix XVI.
A program analyst in the Department of Transportation’s Office of the  CIO provided oral comments that were technical in nature that we  addressed, as appropriate.
The Acting Director, Budget and Administrative Management in  Treasury's Office of the CIO, provided oral comments stating that the  department concurred with our findings and recommendations. The  official further stated that the department recognized its shortcomings  and was working to correct them.
USAID’s Assistant Administrator, Bureau for Management, did not  address whether the agency agreed or disagreed with our overall  findings or recommendations but commented on our evaluation of two  practices, which we addressed, as appropriate. USAID’s written  comments, along with our response, are reproduced in appendix XVII.
The Secretary of VA stated that the department concurred with the  recommendations in the report and provided comments on actions that  it has taken, or planned to take, in response. We modified the report  based on these comments, as appropriate. VA’s written comments, along  with our responses, are reproduced in appendix XVIII.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  secretaries of the Departments of Agriculture, the Air Force, the Army,  Commerce, Defense, Education, Energy, Health and Human Services,  Homeland Security, Housing and Urban Development, the Interior, Justice,  Labor, the Navy, State, Transportation, the Treasury, and Veterans Affairs;  the administrators of the Environmental Protection Agency, General  Services Administration, National Aeronautics and Space Administration,  Small Business Administration, and U.S. Agency for International  Development; the commissioners of the Nuclear Regulatory Commission  and the Social Security Administration; and the directors of the National  Science Foundation, Office of Management and Budget, and Office of  Personnel Management. We will also make copies available to others upon  request. In addition, this report will be available at no charge on the GAO  Web site at http://www.gao.gov.
If you have any questions on matters discussed in this report, please  contact me at (202) 512-9286 or Linda J. Lambert, Assistant Director, at  (202) 512-9556. We can also be reached by e-mail at pownerd@gao.gov and  lambertl@gao.gov, respectively.
Other contacts and key contributors to this report are listed in appendix  XIX.
section_title: Recommendations to Departments and Agencies
subsections: section_title: Agriculture
paragraphs: To improve the department’s information technology (IT) strategic  planning/performance measurement processes, we recommend that the  Secretary of Agriculture take the following six actions:  document the department’s IT strategic management processes and how  they are integrated with other major departmental processes, such as  the budget and human resources management;  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by the Federal Information Security Management Act  (FISMA) and include a description of major IT acquisitions contained in  its capital asset plan that bear significantly on its performance goals;  implement a process for assigning roles and responsibilities for  achieving the department’s IT goals;  develop performance measures related to the effectiveness of controls  to prevent software piracy;  track actual-versus-expected performance for the department’s  enterprisewide IT performance measures in its information resources  management (IRM) plan; and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Agriculture take the following four  actions:  include a description of the relationship between the IT investment  management process and the department’s enterprise architecture in its  IT capital planning and investment control guide and require that IT  investments be in compliance with the agency’s enterprise architecture;  document the alignment and coordination of responsibilities of the  department’s various IT investment management boards for decision  making related to IT investments, including cross-cutting investments;  establish a policy requiring that proposed IT investments support work  processes that have been simplified or redesigned to reduce costs and  improve effectiveness and that makes maximum use of commercial-off- the-shelf (COTS) software; and  establish a policy requiring modularized IT investments.
section_title: Air Force
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of the Air Force  take the following two actions:  establish a documented process for measuring progress against the  department’s IT goals and assign roles and responsibilities for achieving  these goals; and  develop IT performance measures related to the IT goals in the  department’s information strategy, including measures such as those  contained in practice 1.9 in our report, and track actual-versus-expected  performance.
To improve the department’s IT investment management processes, we  recommend that the Secretary of the Air Force take the following four  actions:  include a description of the relationship between the IT investment  management process and the department’s enterprise architecture, and  an identification of external and environmental factors in its portfolio  management guide;  include costs, benefits, schedule, and risk elements as well as measures  such as net benefits, net risks, and risk-adjusted return-on-investment in  the department’s project selection criteria;  implement a scoring model and develop a prioritized list of IT  investments as part of its project selection process; and  document the role, responsibility, and authority of its IT investment  management boards, including work processes, alignment, and  coordination of decision making among its various boards, and  document processes for controlling and evaluating IT investments, such  as those outlined in practices 2.15, 2.16, 2.17, and 2.18.
section_title: Army
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of the Army  take the following action:  complete the development of IT performance measures related to the  Army’s enterprisewide IT goals, including measures such as those in  practice 1.9 in our report, and track actual-versus-expected  performance.
To improve the department’s IT investment management processes, we  recommend that the Secretary of the Army take the following four actions:  include a description of the relationship between the IT investment  management process and the department’s enterprise architecture in  the department’s IT capital planning and investment control guide;  document the alignment and coordination of responsibilities of its  various IT investment management boards for decision making related  to IT investments;  include costs, benefits, schedule, and risk elements as well as measures  such as net benefits, net risks, and risk-adjusted return-on-investment in  the department’s project selection criteria; and  involve the department’s IT investment management boards in  controlling and evaluating IT investments, including the development  and documentation of oversight processes such as those in practices  2.15, 2.16, 2.17, and 2.18.
section_title: Commerce
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of Commerce  take the following four actions:  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;  document its process of developing IT goals in support of agency needs,  measuring progress against these goals, and assigning roles and  responsibilities for achieving these goals;  develop performance measures related to the department’s IT goals in  its IRM plan, and track actual-versus-expected performance for these IT  performance measures; and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Commerce take the following eight  actions:  document the alignment and coordination of responsibilities of the  department’s various IT investment management boards for decision  making related to IT investments;  establish a policy requiring that proposed IT investments support work  processes that have been simplified or redesigned to reduce costs and  improve effectiveness and that makes maximum use of COTS software;   include net risks and risk-adjusted return-on-investment in the  department’s project selection criteria;  establish a policy requiring modularized IT investments;  develop decision-making rules to help guide the investment  management board’s oversight of IT investments during the control  phase;  require that reports of deviations in systems capability in a project be  submitted to the IT investment management board;  develop an early warning mechanism that enables the investment  management board to take corrective action at the first sign of cost,  schedule, or performance slippages; and  require postimplementation reviews be completed and the results  reported to its investment management board.
section_title: Defense
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of Defense take  the following three actions:  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA, align its performance measures with the goals  in the plan, and include a description of major IT acquisitions contained  in its capital asset plan that bear significantly on its performance goals;  establish a documented process for measuring progress against the  department’s IT goals;   develop IT performance measures related to its IT goals, including, for  example, the measures contained in practice 1.9 in our report and track  actual-versus-expected performance.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Defense take the following action:  document, as part of its planned IT portfolio management process, how  this process relates to other departmental processes and the  department’s enterprise architecture, and document the external and  environmental factors that influence the process.
section_title: Education
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of Education  take the following four actions:  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;   establish and document a process for measuring progress against the  department’s IT goals in its IRM plan and for assigning roles and  responsibilities for achieving these goals;  develop performance measures related to how IT contributes to  program productivity, the effectiveness and efficiency of agency  operations, and the effectiveness of controls to prevent software piracy;  and  track actual-versus-expected performance for the department’s  enterprisewide IT performance measures in its IRM plan.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Education take the following five actions:  document the alignment and coordination of responsibilities of the  department’s various IT investment management boards for decision  making related to IT investments;  establish a policy requiring that proposed IT investments support work  processes that have been simplified or redesigned to reduce costs;  include net risks and risk-adjusted return-on-investment in the  department’s project selection criteria;  develop a process to use independent verification and validation  reviews, when appropriate; and  track the resolution of corrective actions for under-performing projects  and report the results to the investment management board.
section_title: Energy
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of Energy take  the following six actions:  document how its IT management operations and decisions are  integrated with human resources management;  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;  develop a goal related to how IT contributes to program productivity;  develop performance measures related to how IT contributes to  program productivity and the effectiveness of controls to prevent  software piracy;   develop and link performance measures to the department’s  enterprisewide goals in its IRM plan and track actual-versus-expected  performance for these measures; and   develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Energy take the following four actions:  include interfaces in its inventory of the agency’s major information  systems, implement a standard, documented procedure to maintain this  inventory, and develop a mechanism to use the inventory as part of  managerial decision making;  prioritize the department’s IT proposals;  establish a policy requiring modularized IT investments; and  document the role, responsibility, and authority of its IT investment  management boards, including work processes, alignment, and  coordination of decision making among its various boards, and  document the processes for controlling and evaluating IT investments,  such as those in practices 2.15, 2.16, 2.17, and 2.18.
section_title: Environmental Protection Agency
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Administrator of the Environmental  Protection Agency take the following six actions:  document the agency’s IT strategic management processes and how  they are integrated with other major departmental processes, such as  the budget and human resources management;  include in the agency’s annual performance plan the resources and time  periods required to implement the information security program plan  required by FISMA;  develop a documented process to develop IT goals in support of agency  needs, measure progress against these goals, and assign roles and  responsibilities for achieving these goals;  develop performance measures related to the effectiveness of controls  to prevent software piracy;  track actual-versus-expected performance for the agency’s measures  associated with the IT goals in its IRM plan; and  develop a mechanism for benchmarking the agency’s IT management  processes, when appropriate.
To improve the agency’s IT investment management processes, we  recommend that the Administrator of the Environmental Protection  Agency take the following three actions:  include net risks, risk-adjusted return-on-investment, and qualitative  criteria in the agency’s project selection criteria;  establish a policy requiring modularized IT investments; and  fully implement an IT investment management control phase, including  the elements contained in practices 2.15, 2.16, and 2.17.
section_title: General Services Administration
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Administrator of the General Services  Administration take the following four actions:  include in the agency’s annual performance plan the resources and time  periods required to implement the information security program plan  required by FISMA;   develop performance measures related to the effectiveness of controls  to prevent software piracy;  track actual-versus-expected performance for each of the agency’s  measures associated with the IT goals in its IRM plan; and  develop a mechanism for benchmarking the agency’s IT management  processes, when appropriate.
To improve the agency’s IT investment management processes, we  recommend that the Administrator of the General Services Administration  take the following four actions:  develop work processes and decision-making processes for the agency’s  investment management boards;  establish a policy requiring modularized IT investments;  help guide the oversight of IT investments by developing clear decision- making rules for its IT investment management board and by requiring  that IT projects report on deviations in system capability; and  track the resolution of corrective actions for under-performing projects  and report the results to the investment management board.
section_title: Health and Human Services
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of Health and  Human Services take the following six actions:  document the department’s IT strategic management processes and how  they are integrated with its budget processes;  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA and include a description of major IT  acquisitions contained in its capital asset plan that bear significantly on  its performance goals;  establish a documented process for measuring progress against the  department’s IT goals;  develop performance measures related to the effectiveness of controls  to prevent software piracy;  track actual-versus-expected performance for its enterprisewide IT  performance measures in its IRM plan; and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Health and Human Services take the  following 10 actions:  revise the department’s IT investment management policy to include (1)  how this process relates to other agency processes, (2) an identification  of external and environmental factors, (3) a description of the  relationship between the process and the department’s enterprise  architecture, and (4) the use of independent verification and validation  reviews, when appropriate.  develop procedures for the department’s enterprisewide investment  management board to document and review IT investments;  document the alignment and coordination of responsibilities of the  department’s various IT investment management boards for decision  making related to IT investments;  implement a standard, documented procedure to maintain the  department’s inventory of major information systems and develop a  mechanism to use the inventory as part of managerial decision making;  establish a policy requiring that proposed IT investments support work  processes that have been simplified or redesigned to reduce costs and  improve effectiveness;  implement a structured IT selection process that includes processes and  criteria such as those in practices 2.12 and 2.13;  develop decision-making rules to help guide the investment  management board’s oversight of IT investments during the control  phase;  require the investment management board to review projects at major   track the resolution of corrective actions for under-performing projects  and report the results to the investment management board; and  revise the department’s investment management policy to require  postimplementation reviews to address validating benefits and costs,  and conduct such reviews.
section_title: Housing and Urban Development
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of Housing and  Urban Development take the following six actions:  document the roles and responsibilities of the chief financial officer and  program managers in IT strategic planning and how the department’s IT  management operations and decisions are integrated with human  resources management;  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;  develop a documented process to develop IT goals in support of agency  needs, measure progress against these goals, and assign roles and  responsibilities for achieving these goals;  develop performance measures related to how IT contributes to  program productivity and the effectiveness of controls to prevent  software piracy;   track actual-versus-expected performance for the department’s  enterprisewide IT performance measures in its IRM plan; and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Housing and Urban Development take the  following five actions:  establish a policy requiring that proposed IT investments support work  processes that have been simplified or redesigned to reduce costs and  improve effectiveness and that makes maximum use of COTS software;  include net risks and risk-adjusted return-on-investment in the  department’s project selection criteria;  establish a policy requiring modularized IT investments;  require IT projects to report on deviations in system capability and  monitor IT projects at key milestones; and  develop a process to use independent verification and validation  reviews, when appropriate.
section_title: Interior
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of the Interior  take the following six actions:  document the department’s IT strategic management processes and how  they are integrated with other major departmental processes, including  organizational planning, budget, financial management, human  resources management, and program decisions;  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA and include a description of major IT  acquisitions contained in its capital asset plan that bear significantly on  its performance goals;  develop a documented process to develop IT goals in support of agency  needs, measure progress against these goals, and assign roles and  responsibilities for achieving these goals;  develop performance measures related to the effectiveness of controls  to prevent software piracy;   track actual-versus-expected performance for the department’s  enterprisewide IT performance measures in its IRM plan; and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of the Interior take the following five  actions:  establish a policy requiring that proposed IT investments support work  processes that have been simplified or redesigned to reduce costs and  improve effectiveness;  include cost and schedule in the department’s project selection criteria  and prioritize its IT proposals;  establish a policy requiring modularized IT investments;   require that corrective actions be undertaken, tracked, and reported to  the investment management board for under-performing projects; and  implement an evaluation process for IT investments that addresses the  elements of practice 2.18.
section_title: Justice
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Attorney General take the  following six actions:  document the department’s IT strategic management processes;  document how the department’s IT management operations and  decisions are integrated with human resources management processes;  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;  develop a documented process to develop IT goals in support of agency  needs, measure progress against these goals, and assign roles and  responsibilities for achieving these goals;  develop performance measures related to the department’s IT goals in  its IRM plan, and track actual-versus-expected performance for these IT  performance measures; and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Attorney General take the following five actions:  develop work processes and procedures for the department’s  investment management boards, including aligning and coordinating IT  investment decision making among its various boards;  establish a policy requiring that IT investments be in compliance with  the agency’s enterprise architecture;  include net risks and risk-adjusted return-on-investment in the  department’s project selection criteria;  implement a scoring model and develop a prioritized list of investments  as part of the department’s project selection process; and  require that corrective actions be undertaken, tracked, and reported to  the investment management board for under-performing projects.
section_title: Labor
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of Labor take  the following five actions:  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;  develop a documented process to develop IT goals in support of agency  needs, measure progress against these goals, and assign roles and  responsibilities for achieving these goals;  develop a goal related to how IT contributes to program productivity;  develop performance measures related to how IT contributes to  program productivity, efficiency, and the effectiveness of controls to  prevent software piracy, and track actual-versus-expected performance;  and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Labor take the following five actions:  include a description of the relationship between the IT investment  management process and the department’s enterprise architecture in  the department’s IT capital planning and investment control guide;  include net risks and risk-adjusted return-on-investment in its project  selection criteria;   establish a policy requiring modularized IT investments;   develop decision-making rules to help guide the investment  management board’s oversight of IT investments during the control  phase; and   develop an early warning mechanism that enables the investment  management board to take corrective action at the first sign of cost,  schedule, or performance slippages.
section_title: National Aeronautics and Space Administration
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Administrator of the National  Aeronautics and Space Administration take the following seven actions:  document the agency’s IT strategic management processes;  document how the agency’s IT management operations and decisions  are integrated with human resources management processes;  include in the agency’s annual performance plan the resources and time  periods required to implement the information security program plan  required by FISMA;  develop a documented process to develop IT goals in support of agency  needs, measure progress against these goals, and assign roles and  responsibilities for achieving these goals;  develop performance measures related to the effectiveness of controls  to prevent software piracy;  track actual-versus-expected performance for the agency’s  enterprisewide IT performance measures in its IRM plan; and  develop a mechanism for benchmarking the agency’s IT management  processes, when appropriate.
To improve the agency’s IT investment management processes, we  recommend that the Administrator of the National Aeronautics and Space  Administration take the following four actions:  revise the agency’s IT investment management policy and guidance to  describe the relationship of this process to the agency’s enterprise  architecture;  include interfaces in its inventory of the agency’s major information  systems, implement a standard, documented procedure to maintain this  inventory, and develop a mechanism to use the inventory as part of  managerial decision making;  within the agency’s IT investment selection process, implement a  mechanism to identify possible conflicting, overlapping, strategically  unlinked, or redundant proposals; implement a scoring model; and  develop a prioritized list of investments; and  document the role, responsibility, and authority of its IT investment  management boards, including work processes, alignment, and  coordination of decision making among its various boards, and  document the processes for controlling and evaluating IT investments,  such as those in practices 2.15, 2.16, 2.17, and 2.18.
section_title: National Science Foundation
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Director of the National Science  Foundation take the following five actions:  document the agency’s IT strategic management processes;  include in the agency’s annual performance plan the resources and time  periods required to implement the information security program plan  required by FISMA;   implement a process for assigning roles and responsibilities for  achieving its IT goals;   develop performance measures related to the effectiveness of controls  to prevent software piracy; and  develop a mechanism for benchmarking the agency’s IT management  processes, when appropriate.
To improve the agency’s IT investment management processes, we  recommend that the Director of the National Science Foundation take the  following four actions:  develop an IT investment management guide that includes a description  of the relationship between the IT investment management process and  the agency’s other organizational plans and processes and its enterprise  architecture, and identify external and environmental factors that  influence the process in the agency’s IT capital planning and investment  control policy;  implement a structured IT selection process that includes the elements  of practices 2.12 and 2.13;   involve the department’s IT investment management board in  controlling and evaluating IT investments, including the development  and documentation of oversight processes such as those in practices  2.15, 2.16, 2.17, and 2.18; and  define and document the elements of the agency’s postimplementation  reviews.
section_title: Navy
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of the Navy  take the following three actions:  develop a documented process to measure progress against the  department’s enterprisewide IT goals and assign roles and  responsibilities for achieving these goals;  develop an IT goal related to service delivery to the public; and  develop IT performance measures related to the department’s IT goals,  including, at a minimum, measures contained in practice 1.9 in our  report, and track actual-versus-expected performance.
To improve the department’s IT investment management processes, we  recommend that the Secretary of the Navy take the following four actions:  include net risks and risk-adjusted return-on-investment in the  department’s project selection criteria;   implement a structured IT selection process that includes the elements   involve all elements of the department’s IT investment management  board governance process in selecting, controlling, and evaluating IT  investments; and  document the role, responsibility, and authority of its IT investment  management boards, including work processes, alignment, and  coordination of decision making among its various boards, and  document the processes for controlling and evaluating IT investments,  such as those outlined in practices 2.15, 2.16, 2.17, and 2.18.
section_title: Nuclear Regulatory Commission
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Commissioner of the Nuclear  Regulatory Commission take the following five actions:  document the agency’s roles and responsibilities for its IT strategic  management processes and how IT planning is integrated with its  budget and human resources planning;  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;  develop a documented process to assign roles and responsibilities for  achieving its enterprisewide IT goals;  develop performance measures related to the effectiveness of controls  to prevent software piracy; and  develop performance measures for the agency’s enterprisewide goals in  its IRM plan, and track actual-versus-expected performance for these  measures.
To improve the agency’s IT investment management processes, we  recommend that the Commissioner of the Nuclear Regulatory Commission  take the following five actions:  include a description of the relationship between the IT investment  management process and the department’s other organizational plans  and processes and its enterprise architecture, and identify external and  environmental factors that influence the process in the agency’s IT  capital planning and investment control policy;  develop work processes and procedures for the agency’s investment   implement a standard, documented procedure to maintain its IT asset  inventory, and develop a mechanism to use the inventory as part of  managerial decision making;  develop a structured IT investment management selection process that  includes project selection criteria, a scoring model, and prioritization of  proposed investments; and  document the role, responsibility, and authority of its IT investment  management boards, including work processes and control, and  evaluate processes that address the oversight of IT investments, such as  what is outlined in practices 2.15, 2.16, 2.17, and 2.18.
section_title: Office of Personnel Management
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Director of the Office of Personnel  Management take the following four actions:  include in the agency’s annual performance plan the resources and time  periods required to implement the information security program plan  required by FISMA;  develop performance measures related to the effectiveness of controls  to prevent software piracy;  track actual-versus-expected performance for the agency’s  enterprisewide IT performance measures in its IRM plan; and  develop a mechanism for benchmarking the agency’s IT management  processes, when appropriate.
To improve the agency’s IT investment management processes, we  recommend that the Director of the Office of Personnel Management take  the following four actions:  develop work processes and procedures for the agency’s investment  management board, including establishing criteria for defining major  systems and documenting a process for handling cross-functional  investments;  implement a policy requiring that proposed IT investments support  work processes that have been simplified or redesigned to reduce costs  and improve effectiveness and that makes maximum use of COTS  software;  establish a policy requiring modularized IT investments; and  require that corrective actions be undertaken, tracked, and reported to  the investment management board for under-performing projects.
section_title: Small Business Administration
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Administrator of the Small Business  Administration take the following five actions:  document the agency’s IT strategic management processes;  include in the agency’s annual performance plan the resources and time  periods required to implement the information security program plan  required by FISMA;  develop a documented process to develop IT goals in support of agency  needs, measure progress against these goals, and assign roles and  responsibilities for achieving these goals;  develop performance measures related to the agency’s IT goals in its  IRM plan, including, at a minimum, measures related to how IT  contributes to program productivity, efficiency, effectiveness, the  overall performance of its IT programs, and the effectiveness of controls  to prevent software piracy, and track actual-versus-expected  performance for these IT performance measures; and  develop a mechanism for benchmarking the agency’s IT management  processes, when appropriate.
To improve the agency’s IT investment management processes, we  recommend that the Administrator of the Small Business Administration  take the following two actions:  document a process that the investment management board can invoke  final decision-making authority over IT investments addressed by lower- level boards; and  implement a policy requiring that proposed IT investments support  work processes that have been simplified or redesigned to reduce costs.
section_title: Social Security Administration
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Commissioner of the Social Security  Administration take the following three actions:  include in its annual performance plan the resources and time periods  required to implement the information security program plan required  by FISMA;  develop performance measures related to the performance of the  agency’s IT programs and the effectiveness of controls to prevent  software piracy; and  develop a mechanism for benchmarking the agency’s IT management  processes, when appropriate.
To improve the agency’s IT investment management processes, we  recommend that the Commissioner of the Social Security Administration  take the following four actions:  develop work processes and procedures for the agency’s investment   establish a policy requiring modularized IT investments;  document the role, responsibility, and authority of its IT investment  management board for the oversight of IT investments, such as what is  outlined in practices 2.15, 2.16, and 2.18; and  require that corrective actions be tracked and reported to the  investment management board for under-performing projects.
section_title: State
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of State take  the following two actions:  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA; and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of State take the following five actions:  implement a policy requiring that proposed IT investments support  work processes that have been simplified or redesigned to reduce costs  and improve effectiveness and that makes maximum use of COTS  software;  establish a policy requiring modularized IT investments;  include risk-adjusted return-on-investment in the department’s project   revise the department’s draft IT investment management policy to  include reviewing projects at major milestones; and   fully implement an IT investment management control phase, including  the elements contained in practices 2.16 and 2.17.
section_title: Transportation
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of  Transportation take the following five actions:  document its IT strategic planning process;   include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;  develop a goal related to how IT contributes to program productivity;  develop performance measures related to the department’s IT goals in  its IRM plan, and track actual-versus-expected performance for these IT  performance measures; and  develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Transportation take the following six  actions:  document the alignment and coordination of responsibilities of the  department’s various IT investment management boards for decision  making related to IT investments;  implement a policy requiring that proposed IT investments support  work processes that have been simplified or redesigned to reduce costs  and improve effectiveness and that makes maximum use of COTS  software;  prioritize the department’s IT proposals;  establish a policy requiring modularized IT investments;  develop and document decision-making rules to help guide the  investment management board’s oversight of IT investments during the  control phase; and   as part of the department’s control phase, employ an early warning  mechanism, and use independent verification and validation reviews,  when appropriate.
section_title: Treasury
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of the Treasury  take the following four actions:  include in the department’s annual performance plan the resources and  time periods required to implement the information security program  plan required by FISMA;   develop performance measures related to the effectiveness of controls  to prevent software piracy;  develop performance measures related to the department’s IT goals in  its IRM plan, and track actual-versus-expected performance for these IT  performance measures; and   develop a mechanism for benchmarking the department’s IT  management processes, when appropriate.
To improve the department’s IT investment management processes, we  recommend that the Secretary of the Treasury take the following eight  actions:  develop a capital planning and investment control guide that includes,  for example, the elements of practice 2.1;  develop work processes and procedures for the agency’s IT investment  management board, and document the alignment and coordination of  responsibilities of its various boards for decision making related to  investments, including the criteria for which investments—including  cross-cutting investments—will be reviewed by the enterprisewide  board;  use the department’s IT asset inventory as part of managerial decision  making, including using it to identify the potential for asset duplication;  establish a policy requiring that proposed IT investments support work  processes that have been simplified or redesigned to reduce costs and  improve effectiveness and that makes maximum use of COTS software;  implement a structured IT selection process that includes the elements  of practices 2.12 and 2.13;  establish a policy requiring modularized IT investments;  implement an IT investment management process that includes a  control phase that addresses, for example, the elements of practices  2.15, 2.16, and 2.17; and   implement an IT investment management process that includes an  evaluation phase that addresses, for example, the elements of practice  2.18.
section_title: U.S. Agency for International Development
paragraphs: To improve the agency’s IT strategic planning/performance measurement  processes, we recommend that the Administrator of the U.S. Agency for  International Development take the following two actions:  include in the agency’s annual performance plan the resources and time  periods required to implement the information security program plan  required by FISMA; and  develop a mechanism for benchmarking the agency’s IT management  processes, when appropriate.
To improve the agency’s IT investment management processes, we  recommend that the Administrator of the U.S. Agency for International  Development take the following nine actions:  develop work processes and procedures for the agency’s IT investment   establish a policy requiring that IT investments be in compliance with  the agency’s enterprise architecture;  develop a policy requiring that proposed IT investments support work  processes that have been simplified or redesigned to reduce costs and  improve effectiveness and that makes maximum use of COTS software;  include net risks, risk-adjusted return-on-investment, and qualitative  criteria in the agency’s project selection criteria;  within the agency’s IT investment selection process, implement a  mechanism to identify possible conflicting, overlapping, strategically  unlinked, or redundant proposals;  develop a policy requiring modularized IT investments;  develop decision-making rules, review projects at major milestones, and  require projects to report on deviations in system capability to help  guide the oversight of IT investments by the agency’s investment  management board during the control phase;  as part of the agency’s control phase, employ an early warning  mechanism, and use independent verification and validation reviews,  when appropriate; and  require that corrective actions be undertaken, tracked, and reported to  the investment management board for under-performing projects.
section_title: Veterans Affairs
paragraphs: To improve the department’s IT strategic planning/performance  measurement processes, we recommend that the Secretary of Veterans  Affairs take the following four actions:  include in the department’s annual performance plan the resources  required to implement the information security program plan required  by FISMA;  develop a documented process to measure progress against the  department’s IT goals, and assign roles and responsibilities for achieving  these goals;  develop performance measures related to the effectiveness of controls  to prevent software piracy; and  track actual-versus-expected performance for the department’s  enterprisewide IT performance measures in its IRM plan.
To improve the department’s IT investment management processes, we  recommend that the Secretary of Veterans Affairs take the following two  actions:  document the alignment and coordination of responsibilities of the  department’s various IT investment management boards for decision  making related to IT investments, including cross-cutting investments;  and  within the agency’s IT investment selection process, implement a  mechanism to identify possible conflicting, overlapping, strategically  unlinked, or redundant proposals, and prioritize its IT investments.
section_title: Comments from the Department of Agriculture
section_title: Comments from the Department of Commerce
section_title: Comments from the Department of Defense (including comments from the Departments of the Air Force, Army, and Navy)
paragraphs: The following are GAO’s comments on the Department of Defense’s (DOD)  letter dated December 5, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. DOD provided its annual report to the President and the Congress,  which included its fiscal year 2004 performance plan. Based on a  review of this plan, we modified our report. 2. We disagree that the cited objective fully addresses this issue.
Specifically, although this objective addresses e-government, the  wording of the objective, its description, and the discussion of related  initiatives do not explicitly address service delivery to the public.  Accordingly, we did not modify our report.  3. Our review of the acquisition management process documentation  provided by the Navy did not support that the department’s selection  criteria include net risks and risk-adjusted return-on-investment.  Accordingly, we did not modify our report.
section_title: Comments from the Department of Education
paragraphs: The following are GAO’s comments on the Department of Education’s letter  dated December 10, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. We agree that Education requires IT investments to have performance  measures. However, our practice dealt with enterprise-level measures,  such as those found in the department’s IRM plan, not project-specific  measures.  Education reported that the performance measures in its  IRM plan do not measure how IT contributes to program productivity  and the efficiency and effectiveness of agency operations. Accordingly,  we did not modify our report. 2. We modified our assessment of practice 2.6 in this report and deleted  the related recommendation based on our evaluation of additional  documentation provided by Education.
section_title: Comments from the Environmental Protection Agency
paragraphs: The following are GAO’s comments on the Environmental Protection  Agency’s (EPA) letter dated December 9, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. As we reported and EPA acknowledged, its documentation on IT  strategic planning and investment management was not complete or  finalized. For example, the partial rating we gave EPA for its IT  management and strategic planning practices—practices 1.1 and 1.2— matched the agency’s own self-assessment in these areas. Specifically,  our review of planning documents cited by EPA in its self-assessment  found that while the agency had documented agencywide roles and  responsibilities for planning and managing IT resources and had  documented its process to integrate the IT investment management  process with the budget, EPA had not addressed other key elements of  the practices. As an example, EPA had not fully documented the  method by which it defines program information needs and develops  strategies, systems, and capabilities to meet those needs. Since EPA  provided no additional documentation, our practice assessment and  our related recommendations remain unchanged. 2. As stated in our report, practice 1.7 refers to the documentation of the  process used to develop IT goals and measures and the responsibility  for achieving them. As EPA states in its comments, it is currently  working on documenting this process. Accordingly, we did not modify  our report.
section_title: Comments from the General Services Administration
paragraphs: The following are GAO’s comments on the General Services  Administration’s (GSA) letter dated December 9, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. We based our evaluation on the agency’s self-assessment and  comments made by GSA’s Director, Office of Policy and Plans.  However, based on GSA’s representation in commenting on our draft,  we changed our evaluation of the referenced practice. 2. The Clinger-Cohen Act requires agencies to include in its information  resources management (IRM) plan the identification of a major IT  acquisition program(s), or any phase or increment of that program, that  significantly deviated from cost, performance, or schedule goals  established by the program. As we acknowledge in this report,  agencies, which would include GSA, identified other mechanisms that  they use to track and report cost, schedule, and performance  deviations. Moreover, we evaluated agencies as a “partially” instead of a  “no” in this practice to take into account that the agency had the  required information, although it was not in the prescribed format.   Accordingly, we did not modify our report. 3. The Federal Information Security Management Act of 2002 requires  agencies to include in the performance plans required by the  Government Performance and Results Act the resources and time  periods to implement their information security program. As we noted  in this report, agencies, which would include GSA, commonly stated  that they had this information but that it was in another document.  Nevertheless, this does not negate the need for having the agency  report to the Congress in the form that it requires. This is particularly  important since performance plans are public documents. Accordingly,  we did not modify our report. 4. GSA’s new documentation illustrates that it has performance measures  for each of the IT goals in its IRM plan. However, GSA did not provide  evidence that it was tracking actual versus expected performance for  measures associated with one of its goals. We revised our report to  reflect GSA’s new documentation and our evaluation. 5. We revised our report on the basis of this new documentation. 6. GSA’s highest-level IT investment management board is its Executive  Committee. GSA did not provide a charter or any other evidence of  policies and procedures for this committee. We therefore did not  modify our report. 7. The additional documentation provided by GSA (1) does not address  decision-making rules and (2) illustrates that GSA uses a monthly  project control report on cost, schedule, and performance status, but  the report does not explicitly address deviations in system capability. In  addition, according to GSA’s capital planning and investment control  order, the format of the report is left to the applicable organization,  thereby making it less likely that the investment management boards  are obtaining consistent information. We therefore did not modify our  report. 8. We agree that GSA’s capital planning and investment control order  requires that projects that have significant variances are to provide “get  well” plans and that monthly control reports are used to report on  project cost, schedule, and performance status. However, it is not clear  that these status reports can be used to systemically track corrective  actions. Moreover, according to GSA’s capital planning and investment  control order, the format of the monthly control report is left to the  applicable organization, thereby making it less likely that the status of  corrective actions is being consistently reported. We therefore did not  modify our report. 9. See comment 8. 10. We modified our recommendations based on our evaluation of GSA’s  documentation. See comment 4 for our assessment. 11. Executive Order 13103 requires agencies to use software piracy  performance measures that comply with guidance issued by the federal  CIO Council. The Council, in turn, called on the agencies to develop  such measures. The additional documentation that GSA provided was  an order requiring agency employees to use properly licensed software,  but it does not include performance measures that would demonstrate  that this requirement is being honored. Measuring how well agencies  are combating software piracy is important because it can verify that  the controls that they have put in place are working. Accordingly, we  did not change this part of the recommendation. 12. We modified our recommendation to reflect that GSA requires projects  that have significant variances to develop corrective action plans.  However, the other elements of the recommendation pertaining to the  tracking and reporting on corrective actions remain outstanding. See  comment 8 for additional information.
section_title: Comments from the Department of Health and Human Services
section_title: Comments from the Department of Housing and Urban Development
section_title: Comments from the Department of the Interior
section_title: Comments from the Department of Justice
paragraphs: The following are GAO’s comments on the Department of Justice’s letter  dated December 2, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. GAO has ongoing work looking at OMB’s initiative. However, the  Federal Information Security Management Act of 2002 requires  agencies to include in the performance plans required by the  Government Performance and Results Act the resources and time  periods to implement its information security program. Accordingly, we  did not change the recommendation.
section_title: Comments from the Department of Labor
paragraphs: The following are GAO’s comments on the Department of Labor’s letter  dated December 2, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. Because Labor did not disagree with our characterization of its IT goal,  no changes were made to our report. 2. We agree with Labor’s characterization of its IT strategic goal and order  3-2003. Nevertheless, the recommendation, and related practice 1.7,  refers to the documentation of the process used to develop IT goals and  measures and the responsibility for achieving them. Labor neither  provided documentation of such a process nor took issue with our  assessment of practice 1.7, in which we stated that the agency did not  have this practice in place. Moreover, Labor’s self-assessment  referenced a draft performance measurement guidebook and quarterly  review process in support of this practice. However, these mechanisms  relate to performance measures associated with IT projects, not Labor’s  enterprisewide IT goal. Finally, as we noted in our report, unlike other  agencies in our review, Labor does not have goals in its IRM plan.  Accordingly, we did not change this recommendation.
section_title: Comments from the National Aeronautics and Space Administration
paragraphs: The following are GAO’s comments on the National Aeronautics and Space  Administration’s (NASA) letter dated December 8, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. Our practice dealt with enterprise-level measures, not project-specific  measures. In addition, although we agree that NASA’s IRM plan  included performance measures, the agency generally does not track  actual-versus-expected performance for these enterprisewide  measures.
section_title: Comments from the Nuclear Regulatory Commission
section_title: Comments from the Social Security Administration
paragraphs: The following are GAO’s comments on the Social Security Administration’s  (SSA) letter dated December 3, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. We agree that SSA needs to consider the level of detail that is  appropriate to include in its performance plans so as not to  compromise security. 2. We requested documentation to support SSA’s assertion that it has  performance measures associated with the performance of IT  programs (e.g., the percentage of IT projects that are meeting cost,  schedule, and performance goals), but none were provided.  Accordingly, we did not modify our report. 3. We agree that it is not appropriate to include measures related to the  effectiveness of controls to prevent software piracy in agency  performance plans. Neither our practice nor our recommendation  specifies the document or process that should be used to address  software piracy. 4. As we noted in this report, SSA performs benchmarking in an ad hoc  manner. We believe that taking a more systematic approach is  necessary to ensure that benchmarking is performed at suitable times  using an appropriate methodology. Without a systematic approach, it is  not possible to validate that the agency performs benchmarking “when  appropriate.” Accordingly, we did not modify our report. 5. References to OMB’s Circular A-11 in agency policy documentation  alone do not ensure that these practices are met. In particular, we  believe that agency policies related to modularized IT investments  should be explicit and that it is neither prudent nor practical to rely on  users of SSA’s documentation of its capital planning and investment  control process to review a secondary source.
section_title: Comments from the Department of State
paragraphs: The following are GAO’s comments on the Department of State’s letter  dated December 9, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. We based our evaluation on the agency’s draft Capital Planning and  Investment Control Program Guide that was provided during our  review. However, based on State’s newly finalized Capital Planning and  Investment Control Program Guide, we changed this evaluation in our  report. 2. We based our evaluation on the agency’s draft Capital Planning and  Investment Control Program Guide that was provided at the time of our  review. Based on the final version of the Capital Planning and  Investment Control Program Guide provided by State in its response,  we modified the language in our report, as appropriate. 3. See comment 2. 4. See comment 2.
section_title: Comments from the U.S. Agency for International Development
paragraphs: The following are GAO’s comments on the U.S. Agency for International  Development’s (USAID) letter dated December 9, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. References to OMB’s Circular A-11 in agency policy documentation  alone do not ensure that these practices are met. In particular, we  believe that agency policies related to practices 2.11 and 2.14 should be  explicit and that it is neither prudent nor practical to rely on users of  USAID’s directives to review a secondary source. Regarding USAID’s  comments that it uses the criteria in practices 2.11 and 2.14 as part of  its evaluation and scoring of investments, we agree that the agency  does ask some questions on the use of commercial-off-the-shelf  software and whether the agency uses “successive chunks” within its  proposed IT investment scoring model. However, addressing these  criteria as part of a scoring model does not address our practice  because scoring projects on the basis of the questions asked does not  necessarily preclude projects from continuing if they do not fully meet  the criteria. Additionally, the questions asked as part of the scoring  model do not fully meet the requirements of the practices. Accordingly,  we did not modify our report.
section_title: Comments from the Department of Veterans Affairs
paragraphs: The following are GAO’s comments on the Department of Veterans Affairs’  (VA) letter dated December 5, 2003.
subsections: section_title: GAO Comments
paragraphs: 1. VA’s response indicates that the department will address this  recommendation in the future and, therefore, we did not remove this  recommendation. 2. See comment 1. 3. See comment 1. 4. VA’s monthly performance reports track project-specific measures, not  enterprisewide IT performance measures. VA’s draft IRM plan states  that it will establish metrics to measure performance for IT strategic  initiatives. However, progress toward doing so was not addressed by  VA in its comments. Therefore, we do not believe this recommendation  has been fully addressed.  5. See comment 1. 6. Although VA describes a process followed for reviewing investment  proposals, it did not provide evidence to support that this practice was  actually followed. In addition, VA did not address the element of our  recommendation related to prioritizing its IT investments. Therefore,  we did not remove this recommendation. 7. On the basis of the additional information provided, we agree that the  recommendation has been implemented and modified our report  accordingly.
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: Joseph P. Cruz, Lester P. Diamond, Laurence P. Gill, David B. Hinchman,  Robert G. Kershaw, David F. Plocher, Susan S. Sato, and Patricia D. Slocum  made key contributions to this report.
section_title: GAO’s Mission
paragraphs: The General Accounting Office, the audit, evaluation and investigative arm of  Congress, exists to support Congress in meeting its constitutional responsibilities  and to help improve the performance and accountability of the federal government  for the American people. GAO examines the use of public funds; evaluates federal  programs and policies; and provides analyses, recommendations, and other  assistance to help Congress make informed oversight, policy, and funding  decisions. GAO’s commitment to good government is reflected in its core values of  accountability, integrity, and reliability.
section_title: Obtaining Copies of GAO Reports and Testimony
paragraphs: The fastest and easiest way to obtain copies of GAO documents at no cost is  through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older  products. The Web site features a search engine to help you locate documents  using key words and phrases. You can print these documents in their entirety,  including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and  correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site  daily. The list contains links to the full-text document files. To have GAO e-mail this  list to you every afternoon, go to www.gao.gov and select “Subscribe to   e-mail alerts” under the “Order GAO Products” heading.
subsections: section_title: Order by Mail or Phone
section_title: To Report Fraud, Waste, and Abuse in Federal Programs
section_title: Public Affairs