This is the accessible text file for GAO report number GAO-04-478T
entitled 'Information Technology Management: Improvements Needed in
Strategic Planning, Performance Measurement, and Investment Management
Governmentwide' which was released on March 3, 2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Technology, Information Policy,
Intergovernmental Relations and the Census, Committee on Government
Reform, House of Representatives:
United States General Accounting Office:
GAO:
For Release on Delivery Expected at 1:00 p.m. EST:
Wednesday, March 3, 2004:
Information Technology Management:
Improvements Needed in Strategic Planning, Performance Measurement, and
Investment Management Governmentwide:
Statement of David A. Powner, Director, Information Technology
Management Issues:
GAO-04-478T:
GAO Highlights:
Highlights of GAO-04-478T, testimony before the Subcommittee on
Technology, Information Policy, Intergovernmental Relations and the
Census, Committee on Government Reform, House of Representatives
Why GAO Did This Study:
The federal government spends billions of dollars annually on
information technology (IT) investments that are critical to the
effective implementation of major government programs. To help
agencies effectively manage their substantial IT investments, the
Congress has established a statutory framework of requirements and
roles and responsibilities relating to information and technology
management, that addresses, for example, (1) IT strategic planning/
performance measurement (which defines what an organization seeks to
accomplish, identifies the strategies it will use to achieve desired
results, and then determines how well it is succeeding in reaching
results-oriented goals and achieving objectives) and (2) IT investment
management (which involves selecting, controlling, and evaluating
investments).
GAO was asked to summarize its January 2004 report on IT strategic
planning/performance measurement and investment management
(Information Technology Management: Governmentwide Strategic Planning,
Performance Measurement, and Investment Management Can Be Further
Improved, GAO-04-49, January 12, 2004) and to discuss how agencies can
improve their performance in these areas.
What GAO Found:
GAO recently reported that the use of important IT strategic planning/
performance measurement and investment management practices by 26
major federal agencies was mixed (see figure below). For example,
agencies generally had IT strategic plans and goals, but these goals
were not always linked to specific performance measures that were
tracked. Agencies also largely had IT investment management boards,
but no agency had the practices associated with the oversight of IT
investments fully in place. Although they could not always provide an
explanation, agencies cited a variety of reasons for not having
practices fully in place, including that the chief information officer
position had been vacant and that the process was being revised. By
improving their IT strategic planning, performance measurement, and
investment management, agencies can better ensure that they are being
responsible stewards of the billions of dollars for IT that they have
been entrusted with through the wise investment of these monies.
To help agencies improve in these areas, GAO has made numerous
recommendations to agencies and issued guidance. For example, in the
January 2004 report, GAO made recommendations to the 26 agencies
regarding practices that were not fully in place. In addition, today
GAO is releasing the latest version of its Information Technology
Investment Management (ITIM) framework, which identifies critical
processes for selecting, controlling, and evaluating IT investments
and organizes them into a framework of increasingly mature stages;
thereby providing agencies a road map for improving IT investment
management processes in a systematic and organized manner.
www.gao.gov/cgi-bin/getrpt?GAO-04-478T.
To view the full product, click on the link above. For more
information, contact David Powner at (202) 512-9286 or
pownerd@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
Thank you for the opportunity to join in today's hearing on the
government's information technology (IT) management. This is a critical
topic because, according to the President's most recent budget, the
federal government spends billions of dollars annually on IT--
reportedly investing about $57 billion in fiscal year 2003.[Footnote 1]
Yet these dollars are not always managed wisely. For example, the
Administration reported that of the $60 billion in IT investments
requested for fiscal year 2005, $22 billion--representing 621 major
projects--are currently on its "Management Watch List."[Footnote 2]
This list includes mission-critical projects that need improvement in
the areas of performance measures, earned value management,[Footnote 3]
and/or IT security.
To help agencies effectively manage their substantial IT investments,
the Congress has established a statutory framework of requirements and
roles and responsibilities relating to information and technology
management through laws such as the Paperwork Reduction Act of
1995[Footnote 4] and the Clinger-Cohen Act of 1996. This framework
addresses, for example, IT strategic planning/performance measurement
(which defines what an organization seeks to accomplish, identifies the
strategies it will use to achieve desired results, and then determines
how well it is succeeding in reaching results-oriented goals and
achieving objectives), and investment management (which involves
selecting, [Footnote 5] controlling,[Footnote 6] and evaluating
[Footnote 7] investments).
At your request, today I will summarize our recently issued
report[Footnote 8] on the extent to which 26 agencies[Footnote 9] had
in place 30 important practices associated with key legislative and
other requirements for IT strategic planning/performance measurement
and IT investment management (app. I lists the 30 practices). I will
also discuss how agencies can improve their performance in these areas.
Results in Brief:
The use of important IT strategic planning/performance measurement and
investment management practices--identified based on legislation,
policy, and guidance--by the agencies in our review was mixed;
collectively the agencies had less than 50 percent of the practices
fully in place. For example, agencies generally had IT strategic plans
and goals, but these goals were not always linked to specific
performance measures that were tracked. Without enterprisewide
performance measures that are tracked against actual results, agencies
lack critical information about whether their overall IT activities are
achieving expected goals. In the investment management area, the
agencies largely had IT investment management boards, but no agency had
the practices associated with the oversight of IT investments fully in
place. Executive-level oversight of project-level management
activities provides organizations with increased assurance that each
investment will achieve the desired cost, benefit, and schedule
results. Although they could not always provide an explanation,
agencies cited a variety of reasons for not having practices fully in
place, such as that the chief information officer (CIO) position had
been vacant and that their process was being revised. Regardless of the
reason, these practices are important ingredients for ensuring
effective strategic planning, performance measurement, and investment
management, which, in turn, make it more likely that the billions of
dollars in government IT investments will not be wasted.
To help agencies improve their performance in the IT strategic
planning/performance measurement and IT investment management areas, we
made numerous recommendations to each of the 26 agencies we reviewed.
In addition, at today's hearing we are releasing the latest version of
our Information Technology Investment Management (ITIM)
framework.[Footnote 10] First issued as an exposure draft in May 2000,
this version of the ITIM includes lessons learned from our use of the
framework in our agency reviews and by users of the framework. The
framework identifies critical processes for the successful selection,
control, and evaluation of IT investments and organizes them into a
framework of increasingly mature stages. ITIM offers organizations a
road map for improving their IT investment management processes in a
systematic and organized manner.
Background:
Advances in the use of IT and the Internet are continuing to change the
way that federal agencies communicate, use, and disseminate
information; deliver services; and conduct business. For example,
electronic government (e-government) has the potential to help build
better relationships between government and the public by facilitating
timely and efficient interaction with citizens. To help agencies more
effectively manage IT, the Congress has established a statutory
framework of requirements and roles and responsibilities relating to
information and technology management. In particular, the Paperwork
Reduction Act of 1995 and the Clinger-Cohen Act of 1996 require agency
heads, acting through agency CIOs to, among other things,
* better link their IT planning and investment decisions to program
missions and goals;
* develop and maintain a strategic information resources management
(IRM) plan that describes how IRM activities help to accomplish agency
missions;
* develop and maintain an ongoing process to establish goals for
improving IRM's contribution to program productivity, efficiency, and
effectiveness; methods for measuring progress toward these goals; and
clear roles and responsibilities for achieving these goals;
* develop and implement a sound IT architecture;
* implement and enforce IT management policies, procedures, standards,
and guidelines;
* establish policies and procedures for ensuring that IT systems
provide reliable, consistent, and timely financial or program
performance data; and:
* implement and enforce applicable policies, procedures, standards, and
guidelines on privacy, security, disclosure, and information sharing.
Nevertheless, the agencies face significant challenges in effectively
planning for and managing their IT. Such challenges can be overcome
through the use of a systematic and robust management approach that
addresses critical elements such as IT strategic planning and
investment management.
Agencies Did Not Always Have Strategic Planning/Performance Measurement
and Investment Management Practices in Place:
Federal agencies did not always have in place important practices
associated with IT laws, policies, and guidance related to strategic
planning/performance measurement and investment management (see fig.
1). A well-defined strategic planning process helps to ensure that an
agency's IT goals are aligned with its strategic goals. Moreover,
establishing performance measures and monitoring actual-versus-
expected performance using those measures can help to determine whether
IT is making a difference in improving performance. Finally, an IT
investment management process is an integrated approach to managing
investments that provides for the continuous identification, selection,
control, life-cycle management, and evaluation of IT investments.
Figure 1: Percentage of Agencies' Use of 12 IT Strategic Planning/
Performance Measurement Practices (left) and 18 Investment Management
Practices (right)A:
[See PDF for image]
[A] Percentages do not add to 100 percent due to rounding.
Note: Yes--the practice was in place. Partially--the agency has some,
but not all, aspects of the practice in place. Examples of
circumstances in which the agency would receive this designation
include when (1) some, but not all, of the elements of the practice
were in place; (2) the agency documented that it has the information or
process in place but it was not in the prescribed form (e.g., in a
specific document as required by law or the Office of Management and
Budget); (3) the agency's documentation was in draft form; or (4) the
agency had a policy related to the practice, but evidence supported
that it had not been completely or consistently implemented. No--the
practice was not in place. Not applicable--the practice was not
relevant to the agency's particular circumstances.
[End of figure]
Agency IT officials could not always identify why practices were not in
place, but in those instances in which reasons were identified, a
variety of explanations were provided; for example, that the CIO
position had been vacant, that not including a requirement in the
agency's guidance was an oversight, or that the process was being
revised. Nevertheless, these practices are based on law, executive
orders, Office of Management and Budget (OMB) policies, and our
guidance, and are also important ingredients in ensuring effective
strategic planning, performance measurement, and investment management
that, in turn, make it more likely that the billions of dollars in
government IT investments will be wisely spent.
Agencies' Use of IT Strategic Planning/Performance Measurement
Practices Was Uneven:
Critical aspects of the strategic planning/performance measurement area
include documenting the agency's IT strategic planning processes,
developing IRM plans, establishing goals, and measuring performance to
evaluate whether goals are being met. Although the agencies often had
these practices, or elements of these practices, in place, additional
work remains, as demonstrated by the following examples:
* Strategic planning process. Strategic planning defines what an
organization seeks to accomplish and identifies the strategies it will
use to achieve desired results. A defined strategic planning process
allows an agency to clearly articulate its strategic direction and to
establish linkages among planning elements such as goals, objectives,
and strategies. About half of the agencies had fully documented their
strategic planning processes. Such processes are an essential
foundation for ensuring that IT resources are effectively managed.
* Strategic IRM plans. The Paperwork Reduction Act requires that
agencies indicate in strategic IRM plans how they are applying
information resources to improve the productivity, efficiency, and
effectiveness of government programs. An important element of a
strategic plan is that it presents an integrated system of high-level
decisions that are reached through a formal, visible process. The
Paperwork Reduction Act also requires agencies to develop IRM plans in
accordance with OMB's guidance. However, OMB does not provide cohesive
guidance on the specific contents of IRM plans. Accordingly, although
agencies generally provided OMB with a variety of planning documents to
meet its requirement that they submit an IRM plan, these plans were
generally limited to IT strategic or e-government issues and did not
address other elements of IRM, as defined by the Paperwork Reduction
Act. In particular, these plans generally include individual IT
projects and initiatives, security, and enterprise architecture
elements but do not often address other information functions--such as
information collection, records management, and privacy--or the
coordinated management of all information functions.
OMB IT staff agreed that the agency has not set forth guidance on the
contents of agency IRM plans in a single place, stating that its focus
has been on looking at agencies' cumulative results and not on planning
documents. These staff also noted that agencies account for their IRM
activities through multiple documents (e.g., Information Collection
Budgets[Footnote 11] and Government Paperwork Elimination Act[Footnote
12] plans). Nevertheless, half the agencies indicated a need for OMB to
provide additional guidance on the development and content of IRM
plans. Accordingly, we recommended that OMB develop and disseminate to
agencies guidance on developing IRM plans.
* IT goals. The Paperwork Reduction Act and the Clinger-Cohen Act
require agencies to establish goals that address how IT contributes to
program productivity, efficiency, effectiveness, and service delivery
to the public. We have previously reported that leading organizations
define specific goals, objectives, and measures, use a diversity of
measure types, and describe how IT outputs and outcomes impact
operational customer and agency program delivery
requirements.[Footnote 13] The agencies generally had the types of
goals outlined in the Paperwork Reduction Act and the Clinger-Cohen
Act. However, five agencies did not have one or more of the goals
required by the Paperwork Reduction Act and the Clinger-Cohen Act. It
is important that agencies specify clear goals and objectives to set
the focus and direction for IT performance.
* IT performance measures. The Paperwork Reduction Act, the Clinger-
Cohen Act, and an executive order[Footnote 14] require agencies to
establish a variety of IT performance measures--such as those related
to how IT contributes to program productivity, efficiency, and
effectiveness--and to monitor the actual-versus-expected performance
using those measures. Although the agencies largely had one or more of
the required performance measures in place, these measures were not
always linked to the agencies' enterprisewide IT goals. Moreover, few
agencies monitored actual-versus-expected performance for all of their
enterprisewide IT goals. Specifically, although some agencies tracked
actual-versus-expected outcomes for the IT performance measures in
their performance plans or accountability reports and/or for specific
IT projects, they generally did not track the performance measures that
were specified in their IRM plans. As we have previously reported, an
effective IT performance management system offers a variety of
benefits, including serving as an early warning indicator of problems
and the effectiveness of corrective actions; providing input to
resource allocation and planning; and providing periodic feedback to
employees, customers, stakeholders, and the general public about the
quality, quantity, cost, and timeliness of products and
services.[Footnote 15] Moreover, without enterprisewide performance
measures that are tracked against actual results, agencies lack
critical information about whether their overall IT activities are
achieving expected goals.
* Benchmarking. The Clinger-Cohen Act requires agencies to
quantitatively benchmark agency process performance against public-and
private-sector organizations, where comparable processes and
organizations exist. Benchmarking is used because there may be external
organizations that have more innovative or more efficient processes
than their own processes. Seven agencies in our review had mechanisms
in place--such as policies and strategies--related to benchmarking
their IT processes. In general, however, agencies' benchmarking
decisions were ad hoc. Few agencies had developed a mechanism to
identify comparable external private-or public-sector organizations
and processes and/or had policies related to benchmarking, although all
but 10 of the agencies provided examples of benchmarking that they had
performed. Our previous study of IT performance measurement at leading
organizations found that they had spent considerable time and effort
comparing their performance information with that of other
organizations.[Footnote 16]
Agency IT officials could not identify why strategic planning/
performance measurement practices were not in place in all cases, but
in those instances in which reasons were identified, a variety of
explanations were provided. For example, reasons cited by agency IT
officials included that they lacked the support from agency leadership,
that the agency had not been developing IRM plans until recently and
recognized that the plan needed further refinement, that the process
was being revised, and that requirements were evolving.
Without strong strategic management practices, it is less likely that
IT is being used to maximize improvement in mission performance.
Moreover, without enterprisewide performance measures that are being
tracked against actual results, agencies lack critical information
about whether their overall IT activities, at a governmentwide cost of
billions of dollars annually, are achieving expected goals.
Agencies' Use of IT Investment Management Practices Was Mixed:
Critical aspects of IT investment management include developing well-
supported proposals, establishing investment management boards, and
selecting and controlling IT investments. The agencies' use of
practices associated with these aspects of investment management was
wide-ranging, as follows:
* IT investment proposals. Various legislative requirements, an
executive order, and OMB policies provide minimum standards that govern
agencies' consideration of IT investments. In addition, we have issued
guidance to agencies for selecting, controlling, and evaluating IT
investments.[Footnote 17] Such processes help ensure, for example, that
investments are cost-beneficial and meet mission needs and that the
most appropriate development or acquisition approach is chosen. The
agencies in our review had mixed results when evaluated against these
various criteria. For example, the agencies almost always required that
proposed investments demonstrate that they support the agency's
business needs, are cost-beneficial, address security issues, and
consider alternatives. However, they were not as likely to have fully
in place the Clinger-Cohen Act requirement that agencies follow, to the
maximum extent practicable, a modular, or incremental, approach when
investing in IT projects. Incremental investment helps to mitigate the
risks inherent in large IT acquisitions/developments by breaking apart
a single large project into smaller, independently useful components
with known and defined relationships and dependencies.
* Investment management boards. Our investment management guide states
that establishing one or more IT investment board(s) is a key component
of the investment management process. Such executive-level boards, made
up of business-unit executives, concentrate management's attention on
assessing and managing risks and regulating the trade-offs between
continuing to fund existing operations and developing new performance
capabilities. Almost all of the agencies in our review had one or more
enterprise-level investment management board. However, the investment
management boards for six agencies were not involved, or the agency did
not document the boards' involvement, in the control phase. Maintaining
responsibility for oversight with the same body that selected the
investment is crucial to fostering a culture of accountability by
holding the investment board that initially selected an investment
responsible for its ongoing success.
* Selection of IT investments. During the selection phase of an IT
investment management process, the organization (1) selects projects
that will best support its mission needs and (2) identifies and
analyzes each project's risks and returns before committing significant
funds. To achieve desired results, it is important that agencies have a
selection process that, for example, uses selection criteria to choose
the IT investments that best support the organization's mission and
that prioritizes proposals. Twenty-two agencies used selection criteria
in choosing their IT investments. In addition, about half the agencies
used scoring models[Footnote 18] to help choose their investments.
* Control over IT investments. During the control phase of the IT
investment management process, the organization ensures that, as
projects develop and as funds are spent, the project is continuing to
meet mission needs at the expected levels of cost and risk. If the
project is not meeting expectations or if problems have arisen, steps
are quickly taken to address the deficiencies. In general, the agencies
were weaker in the practices pertaining to the control phase of the
investment management process than to the selection phase and no agency
had the practices associated with the control phase fully in place. In
particular, the agencies did not always have important mechanisms in
place for agencywide investment management boards to effectively
control investments, including decision-making rules for project
oversight, early warning mechanisms, and/or requirements that
corrective actions for under-performing projects be agreed upon and
tracked. Executive level oversight of project-level management
activities provides an organization with increased assurance that each
investment will achieve the desired cost, benefit, and schedule
results.
Among the variety of reasons that agencies cited for not having IT
investment management practices fully in place were that the CIO
position had been vacant, that not including a requirement in the IT
investment management guide was an oversight, and that the process was
being revised. However, in some cases agencies could not identify why
certain practices were not in place. It is important that agencies
address their shortcomings, because only by effectively and efficiently
managing their IT resources through a robust investment management
process can they gain opportunities to make better allocation decisions
among many investment alternatives and to further leverage their IT
investments.
Improving Agencies' IT Strategic Planning/Performance Measurement and
Investment Management:
To help agencies improve their IT strategic planning/performance
measurement and investment management, we have made numerous
recommendations to agencies and issued guidance. Specifically, in our
January 2004 report we made recommendations to the 26 agencies in our
review regarding practices that were not fully in place. These
recommendations addressed issues such as IT strategic planning;
establishing and linking enterprisewide goals and performance measures
and tracking progress against these measures; and selecting,
controlling, and evaluating investments. By implementing these
recommendations, agencies can better ensure that they are using
strategic planning, performance measurement, and investment management
practices that are consistent with IT legislation, executive orders,
OMB policies, and our guidance.
Another mechanism that agencies can use to improve their IT management
is to apply the management frameworks and guides that we have issued,
which are based on our research into IT management best practices and
our evaluations of agency IT management performance.[Footnote 19] In
this vein, today we are releasing the latest version of our ITIM
framework.[Footnote 20] This framework identifies and organizes
critical processes for selecting, controlling, and evaluating IT
investments into a framework of increasingly mature stages (see fig.
2).
Figure 2: The ITIM Stages of Maturity with Critical Processes:
[See PDF for image]
[End of figure]
First issued as an exposure draft in May 2000, this new version of the
ITIM includes lessons learned from our use of the framework in our
agency reviews and from lessons conveyed to us by users of the
framework. In addition, in order to validate the appropriateness of our
changes and to gain the advantage of their experience, we had the new
version reviewed by several outside experts who are familiar with the
ITIM exposure draft and with investment management in a broad array of
public and private organizations.
ITIM can be used to analyze an organization's investment management
processes and to determine its level of maturity. The framework is
useful to many federal agencies because it provides: (1) a rigorous,
standardized tool for internal and external evaluations of an agency's
IT investment management process; (2) a consistent and understandable
mechanism for reporting the results of these assessments to agency
executives, Congress, and other interested parties; and (3) a road map
that agencies can use for improving their investment management
processes. Regarding the first two points, we and selected agency
Inspectors General have used the ITIM to evaluate and report on the
investment management processes of several agencies.[Footnote 21]
Concerning the third point, a number of agencies have recognized the
usefulness of the ITIM framework and have used it to develop and
enhance their investment management strategies. For example, one agency
uses the framework to periodically review its IT investment management
capabilities and has developed an action plan to move through the
stages of maturity.
In summary, our January 2004 report indicates that the federal
government can significantly improve its IT strategic planning,
performance measurement, and investment management. Such improvement
would better ensure that agencies are being responsible stewards of the
billions of dollars for IT with which they have been entrusted, by
helping them to invest these monies wisely. This can be accomplished,
in part, through the expeditious implementation of our recommendations
and the adoption of best practices, which we have incorporated into our
IT management frameworks and guides such as the ITIM.
Mr. Chairman, this completes my prepared statement. I would be happy to
respond to any questions that you or other Members of the Subcommittee
may have at this time.
Contacts:
If you have any questions regarding this statement, please contact me
at (202) 512-9286 or by e-mail at pownerd@gao.gov. Specific questions
related to our January 2004 report may also be directed to Linda
Lambert at (202) 512-9556 or via e-mail at lambertl@gao.gov or Mark
Shaw at (202) 512-6251 or via e-mail at shawm@gao.gov. Questions
related to the ITIM framework can be directed to Lester Diamond at
(202) 512-7957 or via e-mail at diamondl@gao.gov.
[End of section]
Appendix I: Information Technology (IT) Strategic Planning/Performance
Measurement and Investment Management Practices:
Table 1 describes the 12 IT strategic planning/performance measurement
and the 18 IT investment management practices that we used in our
January 2004 report on the government's performance in these
areas.[Footnote 22] We identified these 30 practices after reviewing
major legislative requirements (e.g., the Paperwork Reduction Act of
1995 and the Clinger-Cohen Act of 1996), executive orders, Office of
Management and Budget policies, and our own guidance.
Table 1: IT Strategic Planning/Performance Measurement and Investment
Management Practices:
Practice Number: 1.1;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency has documented its IT strategic management process,
including, at a minimum,
* the responsibilities and accountability for IT resources across the
agency, including the relationship between the chief information
officer (CIO), chief financial officer (CFO), and mission/program
officials; and;
* the method by which the agency defines program information needs and
develops strategies, systems, and capabilities to meet those needs.
Practice Number: 1.2;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency has documented its process to integrate IT management
operations and decisions with organizational planning, budget,
financial management, human resources management, and program
decisions.
Practice Number: 1.3;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency requires that information security management processes be
integrated with strategic and operational planning processes.
Practice Number: 1.4;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency has a process that involves the CFO, or comparable
official, to develop and maintain a full and accurate accounting of IT-
related expenditures, expenses, and results.
Practice Number: 1.5;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency prepares an enterprisewide strategic information resources
management (IRM) plan that, at a minimum,
* describes how IT activities will be used to help accomplish agency
missions and operations, including related resources; and;
* identifies major IT acquisition program(s) or any phase or increment
of that program that has significantly deviated from the cost,
performance, or schedule goals established for the program.
Practice Number: 1.6;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency's performance plan required under GPRA includes;
* a description of how IT supports strategic and program goals,
* the resources and time periods required to implement the information
security program plan required by the Federal Information Security
Management Act (FISMA), and;
* a description of major IT acquisitions contained in the capital
asset plan that will bear significantly on the achievement of a
performance goal.
Practice Number: 1.7;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency has a documented process to;
* develop IT goals in support of agency needs,
* measure progress against these goals, and;
* assign roles and responsibilities for achieving these goals.
Practice Number: 1.8;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency has established goals that, at a minimum, address how IT
contributes to;
* program productivity,
* efficiency,
* effectiveness, and;
* service delivery to the public (if applicable).
Practice Number: 1.9;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency has established IT performance measures and monitors actual-
versus-expected performance that at least addresses;
* how IT contributes to program productivity,
* how IT contributes to the efficiency of agency operations,
* how IT contributes to the effectiveness of agency operations,
* service delivery to the public (if applicable),
* how electronic government initiatives enable progress toward agency
goals and statutory mandates,
* the performance of IT programs (e.g., system development and
acquisition projects), and;
* agency compliance with federal software piracy policy.
Practice Number: 1.10;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency has developed IT performance measures that align with and
support the goals in the GPRA performance plan.
Practice Number: 1.11;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency developed an annual report, included as part of its budget
submission, that describes progress in achieving goals for improving
the efficiency and effectiveness of agency operations and, as
appropriate, the delivery of services to the public through the
effective use of IT.
Practice Number: 1.12;
Practice Description: IT Strategic Planning/Performance
Measurement Practices:
The agency requires that its IT management processes be benchmarked
against appropriate processes and/ or organizations from the public
and private sectors in terms of cost, speed, productivity, and quality
of outputs and outcomes where comparable processes and organizations
in the public or private sectors exist.
Practice Number: IT Investment Management Practices:
Practice Number: 2.1;
Practice Description: IT Investment Management Practices:
The agency has a documented IT investment management process that, at
a minimum,
* specifies the roles of key people (including the CIO) and groups
within the IT investment management process,
* outlines significant events and decision points,
* identifies external and environmental factors that influence the
process,
* explains how the IT investment management process is coordinated
with other organizational plans and processes, and;
* describes the relationship between the investment management process
and the agency's enterprise architecture.
Practice Number: 2.2;
Practice Description: IT Investment Management Practices:
The agency established one or more agencywide IT investment management
boards responsible for selecting, controlling, and evaluating IT
investments that, at a minimum,
* have final project funding decision authority (or provide
recommendations) over projects within their scope of authority, and;
* are composed of key business unit executives.
Practice Number: 2.3;
Practice Description: IT Investment Management Practices:
The agencywide board(s) work processes and decision-making processes
are described and documented.
Practice Number: 2.4;
Practice Description: IT Investment Management Practices:
If more than one IT investment management board exists in the
organization (e.g., at the component level), the organization has;
* documented policies and procedures that describe the processes for
aligning and coordinating IT investment decision making,
* criteria for determining where in the organization different types
of IT investment decisions are made, and;
* processes that describe how cross-functional investments and
decisions (e.g., common applications) are handled.
Practice Number: 2.5;
Practice Description: IT Investment Management Practices:
As part of its investment management process, the agency has available
an annually updated comprehensive inventory of its major information
systems that includes major national security systems and interfaces.
Practice Number: 2.6;
Practice Description: IT Investment Management Practices:
A standard, documented procedure is used so that developing and
maintaining the inventory is a repeatable event, which produces
inventory data that are timely, sufficient, complete, and compatible.
Practice Number: 2.7;
Practice Description: IT Investment Management Practices:
The IT asset inventory is used as part of managerial decision making.
Practice Number: 2.8;
Practice Description: IT Investment Management Practices:
Proposed IT investments are required to document that they have
addressed the following items during project planning:;
* that the project supports the organization's business and mission
needs and meets users' needs,
* whether the function should be performed by the public or private
sector,
* whether the function or project should be performed or is being
performed by another agency,
* that alternatives have been considered, and;
* how security will be addressed.
Practice Number: 2.9;
Practice Description: IT Investment Management Practices:
In considering a proposed IT project, the agency requires that the
project demonstrate that it is economically beneficial through the
development of a business case that at least addresses costs,
benefits, schedule, and risks.
Practice Number: 2.10;
Practice Description: IT Investment Management Practices:
In considering a proposed IT project, the agency requires that the
project demonstrate that it is consistent with federal and agency
enterprise architectures.
Practice Number: 2.11;
Practice Description: IT Investment Management Practices:
The agency requires that the proposed IT investment, at a minimum,
* support work processes that it has simplified or redesigned to
reduce costs and improve effectiveness, and;
* make maximum use of commercial-off-the-shelf (COTS) software.
Practice Number: 2.12;
Practice Description: IT Investment Management Practices:
The agency has established project selection criteria distributed
throughout the organization that include, at a minimum,
* cost, benefit, schedule, and risk elements;
* measures such as net benefits, net risks, and risk-adjusted return
on investment; and;
* qualitative criteria for comparing and prioritizing alternative
information systems investment projects.
Practice Number: 2.13;
Practice Description: IT Investment Management Practices:
The agency has established a structured selection process that, at a
minimum,
* selects IT proposals using selection criteria;
* identifies and addresses possible IT investments and proposals that
are conflicting, overlapping, strategically unlinked, or redundant;
* prioritizes proposals; and;
* is integrated with budget, financial, and program management
decisions.
Practice Number: 2.14;
Practice Description: IT Investment Management Practices:
Agency policy calls for investments to be modularized (e.g., managed
and procured in well-defined useful segments or modules that are short
in duration and small in scope) to the maximum extent achievable.
Practice Number: 2.15;
Practice Description: IT Investment Management Practices:
The agencywide investment management board(s) has written policies and
procedures for management oversight of IT projects that cover, at a
minimum,
* decision-making rules for project oversight that allow for
terminating projects, when appropriate;
* current project data, including expected and actual cost, schedule,
and performance data, to be provided to senior management periodically
and at major milestones;
* criteria or thresholds related to deviations in cost, schedule, or
system capability actuals versus expected project performance; and;
* the generation of an action plan to address a project's problem(s)
and track resolution.
Practice Number: 2.16;
Practice Description: IT Investment Management Practices:
The agencywide investment management board(s) established an oversight
mechanism of funded investments that, at a minimum,
* determines whether mission requirements have changed;
* determines whether the investment continues to fulfill ongoing and
anticipated mission requirements;
* determines whether the investment is proceeding in a timely manner
toward agreed-upon milestones;
* employs early warning mechanisms that enable it to take corrective
action at the first sign of cost, schedule, or performance slippages;
and;
* includes the use of independent verification and validation (IV&V)
reviews of under- performing projects, where appropriate.
Practice Number: 2.17;
Practice Description: IT Investment Management Practices:
Corrective actions for under- performing projects are agreed upon,
documented, and tracked by the agencywide investment management
board(s).
Practice Number: 2.18;
Practice Description: IT Investment Management Practices:
The agencywide investment management board(s) requires that
postimplementation reviews be conducted to;
* validate expected benefits and costs and;
* document and disseminate lessons learned.
Source: GAO.
[End of table]
[End of section]
FOOTNOTES
[1] Office of Management and Budget, Budget of the U.S. Government,
Fiscal Year 2005, Report on IT Spending for the Federal Government for
Fiscal Years 2003, 2004, and 2005. We did not verify this data.
[2] Office of Management and Budget, Budget of the U.S. Government,
Fiscal Year 2005, Analytical Perspectives. We did not verify these
data.
[3] Earned value management is a project management tool that
integrates the investment scope of work with schedule and cost elements
for optimum investment planning and control.
[4] The Paperwork Reduction Act of 1995 revised the information
resources management responsibilities established under the Paperwork
Reduction Act of 1980, as amended in 1986.
[5] During the selection phase the organization (1) identifies and
analyzes each project's risks and returns before committing significant
funds to any project and (2) selects those IT projects that will best
support its mission needs.
[6] During the control phase the organization ensures that, as projects
develop and investment expenditures continue, the project is continuing
to meet mission needs at the expected levels of cost and risk. If the
project is not meeting expectations or if problems have arisen, steps
are quickly taken to address the deficiencies.
[7] During the evaluation phase, actual versus expected results are
compared once projects have been fully implemented. This is done to
(1) assess the project's impact on mission performance, (2) identify
any changes or modifications to the project that may be needed, and
(3) revise the investment management process based on lessons learned.
[8] U.S. General Accounting Office, Information Technology Management:
Governmentwide Strategic Planning, Performance Measurement, and
Investment Management Can Be Further Improved, GAO-04-49 (Washington,
D.C.: Jan. 12, 2004).
[9] We reviewed 23 entities identified in 31 U.S.C. 901 and the 3
military services. These were the Departments of Agriculture, the Air
Force, the Army, Commerce, Defense, Education, Energy, Health and Human
Services, Housing and Urban Development, the Interior, Justice, Labor,
the Navy, State, Transportation, the Treasury, and Veterans Affairs;
and the Environmental Protection Agency, General Services
Administration, National Aeronautics and Space Administration,
National Science Foundation, Nuclear Regulatory Commission, Office of
Personnel Management, Small Business Administration, Social Security
Administration, and U.S. Agency for International Development.
[10] U.S. General Accounting Office, Information Technology Investment
Management: A Framework for Assessing and Improving Process Maturity,
GAO-04-394G (Washington, D.C.: March 2004).
[11] Each year, OMB's Office of Information and Regulatory Affairs
publishes an Information Collection Budget by gathering data from
executive branch agencies on the total number of burden hours it
approved for collection of information at the end of the fiscal year
and agency estimates of the burden for the coming fiscal year.
[12] In fulfilling its responsibilities under this act, OMB requires
agencies to report to OMB on their plans for providing the public with
the option of submitting, maintaining, and disclosing required
information electronically, instead of on paper.
[13] U.S. General Accounting Office, Executive Guide: Measuring
Performance and Demonstrating Results of Information Technology
Investments, GAO/AIMD-98-89 (Washington, D.C.: March 1998).
[14] Executive Order 13103, Computer Software Piracy (September 30,
1998).
[15] GAO/AIMD-98-89.
[16] GAO/AIMD-98-89.
[17] For example, see GAO-04-394G.
[18] With a scoring model, the assessment body typically attaches
numerical scores and "relative value" weights to each of the individual
selection criteria. Investments are then assessed relative to these
scores and then against weights associated with each individual
criterion. Finally, the weighted scores are summed to create a
numerical value for each investment.
[19] For example, see U.S. General Accounting Office, Information
Technology: A Framework for Assessing and Improving Enterprise
Architecture Management (Version 1.1), GAO-03-584G (Washington, D.C.:
April 2003) and GAO/AIMD-98-89.
[20] GAO-04-394G.
[21] For example, see U.S. General Accounting Office, Information
Technology: Departmental Leadership Crucial to Success of Investment
Reforms at Interior, GAO-03-1028 (Washington, D.C.: Sept. 12, 2003);
Bureau of Land Management: Plan Needed to Sustain Progress in
Establishing IT Investment Management Capabilities, GAO-03-1025
(Washington, D.C.: Sept. 12, 2003); United States Postal Service:
Opportunities to Strengthen IT Investment Management Capabilities,
GAO-03-3 (Washington, D.C.: Oct. 15, 2002); Information Technology: DLA
Needs to Strengthen Its Investment Management Capability, GAO-02-314
(Washington, D.C.: Mar. 15, 2002); and Information Technology: INS
Needs to Strengthen Its Investment Management Capability, GAO-01-146
(Washington, D.C.: Dec. 29, 2000).
[22] U.S. General Accounting Office, Information Technology Management:
Governmentwide Strategic Planning, Performance Measurement, and
Investment Management Can Be Further Improved, GAO-04-49 (Washington,
D.C.: Jan. 12, 2004).