a

GAO

United States Government Accountability Office

Report to the Chairman, Committee on 

Finance, U.S. Senate

October 2005 INFORMATION 

TECHNOLOGY 

HHS Has Several 

Investment 

Management 

Capabilities in Place, 

but Needs to Address 

Key Weaknesses

GAO-06-11
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-06-11.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact David Powner, 

202-512-9286, pownerd@gao.gov. 

Highlights of GAO-06-11, a report to the 

Chairman, Committee on Finance, U.S. 

Senate 

October 2005

INFORMATION TECHNOLOGY 

HHS Has Several Investment 

Management Capabilities in Place, but 

Needs to Address Key Weaknesses 

Judged against the criteria of GAO’s framework for information technology 

investment management (ITIM), which measures the maturity of an 

organization’s investment management processes, HHS has established 63 

percent of the foundational practices that it needs to manage its IT 

investments individually; and 30 percent to manage its investments as a 

portfolio (see table below). Specifically, HHS has implemented processes to 

ensure that projects support business needs and meet users’ requirements, 

established a process for selecting investments, and has created portfolio 

selection criteria. However, weaknesses remain in several areas. The 

department’s senior investment board does not regularly review component 

agencies’ IT investments, leaving close to 90 percent of its discretionary 

investments without an appropriate level of executive oversight. In addition, 

HHS does not evaluate the performance of its portfolio on a continuing basis 

or conduct postimplementation reviews. Finally, HHS currently has no 

structured mechanism in place to ensure that the component agencies are 

defining and implementing investment processes that are aligned with those 

of the department. Until HHS establishes the practices it needs to effectively 

manage its IT investments, executives cannot be assured that they are 

appropriately selecting, managing, and evaluating the mix of investments 

that will maximize returns to the organization, taking into account the 

appropriate level of risk. 

HHS has initiated efforts to improve its investment management 

processes, but has not coordinated these and additional efforts that 

would be needed to address the weaknesses we identify in a 

comprehensive plan that defines and prioritizes improvements to the 

investment process. Such a plan is instrumental in helping HHS to 

coordinate and guide its improvement efforts and sustain its commitment 

to the efforts already under way. Without such a plan and procedures for 

implementing it, the department risks being unable to effectively 

establish mature investment management capabilities. As a result, 

executives may not be able to make informed and prudent investment 

decisions in managing HHS’s multibillion-dollar IT budget. 

HHS’s Current IT Investment Management Capabilities 

Stage 2: Building the 

investment foundation 

Percentage 

of key 

practices 

executed 

Stage 3: Developing a 

complete investment 

portfolio 

Percentage 

of key 

practices 

executed 

Instituting the investment 

board 

63 Defining the portfolio criteria 71 

Meeting business needs 100 Creating the portfolio 43 

Selecting an investment 70 Evaluating the portfolio 0 

Providing investment 

oversight 

0 Conducting 

postimplementation reviews 

0 

Capturing investment 

information 

83 

Overall 63 Overall 30 

Source: GAO. 

The Department of Health and 

Human Services (HHS) is one of 

the largest federal agencies, the 

nation’s largest health insurer, and 

the largest grant-making agency in 

the federal government. The 

department manages over 300 

programs that serve to improve the 

health and well-being of the 

American public. To support these 

programs, the department funds 

numerous information technology 

(IT); in fiscal year 2006, it plans to 

spend over $5 billion on IT. GAO 

was asked to evaluate HHS’s 

processes for making IT investment 

management decisions. 

Specifically, the objectives of this 

review were to (1) assess the 

department’s capabilities for 

managing its IT investments and 

(2) determine what plans, if any, 

the department might have for 

improving those capabilities. 

What GAO Recommends

To strengthen HHS’s investment 

management capability, GAO 

recommends that HHS develop and 

implement a plan to address the 

weaknesses identified in this 

report. In written comments on a 

draft of this report, HHS generally 

agreed with our findings and 

recommendations and stated that it 

will leverage the report in its 

continuing efforts to improve its 

investment management processes. 
Page i GAO-06-11 HHS’s IT Investment Management Process

Contents

Letter 1

Results in Brief 2

Background 4

HHS Has Established Many Key Practices for Managing Its 

Investments, but Has Provided Limited Guidance and Oversight to 

Component Agencies Processes 19

HHS Does Not Have a Plan to Coordinate and Guide Improvement 

Efforts 47

Conclusions 48

Recommendations for Executive Action 48

Agency Comments 50

Appendixes

Appendix I: Objectives, Scope, and Methodology 53

Appendix II: Comments from the Department of Health and Human 

Services 56

Appendix III: GAO Contact and Staff Acknowledgments 60

Tables Table 1: Estimated IT Budget of HHS Component Agencies for 

Fiscal Year 2006 7

Table 2: Stage 2 Critical Processes—Building the Investment 

Foundation 20

Table 3: Summary of Results for Stage 2 Critical Processes and Key 

Practices 21

Table 4: Instituting the Investment Board 25

Table 5: Meeting Business Needs 27

Table 6: Selecting an Investment 30

Table 7: Providing Investment Oversight 33

Table 8: Capturing Investment Information 36

Table 9: Stage 3 Critical Processes—Developing a Complete 

Investment Portfolio 37

Table 10: Summary of Results for Stage 3 Critical Processes and Key 

Practices 38

Table 11: Defining the Portfolio Criteria 40

Table 12: Creating the Portfolio 42

Table 13: Evaluating the Portfolio 44

Table 14: Conducting Postimplementation Reviews 45
Contents

Page ii GAO-06-11 HHS’s IT Investment Management Process

Figures Figure 1: Simplified HHS Organizational Chart 5

Figure 2: HHS Discretionary IT Investments for Fiscal Year 2006 6

Figure 3: Detailed Breakdown of HHS’s Investment Management 

Process 15

Figure 4: The Five ITIM Stages of Maturity with Critical 

Processes 18

Abbreviations

CPIC Capital Planning and Investment Control

CIO Chief Information Officer

HHS Department of Health and Human Services

IT information technology

ITIM information technology investment management framework

ITIRB Information Technology Investment Review Board

PMT Portfolio Management Tool

PIR postimplementation reviews

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately.
Page 1 GAO-06-11 HHS’s IT Investment Management Process

United States Government Accountability Office

Washington, D.C. 20548

A

October 28, 2005 Letr

The Honorable Charles E. Grassley

Chairman, Committee on Finance

United States Senate

Dear Mr. Chairman:

The Department of Health and Human Services (HHS) is one of the largest 

federal agencies, the nation’s largest health insurer, and the largest grantmaking agency in the federal government. The department manages over 

300 programs that serve to improve the health and well-being of the 

American public and is comprised of several component agencies covering 

a wide range of activities including conducting and sponsoring medical and 

social science research, guarding against the outbreak of infectious 

diseases, assuring the safety of food and drugs, and providing health care 

services and insurance. It also manages and funds a variety of information 

technology (IT) initiatives ranging from those facilitating the payment of 

claims for Medicare and Medicaid services to those supporting health 

surveillance and communications. In fiscal year 2006, the department plans 

to spend over $5 billion on information technology—the third largest IT 

expenditure in the federal budget.1

This report is one of two we prepared in response to your request that we 

evaluate HHS’s information technology investment management 

capabilities.2

 It focuses on HHS’s processes for making IT investment 

management decisions and evaluates how well these processes compare 

with the accepted practices presented in our IT investment management 

(ITIM) framework.3

 This framework provides a method for evaluating and 

1

Office of Management and Budget, Budget of the U.S. Government, Fiscal Year 2006, 

Report on IT Spending for the Federal Government for Fiscal Years 2004, 2005, and 2006.

We did not verify these data.

2

Our second report, GAO, Information Technology: Centers for Medicare & Medicaid 

Services Needs to Establish Critical Investment Management Capabilities, GAO-06-12

(Washington, D.C.: Oct. 28, 2005), addresses (1) the agency’s capabilities for managing its IT 

investments, (2) determining any plans the agency might have for improving these 

capabilities, and (3) examining the agency’s process for approving and monitoring the state 

Medicaid management systems it funds. 

3

GAO, Information Technology Investment Management: A Framework for Assessing and 

Improving Process Maturity, GAO-04-394G (Washington, D.C.: March 2004).
Page 2 GAO-06-11 HHS’s IT Investment Management Process

assessing how well an agency is selecting and managing its IT resources. As 

we agreed with your office, our objectives were to (1) assess the 

department’s capabilities for managing its IT investments and 

(2) determine any plans the department might have for improving those 

capabilities. To address these objectives, we analyzed documents and 

interviewed agency officials to (1) validate and update HHS’s selfassessments of key practices in the framework and (2) evaluate HHS’s 

plans for improving its capabilities. We performed our work from January 

through September 2005, in accordance with generally accepted 

government auditing standards. Appendix I contains details about our 

objectives, scope, and methodology.

Results in Brief Because of the management attention that has been given to IT investment 

management, HHS has established over half of the foundational practices 

needed to manage its IT investments individually and about 30 percent of 

the key practices needed to effectively manage its portfolio of investments. 

For example, HHS has implemented many of the practices required to 

ensure that (1) projects support business needs and meet users’ 

requirements, (2) a well-defined and disciplined process is used to select IT 

investments, (3) investment information is captured in a repository for 

decision makers, and (4) IT portfolio selection criteria are developed and 

maintained. However, critical weaknesses remain in several areas. 

Specifically, HHS lacks

• business representation on its senior IT investment review board of 

component agencies to carry out its full scope of responsibilities,

• an established process for the IT investment board to regularly review a 

defined set of the component agencies’ IT investments and maintain 

visibility of other investments,

• criteria for assessing portfolio performance or regularly reviewing the 

performance of the organization’s investment portfolio, and

• processes for conducting postimplementation reviews (PIR) of its IT 

investments.

The department also does not have a structured mechanism in place for 

ensuring that component agencies define and implement investment 

management processes that are aligned with those of the department. Until 

the department fully establishes all foundational and portfolio-level 
Page 3 GAO-06-11 HHS’s IT Investment Management Process

practices and establishes a mechanism to ensure that component agencies 

define and implement processes that are aligned with those of the 

department, executives cannot be assured that they are appropriately 

selecting, managing, and evaluating the mix of investments that will 

maximize returns to the organization, taking into account the appropriate 

level of risk. 

HHS has initiated steps to improve its investment management process; 

however, these steps do not fully address the weaknesses we identify in 

this report, nor are they coordinated along with other needed improvement 

efforts into a plan that (1) is based on an assessment of strengths and 

weaknesses; (2) specifies measurable goals, objectives, and milestones; (3) 

specifies needed resources; (4) assigns clear responsibility and 

accountability for accomplishing tasks; and (5) is approved by senior 

management. Without such a plan and procedures for implementing it, the 

department risks being unable to effectively establish mature investment 

management capabilities. As a result, executives may not be able to make 

informed and prudent investment decisions in managing the department’s 

annual multibillion-dollar IT budget.

To further strengthen HHS’s investment management capability, we are 

recommending that the department develop and implement a plan aimed at 

addressing the weaknesses that we identify in this report.

In commenting on a draft of this report, HHS generally agreed with our 

findings and recommendations and stated that it will leverage the report in 

its efforts to improve its investment management processes. However, it 

expressed differing perspectives on the inclusion of component agency 

business representation on the investment review and the performance of 

postimplementation reviews. Specifically, the department commented that 

it used a hierarchy of investment reviews combined with investment review 

board members representing mission support areas such as Finance, 

Acquisition, and Human Resources, to provide a structure for making the 

business decisions regarding the department’s investments. Nevertheless, 

we reiterate the importance of having business representation from 

component agencies to make these decisions. In addition, the department 

stated that it was performing postimplementation reviews in an informal 

manner through closeout reviews of investments that have recently been 

implemented and annual reviews of systems in operations and 

maintenance. However, neither of these reviews currently identify lessons 

learned or capture benefits realized, key elements of postimplementation 

reviews. 
Page 4 GAO-06-11 HHS’s IT Investment Management Process

Background

HHS’s Mission, 

Organizational Structure, 

and Use of IT 

HHS is the primary organization within the federal government that is 

devoted to protecting the health of Americans. It provides essential human 

services, such as ensuring food and drug safety and assisting needy 

families. HHS administers more grant dollars than all other federal 

agencies combined, providing over $200 billion of the more than $350 

billion in federal funds that were awarded to states and other entities in 

fiscal year 2002, the most recent year for which these data are available. 

For fiscal year 2005, HHS had a budget of $581 billion and a workforce of 

over 67,000 employees. 

To accomplish its mission, HHS is comprised of 12 component agencies4

and several staff offices that cover a wide range of activities—including 

conducting and sponsoring medical and social science research, guarding 

against the outbreak of infectious diseases, assuring the safety of food and 

drugs, and providing health care services and insurance. The Office of the 

Secretary consists of several staff divisions and offices, including the Office 

of the Assistant Secretary for Budget, Technology, and Finance. The HHS 

Office of the Chief Information Officer (CIO) is located within this staff 

office (see fig. 1). 

4

HHS refers to its component agencies as operating divisions.
Page 5 GAO-06-11 HHS’s IT Investment Management Process

Figure 1: Simplified HHS Organizational Chart

Component Agencies

Source: HHS.

HHS Secretary

Deputy Secretary

Administration for Children and

Families

Administration on Aging

Centers for Medicare &

Medicaid Services

Agency for Healthcare Research

and Quality

Centers for Disease Control and

Prevention

Agency for Toxic Substances and

Disease Registry

Food and Drug Administration

Health Resources and Services

Administration

Indian Health Service

Substance Abuse and Mental

Health Services Administration

Administration and Management

Budget, Technology, and Finance

Health

Legislation

Planning and Evaluation

Public Affairs

Public Health Emergency

Preparedness

Appeals Board

Civil Rights

Inspector General

General Counsel

Chief of Staff

Program Support Center

Staff Divisions and Offices

National Institutes of Health

Faith-Based and 

Community Initiatives

Global Health Affairs 

Intergovernmental Affairs & 

Secretary's Regional 

Representatives
Page 6 GAO-06-11 HHS’s IT Investment Management Process

Information technology investments play a critical role in helping HHS 

carry out its diverse mission. According to the President’s most recent 

budget, HHS expects to spend about $5 billion in IT in fiscal year 2006, 

making the department’s IT investment budget the third largest in the 

federal government. As figure 2 illustrates, approximately $3 billion is 

designated as grants to states for investments for Medicaid programs and 

other purposes, such as child support enforcement systems. Approximately 

$2 billion is for discretionary investment spending, of which 89 percent is 

used to fund IT investments for component agencies; 7 percent is invested 

in HHS enterprisewide initiatives;5

 and 4 percent is used to fund other 

initiatives, including Office of the Inspector General IT investments. 

Figure 2: HHS Discretionary IT Investments for Fiscal Year 2006 (in millions)

Table 1 provides additional information about the component agencies and 

their estimated IT budget for fiscal year 2006. 

5

Enterprisewide initiatives are mission-support and administrative systems that are used by 

all component agencies. 

Source: GAO analysis of Office of Management and Budget data.

Total: $2,312

Other - $101

4%

7%

89%

Component agency initiatives - $2,049

HHS enterprisewide initiatives - $162
Page 7 GAO-06-11 HHS’s IT Investment Management Process

Table 1: Estimated IT Budget of HHS Component Agencies for Fiscal Year 2006 

Source: GAO analysis based on Office of Management and Budget and HHS data.

a

Office of Management and Budget, Budget of the U.S. Government, Fiscal Year 2006, Report on IT 

Spending for the Federal Government for Fiscal Years 2004, 2005, and 2006. We did not verify these 

data.

b

The Agency for Toxic Substances and Disease Registry investments are included in the total for 

Centers for Disease Control and Prevention.

Component agency Mission

Estimated budget

for FY 2006

(in millions)a

Centers for Medicare & 

Medicaid Services

To administer the Medicare program and work in partnership with the states to 

administer Medicaid and the State Children’s Health Insurance Program. The agency 

also enforces health insurance portability standards and is responsible for 

implementing a number of statutory provisions that have been enacted in recent years, 

including the Medicare Prescription Drug, Improvement, and Modernization Act of 

2003.

$780

National Institutes of 

Health

To extend healthy life and reduce the burdens of illness and disability by pursuing 

fundamental knowledge about the nature and behavior of living systems and the 

application of that knowledge. 

$479

Centers for Disease 

Control and Prevention

To promote health and quality of life by preventing and controlling disease, injury, and 

disability.

$309b

Food and Drug 

Administration

To protect the public health by ensuring the safety, efficacy, and security of human and 

veterinary drugs, biological products, medical devices, the nation’s food supply, 

cosmetics, and products that emit radiation.

$194

Agency for Healthcare 

Research and Quality

To improve the quality, safety, efficiency, and effectiveness of health care for all 

Americans.

$65

Indian Health Service To raise the physical, mental, social, and spiritual health of American Indians and 

Alaska Natives.

$57

Health Resources and 

Services Administration

To provide national leadership, program resources, and services needed to improve 

access to culturally competent, quality health care.

$51

Program Support Center To provide a full range of program support services to all components of HHS and other 

federal agencies, primarily in the areas of Human Resources, Health Resources, 

Acquisition Services, Administrative Services, and Financial Management.

$44

Substance Abuse and 

Mental Health Services 

Administration

To build resilience and facilitate recovery for people with or at risk for substance abuse 

and mental illness.

$35

Administration for Children 

and Families

To administer federal programs that promote the economic and social well-being of 

families, children, individuals, and communities.

$34

Administration on Aging To promote the dignity and independence of older people, and to help society prepare 

for an aging population by serving as an advocate for older people, and by overseeing 

the development of a comprehensive and coordinated system of care that is responsive 

to the needs and preferences of older people and their family caregivers.

$2

Agency for Toxic 

Substances and Disease 

Registry

To provide health information and take public health actions in order to prevent harmful 

exposures and disease related to toxic substances.

$0b

Total $2.0 billion
Page 8 GAO-06-11 HHS’s IT Investment Management Process

HHS’ investments reflect the diversity of the department’s missions and 

operating environments. For example, HHS currently has several 

enterprisewide IT initiatives that enable stakeholders to advance the 

causes of better health, safety, and well-being for American people. These 

initiatives include:

• Unified Financial Management System, a new core financial system, to 

help management monitor budgets, conduct operations, evaluate 

program performance, and make financial and programmatic decisions. 

As a core financial system, it will interface with an estimated 110 other 

HHS information systems.6

• The Office of the Assistant Secretary for Public Health Emergency 

Preparedness maintains a command center where it can coordinate the 

response to public health emergencies from one centralized location. 

This center is equipped with satellite teleconferencing capability, 

broadband Internet hookups, and analysis and tracking software.

In addition, HHS’s component agencies have several projects and systems 

that are critical to the effective implementation of HHS’s mission, including 

the following: 

• The Food and Drug Administration’s Automated Drug Information 

Management System is to be developed as a fully electronic information 

management system that will receive, evaluate, and disseminate 

information about investigational and marketing submissions for human 

drugs and therapeutic biologics. 

• The National Institutes of Health’s major IT initiative, the Clinical 

Research Information System, is a comprehensive effort to modernize 

the systems that support clinical care and the agency’s collection of 

research data for the intramural clinical research programs. 

6

GAO, Financial Management Systems: Lack of Disciplined Processes Puts 

Implementation of HHS’ Financial System at Risk, GAO-04-1008 (Washington, D.C.: Sept. 

23, 2004).
Page 9 GAO-06-11 HHS’s IT Investment Management Process

• The Centers for Disease Control and Prevention’s major IT initiative, 

Public Health Information Network, is a national initiative to implement 

a multiorganizational business and technical architecture for public 

health information systems.7

Prior Reviews Identified 

Weaknesses in HHS’s IT 

Investment Management 

Process

In January 2004, we reported8 on a broad view of the government’s 

implementation of investment management practices at 26 major 

departments and agencies, including HHS. We also reported—and HHS 

acknowledged—that there were serious weaknesses in investment 

management. Notably, the department had not yet established selection 

criteria for project investments or a requirement that investments support 

work processes that have been simplified or redesigned. In addition, the 

department did not have decision-making rules to guide oversight of IT 

investments, review projects at major milestones, or systematically track 

corrective actions. Accordingly, we made several recommendations, 

including that HHS revise its investment management policy and require 

PIRs to address validating benefits and costs. In response to our 

recommendations, the department has been modifying several of its 

investment management policies, including its capital planning and 

investment control guidance and its governance policies. 

More recently, in June 2005, we reported9

 that the HHS IT Investment 

Review Board had conducted only budgetary reviews of the Centers for 

Disease Control and Prevention’s Public Health Information Network and 

some of its initiatives, until this past February, when HHS initiated steps for 

better monitoring of system development projects. We concluded that until 

management implements a systematic method for IT investment reviews, it 

will have difficulty minimizing risks while maximizing returns on these 

critical public health investments.

7

GAO, Information Technology: Federal Agencies Face Challenge in Implementing 

Initiatives to Improve Public Health Infrastructure, GAO-05-308 (Washington, D.C.: June 

10, 2005).

8

GAO, Information Technology Management: Governmentwide Strategic Planning, 

Performance, Measurement, and Investment Management Can Be Further Improved,

GAO-04-49 (Washington, D.C.: Jan. 12, 2004).

9

GAO-05-308.
Page 10 GAO-06-11 HHS’s IT Investment Management Process

HHS’s Approach to 

Investment Management

HHS has several groups and individuals involved in managing both the 

enterprisewide and component agency IT investments.10 They are involved 

from reviewing and approving a proposed IT project, through the process 

of budgeting for it, monitoring it through implementation, and evaluating it 

at its conclusion. The composition, roles, and responsibilities of these 

individuals and groups are described below:

Information Technology Investment Review Board (ITIRB)—Chaired by 

HHS’s CIO, this board is responsible for selecting, controlling, and 

evaluating all departmental IT investments. Members include the Deputy 

Assistant Secretary for Budget, Finance, Performance and Planning; the 

Directors for Acquisition Management Policy and Human Resources; and 

the component agency CIOs. The board is supported by an executive 

secretary who is responsible for, among other things, managing the flow of 

IT investment documentation, scheduling meetings, and assisting the 

members in preparing for their meetings. Currently, this board reviews all 

enterprisewide investments and delegates responsibilities for component 

agency investments to each individual component agencies investment 

review boards in accordance with departmental policies and procedures.

CIO Council—Also chaired by the HHS CIO and comprised of component 

agency CIOs, this board advises the HHS ITIRB on the technical soundness 

of all IT investments that require departmental review and provides 

recommendations regarding, among other things, technical aspects of 

affordability, soundness of design, risk, and compliance with architectural 

and security standards.

Critical Partners—Comprised of departmental officials from various 

functional areas, including enterprise architecture, security and privacy, 

acquisition management, finance, budget, human resources, and

e-government; this group is responsible for ensuring that most 

investments11 comply with the HHS policy in each of the functional areas 

and for advising the HHS ITIRB and individual IT investment managers on 

10We did not evaluate HHS administrative processes for managing IT grants to states 

because according to officials, both the department and component agencies CIOs are not 

directly involved in the approval or oversight of those IT investments. 

11According to HHS IT officials, for the fiscal year 2006 budget formulation, the business 

cases and Select forms were updated for investments that represented 80 percent of the 

entire HHS IT portfolio dollar value. The remaining 20 percent are nonmajor investments 

requesting less than $4.5 million in fiscal year 2006. 
Page 11 GAO-06-11 HHS’s IT Investment Management Process

issues in their areas of expertise. Each review results in a determination 

whether the investment is approved, conditionally approved, or not 

approved. A not approved result is flagged for executive review.

Business Case Quality Review Team—Comprised of component agency 

officials, this group evaluates the justifications for IT investments—both 

formal business cases and information documented in the department’s 

portfolio management tool’s Select forms—against the criteria used by the 

Office of Management and Budget’s to evaluate business cases12 agencies 

submit to the office as part of the formulation of the federal budget13 and 

provides recommendations for improving these justifications. 

Capital Planning and Investment Control (CPIC) 

Reengineering/Portfolio Management Tool (PMT) Implementation 

Team—Chaired by the Office of the CIO officials with representatives from 

the Critical Partners and the Business Case Quality Review Team, this 

group advises the board on issues regarding investment management 

policies and procedures and the implementation of the department’s 

portfolio management tool. 

Investment Managers—Responsible for managing investments in 

accordance with approved cost, schedule, and performance baselines, and 

for maintaining information on project status, control, performance, risk, 

and corrective actions. 

Process for Managing 

Investments

The department has defined a three-phase process for managing 

investments that involves selecting proposed projects and reselecting 

ongoing projects (select phase), controlling ongoing projects through 

development (control phase), and evaluating projects that have been 

deployed (evaluate phase). The department retains direct management of 

HHS enterprisewide IT investments and delegates considerable authority 

for other investments to component agencies. Specifically, the department 

selects ongoing and new component agency investments through the 

12These business cases are generally referred to as “exhibit 300s.”

13The Office of Management and Budget evaluates the business cases against the following 

10 criteria: acquisition strategy, project (investment) management, enterprise architecture, 

alternatives analysis, risk management, performance goals, security and privacy, 

performance-based management system, life-cycle costs formulation, and support the 

President’s Management Agenda.
Page 12 GAO-06-11 HHS’s IT Investment Management Process

process for selecting enterprisewide IT investments described below. 

Controlling and evaluating component agency IT investments are delegated 

to the component agencies, which are required by the department to follow 

a process similar to the one described below.

Each phase of the process for enterprisewide investments is comprised of 

multiple steps that set out requirements needed for the HHS ITIRB to make 

the decision to move forward with the project. 

The purpose of the select phase is to ensure that HHS chooses the projects 

that best support its mission and applies resources to the most important 

and valuable investments. The select phase is also intended to help the 

department justify budget requests by demonstrating sound business cases 

and project plans. To select investments, HHS has established two separate 

components—investment screening for new investment proposals and 

investment scoring and screening for ongoing investments.

During the new investment screening, the investment manager is expected 

to develop a project prospectus, which identifies a specific business need 

and preliminary, high-level system requirements. A high-level 

determination of resource and schedule requirements is also to be 

conducted as part of the business need identification activities. Approval of 

the project prospectus by the HHS ITIRB signifies that the agency agrees 

that the need is critical enough to proceed to the next step in which the 

business case is developed. During business case development, the 

investment manager is required to develop the business case, which 

establishes the lifecycle cost, schedule, benefits, and performance 

baselines and includes an analysis for each investment to identify 

alternatives that may satisfy the needs of the department. In addition, the 

investment managers sign a document called the accountability agreement 

form to accept responsibility for reporting on the project status in 

achieving performance baselines throughout the remaining phases of the 

investment management process.
Page 13 GAO-06-11 HHS’s IT Investment Management Process

After the project is initially approved by the HHS ITIRB, the business cases 

and Select forms for most IT investments are updated annually as part of 

the budget formulation process. (The Select forms are a collection of forms 

with HHS’s portfolio management tool that capture investment data to 

justify funding and ensure adequate project planning during the select 

phase.) The first step within the annual budget formulation process 

requires that all component agencies use the Select forms to report the 

project cost estimates that best represent the level of funding required to 

meet program or business needs. At this point, the Critical Partners and the 

Business Case Quality Review Team score and rank the Select forms using 

the department’s portfolio management tool14 to create a single HHS 

portfolio as well as component agency portfolios to provide 

recommendations to the component agencies for making final adjustments 

to their portfolio ranking. 

Once the component agencies have made the appropriate changes, the 

Office of the CIO develops prioritized IT portfolios for HHS as a whole as 

well as each component agency to present to the HHS ITIRB. The 

departmental board and CIO Council review and comment on the 

prioritized portfolio and submit it to the Secretary’s Budget Council for 

input into their budget deliberations. The Secretary’s Budget Council then 

makes recommendations to the Secretary regarding HHS and component 

agencies’ budgets. Finally, the department submits its approved Secretary’s 

IT budget to the Office of Management and Budget for inclusion in the 

President’s Budget.

Once selected for inclusion in the department’s IT portfolio, each project is 

to be managed by an investment manager and reviewed by the ITIRB on a 

quarterly basis throughout the end of development. The board performs 

reviews of projects that deviate from predetermined budget, schedule, or 

performance milestones established in the business case and works with 

the investment managers to develop a correction action plan. The ITIRB 

must also decide whether to continue to fund the project; rebaseline the 

scope, schedule, or budget; or to terminate the project. 

Once a project has been fully implemented, the HHS ITIRB is to conduct 

annual reviews of all HHS enterprisewide steady state investments—that 

is, investments in operations and maintenance—to determine whether they 

14The department’s portfolio management tool was implemented in May 2004 and has not 

been used yet to support the entire investment management process.
Page 14 GAO-06-11 HHS’s IT Investment Management Process

continue to meet the business needs. In addition, investments that have 

recently completed implementation or a significant phase are to undergo 

PIRs to evaluate actual development events against project management 

plans and to identify lessons learned that can be applied to current and 

future investments. 

Figure 3 illustrates HHS’s investment management process phases and 

steps. The highlighted steps represent the activities that the department 

conducts for both enterprisewide and component agency investments. 
Page 15 GAO-06-11 HHS’s IT Investment Management Process

Figure 3: Detailed Breakdown of HHS’s Investment Management Process

These steps represent the activities that HHS conducts for

both enterprisewide and component agency IT investments.

These steps represent the activities that HHS conducts for

the enterprisewide IT investments and delegates to

component agencies.

Select Phase Control Phase Evaluate Phase

Source: GAO analysis based on HHS documents.

Steady State Reviews

Postimplementation Reviews

ITIRB decision

(termination, when

appropriate)

ITIRB decision

(reassessment, 

annually)

Systems Development and

Testing

ITIRB decision

(rollout or not)

Business Need (Project Prospectus Development)

ITIRB decision

(to be or not

to be)

Requirements Management (Business Case Development)

Step 1 (Submission to HHS Office of the CIO and Review Groups)

Step 2 (Submission to HHS ITIRB)

Step 3 (Submission to Office of Management and Budget)

Finalize IT investment data for Office of Management and Budget submittal.

ITIRB decision

(to approve

or not)

New Investment Screening

Ongoing process of evaluating and funding new investments concepts Investment Scoring and Selection Annual process of prioritizing and selecting ongoing investments for funding

Includes planning for training, 

configuration management, 

quality assurance, risk 

management, and other 

collateral tasks associated with 

the project. A technical strategy 

and the associated plan is 

developed since the project is 

mature enough at this point for 

detailed design. Also during this 

stage, quarterly control reviews 

are conducted to assess the 

project's progress against 

baselines.Typically the project 

management plan is revisited. Performed when projects are 

completed to evaluate how well 

the project was executed.

Annually conducted as part of 

the budget formulation process 

to evaluate whether 

investments in the operational 

phase continue to warrant 

funding.

A business case containing a complete description of the project is created 

and justified. Performance measures are established to track the success of 

the project. The cost/benefit analysis is completed to identify alternatives to 

consider to implement the proposed development or change request. A 

high-level project management plan is created to support the business case by 

providing such information as expected acquisition strategy, alternatives 

analysis, and estimated project milestones.

All projects must be reviewed and prioritized for funding. The investment review 

board must evaluate each project concept based on the project's impact and 

the probable benefit to the department. The review will result in a rating for 

each system and then used to prioritize candidate IT investments. Specifically, 

component agencies submit their draft IT portfolio to HHS Office of the CIO 

that are to be included for the annual budget formulation process.The Business 

Case Quality Review Team and the Critical Partners are expected to score and 

rank both the department and component agencies’ IT investments using the 

portfolio management tool and provide recommendations to the component 

agencies to make final adjustments to their portfolio ranking. Based on these 

reviews, the Office of the CIO develops a prioritized HHS IT portfolio to present 

to the HHS ITIRB for review.

The final prioritized HHS IT portfolio is presented to the HHS ITIRB to make 

recommendations to the Secretary's Budget Council regarding component 

agencies' IT budgets. Each component agency's IT portfolio is displayed in 

priority order, showing where each investment falls within the overall IT 

portfolio.

A specific business need and preliminary, high-level systems requirements are 

identified and general scope of the project is established.
Page 16 GAO-06-11 HHS’s IT Investment Management Process

ITIM Maturity Framework The ITIM framework is a maturity model composed of five progressive 

stages of maturity that an agency can achieve in its investment 

management capabilities.15 It was developed on the basis of our research 

into the IT investment management practices of leading private- and publicsector organizations. In each of the five stages, the framework identifies 

critical processes for making successful IT investments. The maturity 

stages are cumulative; that is, in order to attain a higher stage the agency 

must have institutionalized all of the critical processes at the lower stages, 

in addition to the higher stage critical processes.

The framework can be used to assess the maturity of an agency’s 

investment management processes and as a tool for organizational 

improvement. The overriding purpose of the framework is to encourage 

investment processes that increase business value and mission 

performance, reduce risk, and increase accountability and transparency in 

the decision process. We have used the framework in several of our 

evaluations,16 and a number of agencies have adopted it. These agencies 

have used ITIM for purposes ranging from self-assessment to redesign of 

their IT investment management processes.

ITIM’s five maturity stages represent steps toward achieving stable and 

mature processes for managing IT investments. Each stage builds on the 

lower stages; the successful attainment of each stage leads to improvement 

in the organization’s ability to manage its investments. With the exception 

of the first stage, each maturity stage is composed of “critical processes” 

that must be implemented and institutionalized in order for the 

organization to achieve that stage. These critical processes are further 

broken down into key practices that describe the types of activities that an 

organization should be performing to successfully implement each critical 

process. It is not unusual for an organization to be performing key 

15GAO-04-394G.

16GAO, Information Technology: DLA Needs to Strengthen Its Investment Management 

Capability, GAO-02-314 (Washington, D.C.: Mar. 15, 2002); GAO, United States Postal 

Service: Opportunities to Strengthen IT Investment Management Capabilities, GAO-03-3

(Washington, D.C.: Oct. 15, 2002); GAO, Information Technology: Departmental Leadership 

Crucial to Success of Investment Reforms at Interior, GAO-03-1028 (Washington, D.C.: 

Sept. 12, 2003); GAO, Bureau of Land Management: Plan Needed to Sustain Progress in 

Establishing IT Investment Management Capabilities, GAO-03-1025 (Washington, D.C.: 

Sept. 12, 2003); and GAO, Information Technology: FAA Has Many Investment 

Management Capabilities in Place, but More Oversight of Operational Systems Is Needed, 

GAO-04-822 (Washington, D.C.: Aug. 20, 2004).
Page 17 GAO-06-11 HHS’s IT Investment Management Process

practices from more than one maturity stage at the same time, but efforts 

to improve investment management capabilities should focus on 

implementing all lower stage practices before addressing higher stage 

practices.

In the ITIM framework, Stage 2 critical processes lay the foundation for 

sound IT investment processes by helping the agency to attain successful, 

predictable, and repeatable investment control processes at the project 

level. Specifically, Stage 2 encompasses building a sound investment 

management foundation by establishing basic capabilities for selecting new 

IT projects. It also involves developing the capability to control projects so 

that they finish predictably within established cost and schedule 

expectations and the capability to identify potential exposures to risk and 

put in place strategies to mitigate that risk. The basic selection processes 

established in Stage 2 lays the foundation for more mature selection 

capabilities in Stage 3, which represents a major step forward in maturity, 

in which the agency moves from project-centric processes to a portfolio 

approach, evaluating potential investments by how well they support the 

agency’s missions, strategies, and goals.

Stage 3 requires that an organization continually assess both proposed and 

ongoing projects as parts of a complete investment portfolio—an 

integrated and competing set of investment options. It focuses on 

establishing a consistent, well-defined perspective on the IT investment 

portfolio and maintaining mature, integrated selection (and reselection), 

control, and evaluation processes, which are to be evaluated during PIRs. 

This portfolio perspective allows decision makers to consider the 

interaction among investments and the contributions to organizational 

mission goals and strategies that could be made by alternative portfolio 

selections, rather than to focus exclusively on the balance between the 

costs and benefits of individual investments.

Stages 4 and 5 require the use of evaluation techniques to continuously 

improve both the investment portfolio and the investment processes in 

order to better achieve strategic outcomes. At Stage 4 maturity, an 

organization has the capacity to conduct IT succession activities and, 

therefore, can plan and implement the deselection of obsolete, high-risk, or 

low-value IT investments. An organization with Stage 5 maturity conducts 

proactive monitoring for breakthrough information technologies that will 

enable it to change and improve its business performance. Organizations 

implementing Stages 2 and 3 have in place the selection, control, and
Page 18 GAO-06-11 HHS’s IT Investment Management Process

evaluation processes that are required by the Clinger-Cohen Act of 1996.17

Stages 4 and 5 define key attributes that are associated with the most 

capable organizations.

Figure 4 shows the five ITIM stages of maturity and the critical processes 

associated with each stage.

Figure 4: The Five ITIM Stages of Maturity with Critical Processes

As defined by the model, each critical process consists of “key practices” 

that must be executed to implement the critical process.

17The Clinger-Cohen Act of 1996, 40 U.S.C. §§ 11311-11313.

Source: GAO. 

- Optimizing the investment process 

- Using IT to drive strategic business change

- Improving the portfolio's performance 

- Managing the succession of information systems

- Defining the portfolio criteria 

- Creating the portfolio 

- Evaluating the portfolio 

- Conducting postimplementation reviews

- Instituting the investment board

- Meeting business needs

- Selecting an investment

- Providing investment oversight

- Capturing investment information

Stage 5: Leveraging IT for 

 strategic outcomes

Maturity stages Critical processes

Stage 4: Improving the

 investment process

Stage 3: Developing a complete

 investment portfolio

Stage 2: Building the investment

 foundation

Stage 1: Creating investment awareness - IT spending without disciplined investment processes
Page 19 GAO-06-11 HHS’s IT Investment Management Process

HHS Has Established 

Many Key Practices for 

Managing Its 

Investments, but Has 

Provided Limited 

Guidance and 

Oversight to 

Component Agencies 

Processes

In order to have the capabilities to effectively manage IT investments, an 

agency, at a minimum, should, (1) build an investment foundation by 

putting basic, project-level control and selection practices in place (Stage 2 

capabilities) and (2) manage its projects as a portfolio of investments, 

treating them as an integrated package of competing investment options 

and pursuing those that best meet the strategic goals, objectives, and 

mission of the agency (Stage 3 capabilities). These practices may be 

executed at various organizational levels of the agency, including at the 

component level. However, overall responsibility for their success remains 

at the department level. Therefore, at a minimum, the department should 

effectively oversee component agencies’ IT investment management 

processes.

HHS has executed 24 of the 38 key practices that the ITIM framework 

requires to build a foundation for IT investment management (Stage 2) and 

8 of the 27 key practices required to manage investments as a portfolio 

(Stage 3). However, the department has only provided limited oversight of 

component agencies’ ITIM processes. Until HHS implements and oversees 

a stable investment management process throughout the department, it 

will lack essential management controls over all of its IT investments, and 

it will be unable to ensure that it is appropriately selecting, managing, and 

evaluating the mix of investments that will maximize returns to the 

organization, taking into account the appropriate level of risk. 

HHS Has Established Over 

Half of the Foundational 

Practices Needed to Manage 

Its Investments

At the ITIM Stage 2 level of maturity, an organization has attained 

repeatable, successful IT project-level investment control processes and 

basic selection processes. Through these processes, the organization can 

identify expectation gaps early and take the appropriate steps to address 

them. According to the ITIM, critical processes at Stage 2 include 

(1) defining IT investment board18 operations, (2) identifying the business 

needs for each IT investment, (3) developing a basic process for selecting 

new IT proposals and reselecting ongoing investments, (4) developing 

project-level investment control processes, and (5) collecting information 

18An IT investment board is a decision-making body, made up of senior program, financial, 

and information managers, that is responsible for making decisions about IT projects and 

systems on the basis of comparisons and trade-offs among competing projects, with an 

emphasis on meeting mission goals.
Page 20 GAO-06-11 HHS’s IT Investment Management Process

about existing investments to inform investment management decisions. 

Table 2 describes the purpose of each of these Stage 2 critical processes.

Table 2: Stage 2 Critical Processes—Building the Investment Foundation

Source: GAO.

In the federal government, the agency head and the CIO are responsible for 

effectively managing information technology.19 The agency head, through 

the department-level CIO, is responsible for providing leadership and 

oversight for foundational critical processes by ensuring that written 

policies and procedures are established, repositories of information are 

created that support investment decision making, resources are allocated, 

responsibilities are assigned, and all the activities are properly carried out 

where they may be most effectively executed. In a large and diverse 

organization such as HHS, it is especially critical that the CIO create this 

structure and framework to ensure that the organization is effectively 

managing its investments at every level. This means that the CIO must 

ensure that component agencies have investment management processes 

in place that adequately support the department’s investment management 

process to make certain that funds are being expended on component 

agency investments that will fulfill mission needs.

Critical process Purpose

Instituting the investment board To define and establish an appropriate IT investment management structure and the processes 

for selecting, controlling, and evaluating IT investments. 

Meeting business needs To ensure that IT projects and systems support the organization’s business needs and meet 

users’ needs.

Selecting an investment To ensure that a well-defined and disciplined process is used to select new IT proposals and 

reselect ongoing investments.

Providing investment oversight To review the progress of IT projects and systems, using predefined criteria and checkpoints, in 

meeting cost, schedule, risk, and benefit expectations and to take corrective action when these 

expectations are not being met.

Capturing investment information To make available to decision makers information to evaluate the impacts and opportunities 

created by proposed (or continuing) IT investments.

1940 U.S.C. § 11312(b)(1).
Page 21 GAO-06-11 HHS’s IT Investment Management Process

Because of the management attention that has been given to IT investment 

management, the department has put in place over half of the key practices 

needed to establish the investment foundation. The department has 

satisfied all of the key practices associated with ensuring that projects and 

systems support organizational needs and meet users’ needs. It has 

satisfied most of the key practices associated with identifying and 

collecting investment information, selecting new proposals20 and 

reselecting ongoing investments, and instituting the department’s 

investment review board. 

However, because of its limited involvement in overseeing component 

agency investments, the department has not executed any of the key 

practices related to providing investment oversight.

Table 3 summarizes the status of HHS’s critical processes for Stage 2 and 

shows how many key practices HHS has executed in managing its IT 

investments.

Table 3: Summary of Results for Stage 2 Critical Processes and Key Practices 

Source: GAO.

20According to the ITIM, “new” proposals include both (1) previously submitted IT proposals 

that were not originally selected for funding and (2) IT proposals that have never been 

submitted.

Critical process

Key practices

executed

Total required

by critical

process

Percentage of key

practices executed

Instituting the investment 

board 5 8 63

Meeting business needs 7 7 100

Selecting an investment 7 10 70

Providing investment 

oversight 0 7 0

Capturing investment 

information 5 6 83

Total 24 38 63
Page 22 GAO-06-11 HHS’s IT Investment Management Process

HHS Has Established an 

Investment Review Board, 

but It Is Operating without a 

Comprehensive Process 

Guide 

The establishment of decision-making bodies or boards is a key component 

of the IT investment management process. At the Stage 2 level of maturity, 

organizations define one or more boards, provide resources to support the 

boards’ operations, and appoint members who have expertise in both 

operational and technical aspects of proposed investments. The boards 

should operate according to a written IT investment process guide that is 

tailored to the organization’s unique characteristics, thus ensuring that 

consistent and effective management practices are implemented across the 

organization.21 The organization selects board members to ensure that they 

are knowledgeable about policies and procedures for managing 

investments. Organizations at the Stage 2 level of maturity also take steps 

to ensure that executives and line managers support and carry out the 

decisions of the investment board. According to the ITIM, organizations 

should (1) use an investment management guide as an authoritative 

document to initiate and manage investment processes and (2) provide a 

comprehensive foundation for the policies and procedures that are 

developed for all of the other related processes. (The complete list of key 

practices is provided in table 4.)

The department has executed 5 of the 8 key practices for this critical 

process. The department established an IT investment review board as its 

corporate-level investment board that consists of senior officials, including 

the CIO and the Deputy Assistant Secretaries for Budget, Finance, and 

Performance & Planning. The board is adequately resourced, with most 

support being provided by the Office of the CIO, whose responsibilities 

include developing and modifying the department’s criteria for selecting, 

controlling, and evaluating potential and existing IT investments. In 

addition, the CIO Council reviews the enterprisewide investments for 

technical soundness and provides its recommendations to the board. The 

Critical Partners and Business Case Quality Review Team provide 

additional support to the board by reviewing and scoring most of their IT 

investments. 

21According to the ITIM, a process is a sequence of steps performed for a given purpose, and 

a process guide is a document that specifically defines the manner in which the general IT 

investment guidance will be implemented within the organization.
Page 23 GAO-06-11 HHS’s IT Investment Management Process

To ensure that the board’s decisions are carried out for enterprisewide 

investments, the ITIRB approves an accountability agreement document 

and business case that identify the benefits, costs, and schedule for the 

approved investments. The board then monitors the investments through 

the end of development. HHS requires the component agencies to follow a 

similar process in accordance with departmental policies and procedures. 

We verified that an accountability agreement document was signed and the 

business case identified performance expectations for the two 

enterprisewide IT investments we reviewed—Public Key Infrastructure 

and Enterprise Architecture initiatives.22 Additionally, the board has 

oversight of the development and maintenance of the documented IT 

investment process through the CPIC Reengineering/PMT Implementation 

Team, who provides investment management policy change 

recommendations to the board for approval. 

Although HHS has implemented these key practices, it does not have a 

comprehensive organization-specific process guide to direct the operations 

of the investment board. While the Information Resources Management

policy, guidelines, and standard operating procedures provide general 

guidance on the organization’s investment management process, they do 

not reflect the current investment management process. Moreover, they do 

not constitute an IT investment process guide because they do not 

sufficiently define the investment process. Specifically, the policies and 

procedures do not include information on the roles of the key players such 

as the CIO Council, Critical Partners, Business Case Quality Review Team, 

or the component agency investment review boards. In addition, they do 

not identify the manner in which investment board’s processes are to be 

coordinated with other key organizational plans and processes (such as the 

budget formulation process). HHS has recently drafted a revised 

investment management policy addressing many of these weaknesses; 

however, it has not been finalized, and HHS officials could not provide a 

final issuance date. Without a comprehensive investment management 

process guide, the department lacks the assurance that IT investment 

activities will be coordinated and performed in a consistent and costeffective manner.

22We reviewed two enterprisewide projects—HHS Public Key Infrastructure and HHS 

Enterprise Architecture initiative, and two component agency projects—National Institutes 

of Health’s Electronic Research Administration and Food and Drug Administration’s Mission 

Accomplishment and Regulatory Compliance Services. The projects are described in 

appendix I. 
Page 24 GAO-06-11 HHS’s IT Investment Management Process

Moreover, while HHS has established an IT investment board, the board 

does not have business representation (that is, mission representation) 

from component agencies. Instead, Chief Information Officers represent 

the component agencies. According to HHS’s CIO, the membership of the 

board is adequate for carrying out the investment activities it currently 

performs—primarily focusing on enterprisewide IT investments. However, 

because allocating resources among major IT investments may require 

fundamental trade-offs among a multitude of business objectives, portfolio 

management decisions are essentially business decisions, and therefore 

require sufficient business representation on the board. Until the 

department adjusts its board membership to include business 

representation from component agencies, it will not have assurance that it 

includes those executives who are in the best position to make the full 

range of decisions needed to enable the agency to meet its mission most 

effectively, particularly as it begins to execute its full range of 

responsibility.

Finally, the HHS ITIRB is not operating according to its assigned authority 

and responsibility. The department’s investment management policy and 

the HHS ITIRB’s charter state that the board has oversight responsibility 

for both enterprisewide and a defined set of component agency IT 

investments, including projects that are high risk, crosscutting, and require 

review by the Office of Management and Budget. However, the board 

currently oversees only enterprisewide IT investments. According to HHS 

officials, the department has delegated authority to the component 

agencies to conduct investment reviews; however, the board does not have 

a mechanism in place for ensuring that component agencies are conducting 

such reviews in accordance with department policies and procedures. Until 

the board operates according to its assigned authority, it cannot ensure that 

component agency investments are properly aligned with the organization’s 

objectives or reviewed by the appropriate board.

Table 4 shows the rating for each key practice required to institute the 

investment board. Each of the “executed” ratings shown below represents 

instances where, on the basis of the evidence provided by HHS officials, we 

concluded that the specific key practices were executed by the 

organization.
Page 25 GAO-06-11 HHS’s IT Investment Management Process

Table 4: Instituting the Investment Board

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitments

1. An enterprisewide IT investment 

board composed of senior executives 

from IT and business units is responsible 

for defining and implementing the 

organization’s IT investment governance 

process. 

Not 

executed

Although HHS has an enterprisewide IT investment board that 

is responsible for defining and implementing the organization's 

IT investment governance process and consists of the 

department's senior executives from IT and other supporting 

units, including the CIO, Deputy Assistant Secretaries for 

Budget, Finance, Performance & Planning, and the component 

agencies' CIO, the board does not have business 

representation from component agencies. 

2. The organization has a documented 

IT investment process directing each 

investment board’s operations.

Not 

executed

Although the Information Resources Management policy, 

guidelines, and standard operating procedures provide general 

guidance on the department's investment management 

process, these policies and procedures do not reflect the 

department's current investment management process. In 

addition, these documents do not constitute an investment 

management process guide in that they do not (1) include 

information on the roles of key working groups involved in the 

organization's IT investment processes or (2) identify the 

manner in which investment board's processes are to be 

coordinated with other key organizational plans and processes 

(such as the budget formulation process) or component agency 

investment management processes. HHS is currently revising 

its documented IT investment process to reflect its current 

investment management practices. 

Prerequisites 1. Adequate resources, including people, 

funding, and tools, are provided for 

supporting the operations of each IT 

investment board.

Executed Adequate resources are provided to support the ITIRB's 

operations. The executive secretariat provides operations 

support such as scheduling meetings and managing the flow of 

IT investment documentation. The CIO Council performs 

technical reviews of enterprisewide IT investments and 

provides recommendations to the ITIRB. The Critical Partners 

rank and score most IT investments from a functional 

perspective, while the Business Case Quality Review Team 

ranks and scores these investments against the Office of 

Management and Budget Exhibit 300 quality criteria.

2. The board members understand the 

organization’s IT investment 

management policies and procedures 

and the tools and techniques used in the 

board’s decision-making process.

Executed HHS ITIRB members understand the investment board's 

policies and procedures and the tools and techniques used in 

the board's decision-making process. High-level training has 

been provided to members during past board meetings on an 

informal basis. 

3. Each board’s span of authority and 

responsibility is defined to minimize 

overlaps or gaps among the boards.

Executed HHS’ investment board, the ITIRB, is responsible for defining 

and implementing the organization's IT investment governance 

process.
Page 26 GAO-06-11 HHS’s IT Investment Management Process

Source: GAO.

HHS Has a Process for Ensuring 

That Its Investments Support 

Business Needs and Meet Users’ 

Needs

Defining business needs for each IT project helps to ensure that projects 

and systems support an organization’s business needs and meet users’ 

needs. This critical process ensures that an organization’s business 

objectives and its IT management strategy are linked. According to the 

ITIM, effectively meeting business needs requires, among other things, 

(1) documenting business needs with stated goals and objectives; 

(2) identifying specific users and other beneficiaries of IT projects and 

systems; (3) providing adequate resources to ensure that projects and 

systems support the organization’s business needs and meet users’ needs; 

and (4) periodically evaluating the alignment of IT projects and systems 

with the organization’s strategic goals and objectives. (The complete list of 

key practices is provided in table 5.)

The department has in place all of the key practices for meeting business 

needs. Specifically, HHS has policy and procedures that call for business 

needs to be identified in the business case or the portfolio management 

tool’s Select forms for both proposed and ongoing enterprisewide and 

component agency IT projects. Resources devoted to ensuring that IT 

projects and systems support the organization’s business needs and meet 

users’ needs include the Business Case Quality Review Team, the Critical 

Partners, the portfolio management tool, and detailed procedures and 

associated templates for developing business cases. HHS’s specific 

Activities 1. The enterprisewide investment board 

has oversight responsibilities for the 

development and maintenance of the 

organization's documented IT 

investment process.

Executed While the HHS ITIRB does not directly oversee the 

development and maintenance of HHS’s documented 

investment process, it is involved in this process through the 

CPIC Reengineering/PMT Implementation Team, who provides 

investment management policy change recommendations to 

the HHS ITIRB for approval.

2. Each investment board operates in 

accordance with its assigned authority 

and responsibility.

Not 

executed

While, the HHS ITIRB’s charter assigns the board authority and 

responsibility for reviewing both the enterprisewide and a 

defined set of component agency IT investments, the board 

primarily focuses on enterprisewide IT investments.

3. The organization has established 

management controls for ensuring that 

investment boards’ decisions are carried 

out.

Executed HHS ITIRB has established management controls such as the 

accountability agreement document for ensuring that the 

board’s decisions regarding the enterprisewide IT investments, 

which it directly reviews, are carried out. 

For the two enterprisewide projects we reviewed, we verified 

that management controls were established through the 

accountability agreement document and business cases.

(Continued From Previous Page)

Type of 

practice Key practice Rating Summary of evidence
Page 27 GAO-06-11 HHS’s IT Investment Management Process

business mission, with stated goals and objectives, is defined in the HHS 

Strategic Plan for fiscal years 2004 through 2009. 

Further, HHS defines and documents business needs for both proposed and 

ongoing enterprisewide and component agency IT projects, and identifies 

users and other beneficiaries during its selection activities. In addition, 

according to HHS IT officials, end users participate in project management 

throughout the IT project’s life cycle. For the four projects we reviewed, we 

verified that business needs and specific users and other beneficiaries were 

identified and documented in the business case or in the Select forms 

within HHS’s portfolio management tool. In addition, end users are 

involved in project management throughout the life cycle of the 

enterprisewide investments. For example, users of HHS’s Public Key 

Infrastructure and Enterprise Architecture initiatives participate in project 

management through integrated project teams, which meet approximately 

once a month and are comprised of representatives from the component 

agencies. Because the department has executed all of the key practices 

associated with identifying business needs, it has increased confidence that 

its IT projects will meet both business needs and users’ needs.

Table 5 shows the rating for each key practice required to meet business 

needs and summarizes the evidence that supports these ratings.

Table 5: Meeting Business Needs

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitments

1. The organization has documented 

policies and procedures for identifying IT 

projects or systems that support the 

organization’s ongoing and future 

business needs.

Executed HHS has policies and procedures for ensuring that IT projects and 

systems support the department’s ongoing and future business 

needs.

Prerequisites 1. The organization has a documented 

business mission with stated goals and 

objectives.

Executed The HHS Strategic Plan for fiscal years 2004 through 2009 

defines the agency's mission goals and objectives. 

2. Adequate resources, including people, 

funding, and tools, are provided for 

ensuring that IT projects and systems 

support the organization’s business 

needs and meet users’ needs.

Executed HHS has adequate resources for ensuring that its IT projects and 

systems support the organization's business needs and meet 

users' needs. They include Business Case Quality Review Team, 

Critical Partners, and the portfolio management tool. Also, HHS 

has templates for developing business cases and training 

manuals on the use of the portfolio management tool.
Page 28 GAO-06-11 HHS’s IT Investment Management Process

Source: GAO.

HHS Is Selecting New 

Investments and Reselecting 

Ongoing Investments, but Lacks 

a Fully Documented Process for 

Doing So

Selecting new IT proposals and reselecting ongoing investments require a 

well-defined and disciplined process to provide the agency’s investment 

boards, business units, and developers with a common understanding of 

the process and the cost, benefit, schedule, and risk criteria that will be 

used both to select new projects and to reselect ongoing projects for 

continued funding. According to the ITIM, this critical process requires, 

among other things, (1) making funding decisions for new proposals 

according to an established process; (2) providing adequate resources for 

investment selection activities; (3) using a defined selection process to 

select new investments and reselect ongoing investments; (4) establishing 

criteria for analyzing, prioritizing, and selecting new IT investments and for

Activities 1. The organization defines and 

documents business needs for both 

proposed and ongoing IT projects and 

systems.

Executed HHS policies and procedures call for business needs for 

enterprisewide and component agency ongoing and proposed IT 

projects and systems to be specified in the business case or 

Select forms. 

We verified that business needs were defined and documented 

within the business case or Select forms in the portfolio 

management tool for the four projects we reviewed. 

2. The organization identifies specific 

users and other beneficiaries of IT 

projects and systems.

Executed HHS policy and procedures call for specific users and other 

beneficiaries of both enterprisewide and component agency IT 

projects and systems to be identified in the business case and 

Select forms. 

We verified that customers and stakeholders were defined and 

documented within the business case or Select forms in the 

portfolio management tool for the four projects we reviewed. 

3. Users participate in project 

management throughout an IT project’s 

or system’s life cycle.

Executed According to HHS IT officials, end users participate in project 

management throughout an IT project's or system's life cycle. 

We verified that users participated in project management 

throughout the life cycle of the two enterprisewide projects we 

reviewed. According to HHS Office of the CIO, user participation 

in project management is not addressed at the department level 

for the two component agency projects we reviewed since it is 

delegated to the component agency.

4. The investment board periodically 

evaluates the alignment of its IT projects 

and systems with the organization’s 

strategic goals and objectives and takes 

corrective actions when misalignment 

occurs.

Executed The ITIRB evaluates the alignment of both HHS enterprisewide 

and component agency IT systems through the annual budget 

formulation process and takes corrective action when 

misalignment occurs. 

(Continued From Previous Page)

Type of 

practice Key practice Rating Summary of evidence
Page 29 GAO-06-11 HHS’s IT Investment Management Process

reselecting ongoing investments; and (5) creating a process for ensuring 

that the criteria change as organizational objectives change. (The complete 

list of key practices is provided in table 6.)

HHS has executed 7 of the 10 key practices associated with selecting an 

investment. For example, resources devoted to selection activities include 

the Critical Partners, Business Case Quality Review Team, and portfolio 

management tool, which contains several forms for selecting IT projects 

and systems. HHS also has detailed procedures for using its portfolio 

management tool and developing business cases. The criteria for analyzing, 

prioritizing, selecting and reselecting new and ongoing investments 

address the President’s Management Agenda, HHS strategic goals, and IT 

strategic goals, value, and risk. They are incorporated into the department’s 

portfolio management tool and are reviewed by the investment review 

board and adjusted within the tool annually at the beginning of each budget 

cycle to reflect organizational objectives. This year, HHS added additional 

criteria—a quality score. 

HHS uses its annual budget formulation process to select both 

enterprisewide and component agency proposed and ongoing IT 

investments. We verified that the four projects we reviewed were 

reselected by the department using the annual budget formulation process. 

Although HHS has the above strengths, the department has not executed 

any of the practices associated with documenting policies and procedures. 

Specifically, HHS has not fully documented its process for selecting new IT 

proposals and reselecting ongoing IT investments. Although a number of 

documents address investment selection, they are not linked to provide 

decision makers with a clear understanding of the selection and reselection 

processes. In addition, they do not define the roles and responsibilities for 

all key players involved in these processes. Moreover, although the HHS 

Office of the CIO works directly with the department’s Office of the Budget, 

HHS does not have policies and procedures documenting the integration of 

funding with the process of selecting and reselecting investments. Until the 

department fully documents policies and procedures for selecting new IT 

proposals and reselecting ongoing IT investments, the department will not 

be adequately certain that it is consistently and objectively selecting and 

reselecting investments that best meet the needs and priorities of the 

department.

Table 6 shows the rating for each key practice required to select an 

investment and summarizes the evidence that supports these ratings.
Page 30 GAO-06-11 HHS’s IT Investment Management Process

Table 6: Selecting an Investment

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitments

1. The organization has documented 

policies and procedures for selecting 

new IT proposals.

Not 

executed

Although HHS has a number of documents that address 

investment selection, they are not linked to provide decision 

makers with a common understanding of the selection process. In 

addition, these documents do not define the roles and 

responsibilities for each participating unit involved in the project 

selection process.

2. The organization has documented 

policies and procedures for reselectinga

ongoing IT investments.

Not 

executed

Although HHS has a number of documents that address 

investment reselection, they are not linked to provide the decision 

makers with a common understanding of the selection process. In 

addition, these documents do not define the roles and 

responsibilities for each participating unit involved in the project 

selection process.

3. The organization has policies and 

procedures for integrating funding with 

the process of selecting an investment.

Not 

executed

Although the HHS Office of the CIO works directly with the 

department’s Office of the Budget, HHS does not have policies and 

procedures documenting the integration of funding with the 

process of selecting and reselecting investments. 

Prerequisites 1. Adequate resources, including people, 

funding, and tools, are provided for 

identifying and selecting IT projects and 

systems.

Executed Adequate resources are provided for identifying and selecting IT 

projects and systems. They include the Critical Partners, Business 

Case Quality Review Team, and the department’s portfolio 

management tool, which contains several forms for selecting IT 

projects and systems.

2. Criteria for analyzing, prioritizing, and 

selecting new IT investment 

opportunities have been established.

Executed HHS has established criteria for analyzing, prioritizing, and 

selecting enterprisewide and component agency new IT 

investments. The department selects new IT proposals and 

reselects ongoing investments using the same criteria, which are 

incorporated into its portfolio management tool.

3. Criteria for analyzing, prioritizing, and 

reselecting IT investment opportunities 

have been established.

Executed HHS has established criteria for analyzing, prioritizing, and 

reselecting both enterprisewide and component agency IT 

investments. The department selects new IT proposals and 

reselects ongoing investments using the same criteria, which are 

incorporated into its portfolio management tool.

4. A mechanism exists to ensure that the 

criteria continue to reflect organizational 

objectives.

Executed The HHS ITIRB reviews and adjusts criteria annually at the start of 

each budget cycle and updates the portfolio management tool to 

reflect HHS's objectives.
Page 31 GAO-06-11 HHS’s IT Investment Management Process

Source: GAO.

a

According to the GAO ITIM framework, reselecting is the periodic reconsideration of an investment’s 

continuing value to the organization and the decision to continue funding. It is a recurring process that 

continues for as long as a project is receiving funding.

HHS Does Not Have a Process 

for Effectively Overseeing Its 

Component Agency IT 

Investments

An organization should effectively oversee its IT projects throughout all 

phases of their life cycles. Its investment board should observe each 

project’s performance and progress toward predefined cost and schedule 

expectations as well as each project’s anticipated benefits and risk 

exposure. This does not mean that a departmental board, such as the 

ITIRB, should micromanage each project to provide effective oversight; 

rather it means that the departmental board should be actively involved in 

all IT investments and proposals that are high cost or high risk or have 

significant scope and duration and at a minimum, should, have a 

mechanism for maintaining visibility of other investments. The board 

should also employ early warning systems that enable it to take corrective 

actions at the first sign of cost, schedule, and performance slippages. 

According to the ITIM, effective project oversight requires, among other 

things, (1) having written policies and procedures for management 

oversight; (2) developing and maintaining an approved management plan 

for each IT project; (3) making up-to-date cost and schedule data for each 

project available to the oversight boards; (4) having regular reviews by 

each investment board of each project’s performance against stated 

expectations; and (5) ensuring that corrective actions for each 

underperforming project are documented, agreed to, implemented, and 

tracked until the desired outcome is achieved. (The complete list of key 

practices is provided in table 7.)

Activities 1. The organization uses its defined 

selection process, including predefined 

selection criteria, to select new IT 

investments.

Executed HHS uses its annual budget formulation process to select new IT 

investments. 

We verified that the four projects we reviewed were selected using 

the annual budget formulation activities.

2. The organization uses the defined 

selection process, including predefined 

selection criteria, to reselect ongoing IT 

investments.

Executed HHS uses its annual budget formulation process to reselect 

ongoing IT investments. 

We verified that the four projects we reviewed were reselected 

using the annual budget formulation activities.

3. Executives’ funding decisions are 

aligned with selection decisions.

Executed The HHS ITIRB makes funding decisions for new and ongoing IT 

investments through the department’s budget formulation process, 

which is used to select both enterprisewide and component 

agency investments.

(Continued From Previous Page)

Type of 

practice Key practice Rating Summary of evidence
Page 32 GAO-06-11 HHS’s IT Investment Management Process

The department has not executed any of the seven key practices associated 

with effective project oversight, primarily because of its limited role in 

overseeing component agency IT investments. Specifically, while the 

department has documented standard operating procedures and 

instructional memorandums for oversight of enterprisewide IT 

investments, they are not comprehensive in that they do not specify the 

board’s responsibilities for investment oversight; procedural rules for the 

ITIRB operations and decision making during project oversight; or policies 

and procedures for overseeing component agency IT investments. 

The HHS ITIRB is currently performing regular reviews23 of enterprisewide 

IT projects and systems against stated expectations through reports that 

are available to decision makers on the HHS Intranet. However, the 

department is not regularly reviewing component agency investments that 

are high risk, crosscutting, and require review by the Office of Management 

and Budget, although their policy calls for it. The board also does not have 

a mechanism for maintaining visibility of other component agency 

investments. 

The department delegates oversight of these investments to the component 

agencies but believes it is nonetheless effectively overseeing component 

agency investments through (1) reviews of these investments as part of the 

annual Critical Partner and Business Case Quality reviews performed 

during the annual selection process and the use of (2) earned value 

management data.24 Although the annual reviews may provide insight into 

the status of investments, they are not frequent enough to allow for timely 

identification of problems. Moreover, while HHS officials told us that staff 

responsible for collecting earned value management data on component 

agency investments share significant concerns about the data with the 

ITIRB, they did not have formal documentation clearly supporting this 

issue. In addition, formal procedures for elevating issues to the board have 

not been developed. In the absence of effective board oversight, HHS 

executives will not have the information they need to determine whether 

23HHS conducts quarterly reviews on its enterprisewide investments during the period of 

development and annual reviews of its steady state enterprisewide investments, that is, 

those systems that have completed development and become operational. 

24Earned value management is a project management tool that integrates the investment 

scope of work with schedule and cost elements for investment planning and control. This 

method compares the value of work accomplished during a given period with that of the 

work expected in the period. Differences in expectations are measured in both cost and 

schedule variances.
Page 33 GAO-06-11 HHS’s IT Investment Management Process

component agency projects are being developed on schedule and within 

budget. In addition, the department will run the risk that underperforming 

component agency projects will not be identified in time for corrective 

actions to be taken. 

We verified that HHS provided oversight for the two enterprisewide 

investments, but had delegated oversight activities for the two component 

agency investments we reviewed. 

Table 7 shows the rating for each key practice required to provide 

investment oversight and summarizes the evidence that supports these 

ratings.

Table 7: Providing Investment Oversight

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has documented 

policies and procedures for management 

oversight of IT projects and systems.

Not 

executed

Although HHS has developed standard operating procedures and 

instructional memorandums for oversight of enterprisewide IT 

projects and systems, they do not (1) specify the HHS ITIRB's 

responsibilities when providing investment oversight within its 

domain or (2) procedural rules for the ITIRB's operations and for 

decision making during project oversight. In addition, HHS does 

not have policies and procedures for management oversight of 

component agency investments.

Prerequisites 1. Adequate resources, including people, 

funding, and tools, are provided for IT 

project oversight.

Not 

executed

Although HHS has adequate resources for providing oversight for 

enterprisewide IT investments, the department does not have 

adequate resources for providing oversight for component agency 

IT investments.

2. IT projects and systems, including 

those in steady state (operations and 

maintenance), maintain approved project 

management plans that include expected 

cost and schedule milestones and 

measurable benefit and risk expectations.

Not 

executed

HHS’s policy calls for an accountability agreement document and 

business case, including cost, benefit, schedule, and risk 

expectations, to be available to the ITIRB after approval of an 

enterprisewide IT projects and systems, but there is no similar 

requirement for component agency IT projects and systems. 

We verified that HHS provided oversight for the two 

enterprisewide investments, but had delegated oversight activities 

for the two component agency investments we reviewed.
Page 34 GAO-06-11 HHS’s IT Investment Management Process

Source: GAO.

HHS Has a Defined Process for 

Capturing Investment 

Information

To make good IT investment decisions, an organization must be able to 

acquire pertinent information about each investment and store that 

information in a retrievable format. During this critical process, an 

organization identifies its IT assets and creates a comprehensive repository 

of investment information. This repository provides information to 

investment decision makers to help them evaluate the potential impacts 

and opportunities created by proposed or continuing investments. It can 

provide insights into major IT cost and management drivers and trends. 

The repository can take many forms and need not be centrally located, but 

the collection method should, at a minimum, identify each IT investment 

and its associated components. This critical process may be satisfied by the 

information contained in the organization’s current enterprise architecture, 

augmented by additional information—such as financial information and 

information on risk and benefits—that the investment board may require to 

Activities 1. Data on actual performance (including 

cost, schedule, benefit, and risk 

performance) are provided to the 

appropriate IT investment board.

Not 

executed

Data on actual performance of enterprisewide IT investments are 

provided to the HHS ITIRB; however, the ITIRB does not regularly 

receive data on actual performance of a defined set of component 

agencies’ IT investments and maintain visibility of other 

investments. 

We verified that the two enterprisewide projects provide quarterly 

reports to the ITIRB. For the component agency projects we 

reviewed, this activity is delegated to the component agency and 

is not addressed at the department level.

2. Using verified data, each investment 

board regularly reviews the performance 

of IT projects and systems against stated 

expectations.

Not 

executed

HHS ITIRB quarterly reviews performance of enterprisewide IT 

investments under development and annually reviews 

enterprisewide IT investment in their operational phase of their life 

cycles; however, the investment board does not have a process 

for regularly reviewing the performance of a defined set of 

component agency investments and maintaining visibility of other 

investments. 

3. For each underperforming IT project or 

system, appropriate actions are taken to 

correct or terminate the project or system 

in accordance with defined criteria and 

the documented policies and procedures 

for management oversight.

Not 

executed

The HHS ITIRB takes appropriate actions to correct or terminate 

the enterprisewide IT projects or systems. However, it does not 

take actions to correct or terminate underperforming component 

agency investments because it does not regularly review these 

investments’ performance.

4. The investment board regularly tracks 

the implementation of corrective actions 

for each underperforming project until the 

actions are completed.

Not 

executed

The HHS ITIRB maintains meeting minutes for enterprisewide IT 

investments to ensure that corrective actions are implemented 

and tracked until the desired outcome is achieved. However, it 

does not take actions to correct or terminate underperforming 

component agency investments because it does not regularly 

review these investments’ performance.

(Continued From Previous Page)

Type of 

practice Key practice Rating Summary of evidence
Page 35 GAO-06-11 HHS’s IT Investment Management Process

ensure that informed decisions are being made. According to the ITIM, 

effectively managing this repository requires, among other things, 

(1) developing written policies and procedures for identifying and 

collecting the information; (2) assigning responsibilities for ensuring that 

the information being collected meets the needs of the investment 

management process; (3) identifying IT projects and systems and collecting 

relevant information to support decisions about them; and (4) making the 

information easily accessible to decision makers and others. (The complete 

list of key practices is provided in table 8.)

HHS has executed 5 of the 6 key practices for capturing investment 

information. For example, the department has several documents that 

define the policies and procedures for identifying and collecting investment 

information in its repositories and also assign responsibility to the HHS 

CIO for ensuring that the information collected during project and systems 

identification meets the needs of the investment management process. 

HHS maintains a portfolio management tool, which serves as the primary 

repository for identifying and collecting information about both 

department and component agency IT projects and systems. The 

department’s portfolio management tool is easily accessible to decision 

makers at both the department and component level and the Office of the 

CIO has provided decision makers with various training manuals and 

guidance memorandums. In addition, the department also identifies and 

collects information about enterprisewide IT investments using its 

Intranet. Further, the department recently began collecting earned value 

information through spreadsheets on major HHS IT investments that 

compares planned and actual cost and schedule information. These 

repositories are easily accessible to the board members. 

The key practice HHS has not executed has to do with the captured 

investment information not yet being used by the HHS ITIRB to fully 

support decisions about component agency investments. For example, the 

earned value investment data received from each component agency has 

not been used by the HHS ITIRB for control and evaluation decisions. 

According to agency officials, the department has recently begun 

monitoring the earned value data to identify investments that report cost 

and schedule variances and these officials acknowledge a need to formalize 

the process for doing so. Until HHS’s decision makers use the information 

in the repository to fully support the investment management process, it 

will be unable to effectively evaluate the impacts and opportunities created 

by proposed or continuing investments.
Page 36 GAO-06-11 HHS’s IT Investment Management Process

Table 8 shows the rating for each key practice required to capture 

investment information and summarizes the evidence that supports these 

ratings.

Table 8: Capturing Investment Information

Source: GAO.

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitments

1. The organization has documented policies 

and procedures for identifying and collecting 

information about IT projects and systems to 

support the investment management process.

Executed The department has documented policies and procedures for 

identifying and collecting information about IT projects and 

systems to support the investment management process.

2. An official is assigned responsibility for 

ensuring that the information collected during 

project and systems identification meets the 

needs of the investment management 

process.

Executed The HHS CIO is responsible for ensuring that the information 

collected during project and systems identification meets the 

needs of the investment management process. 

Prerequisite 1. Adequate resources, including people, 

funding, and tools, are provided for identifying 

IT projects and systems and collecting 

relevant investment information about them.

Executed According to the HHS IT officials, adequate resources are 

provided for identifying IT projects and systems and 

collecting relevant investment information about them. 

Activities 1. The organization’s IT projects and systems 

are identified, and specific information is 

collected to support decisions about them.

Executed HHS’s portfolio management tool identifies and collects 

information about both department and component agency 

IT projects and systems to support the investment 

management process as it currently exists. The department 

also identifies and collects relevant investment information 

for the enterprisewide IT investments through the HHS 

Intranet and component agency IT investments through 

spreadsheets that capture earned value data. 

We verified that HHS’s portfolio management tool identifies 

and contains investment information for the four projects we 

reviewed.

2. The information that has been collected is 

easily accessible and understandable to 

decision makers and others.

Executed IT investment decision makers at both the department and 

component agency level have access to HHS’s portfolio 

management tool that is used to capture IT project and 

system information. Instructions on the use and navigation 

through the portfolio management system are available to 

investment management decision makers. In addition, the 

HHS ITIRB can also access the enterprisewide IT investment 

information posted on the HHS Intranet. 

3. The information repository is used by 

investment decision makers and others to 

support investment management.

Not 

executed

While HHS identifies and collects information about IT 

projects and systems to support the investment management 

process, this information has not been used by the HHS 

ITIRB to fully support the control and evaluate decisions for 

component agency IT investments.
Page 37 GAO-06-11 HHS’s IT Investment Management Process

HHS Has Some of the 

Capabilities Needed to Manage 

IT Investments as a Portfolio

Once an agency has attained Stage 2 maturity, it needs to implement critical 

processes for managing its investments as a portfolio (Stage 3). An IT 

investment portfolio is an integrated, agencywide collection of investments 

that are assessed and managed collectively based on common criteria. 

Managing investments as a portfolio is a conscious, continuous, and 

proactive approach to allocating limited resources among an organization’s 

competing initiatives in light of the relative benefits expected from these 

investments. Taking an agencywide perspective enables an organization to 

consider its investments comprehensively, so that collectively the 

investments optimally address the organization’s missions, strategic goals, 

and objectives. Managing IT investments as a portfolio also allows an 

organization to determine its priorities and make decisions about which 

projects to fund and continue to fund based on analyses of the relative 

organizational value and risks of all projects, including projects that are 

proposed, under development, and in operation. Although investments may 

initially be organized into subordinate portfolios—based on, for example, 

business lines or life cycle stages—and managed by subordinate 

investment boards; they should ultimately be aggregated into this 

enterprise-level portfolio.

According to the ITIM framework, Stage 3 maturity includes (1) defining 

the portfolio criteria, (2) creating the portfolio, (3) evaluating the portfolio, 

and (4) conducting postimplementation reviews. Table 9 summarizes the 

purpose of each critical process in Stage 3.

Table 9: Stage 3 Critical Processes—Developing a Complete Investment Portfolio

Source: GAO.

HHS has executed 8 of the 27 key practices required by Stage 3. For 

example, the department’s core IT portfolio selection criteria, including 

Critical process Purpose

Defining the portfolio criteria To ensure that the organization develops and maintains IT portfolio selection criteria that 

support its mission, organizational strategies, and business priorities.

Creating the portfolio To ensure that IT investments are analyzed according to the organization’s portfolio selection 

criteria and to ensure that an optimal IT investment portfolio with manageable risks and returns 

is selected and funded.

Evaluating the portfolio To review the performance of the organization’s investment portfolio(s) at agreed-upon intervals 

and to adjust the allocation of resources among investments as necessary.

Conducting postimplementation 

reviews

To compare the results of recently implemented investments with the expectations that were set 

for them and to develop a set of lessons learned from these reviews.
Page 38 GAO-06-11 HHS’s IT Investment Management Process

cost, benefit, schedule, and risk are approved by the HHS ITIRB. In 

addition, the investment board examines the mix of new and ongoing 

investments and their respective data and analyses to select investments to 

fund. However, many key practices still need to be executed before HHS 

can effectively manage its IT investments from a portfolio perspective. For 

example, HHS has not addressed any of the key practices related to 

evaluating the portfolio or conducting PIRs. Until HHS fully implements the 

critical processes associated with managing its investments as a complete 

portfolio, it will not have the data it needs to make informed decisions 

about competing investments.

Table 10 summarizes the status of HHS’s critical processes for Stage 3, 

showing how many associated key practices it has executed.

Table 10: Summary of Results for Stage 3 Critical Processes and Key Practices 

Source: GAO.

Process for Modifying IT 

Portfolio Selection Criteria Is 

Not Institutionalized 

To manage IT investments effectively, an organization needs to establish 

rules or “portfolio selection criteria” for determining how to allocate scarce 

funding to existing and proposed investments. Thus, developing an IT 

investment portfolio requires defining appropriate cost, benefit, schedule, 

and risk criteria with which to evaluate individual investments in the 

context of all other investments. To ensure that the organization’s strategic 

goals, objectives, and mission will be satisfied by its investments, the 

criteria should have an enterprisewide perspective. Further, if an 

organization’s mission or business needs and strategies change, criteria for 

selecting investments should be reexamined and modified as appropriate. 

Portfolio selection criteria should be disseminated throughout the 

organization to ensure that decisions concerning investments are made in a 

consistent manner and that this critical process is institutionalized. To 

Critical process

Key practices

executed

Total required

by critical

process

Percentage of key

practices executed

Defining the portfolio criteria 5 7 71

Creating the portfolio 3 7 43

Evaluating the portfolio 0 7 0

Conducting 

postimplementation reviews 0 6 0

Total 8 27 30
Page 39 GAO-06-11 HHS’s IT Investment Management Process

achieve this result, project management personnel and others should be 

aware of the criteria and address the criteria in funding submissions for 

projects. Resources required for this critical process typically include the 

time and attention of executives involved in the process, adequate funding, 

and supporting tools. (The complete list of key practices is provided in 

table 11.)

The department has executed 5 of the 7 key practices for this critical 

process. For example, responsibility has been assigned to the HHS Lead 

Capital Planner for managing the development and modification of the IT 

portfolio selection criteria, and adequate resources have been committed 

for portfolio selection activities, including the Critical Partners, portfolio 

management tool project manager, and the Office of the CIO staff. 

Moreover, the project management personal and other stakeholders are 

aware of the portfolio selection criteria that are embedded into the 

department’s portfolio management tool and also contained within policies 

and procedures. 

Finally, the HHS ITIRB approves the core IT selection criteria, including 

cost, benefit, schedule, and risk criteria, based on the organization’s 

mission, goals, strategies, and priorities. Beginning in fiscal year 2004, HHS 

began scoring and ranking approximately 80 percent of its IT investments 

against alignment, value, and risk criteria in order to determine a priority 

score, which is the sum of alignment, value, and risk criteria scores, 

weighted for relative importance. Similarly, for the fiscal year 2007 budget 

formulation process, HHS began collecting investment information on the 

business case quality, Critical Partner reviews, and cost and schedule 

variance to determine a quality score, which is the sum of the business case 

quality, Critical Partner reviews, and cost and schedule variance scores, 

weighted for relative importance. The HHS ITIRB evaluates and annually 

adjusts its portfolio selection criteria within the portfolio management tool.

Despite these important steps in defining portfolio selection criteria, 

weaknesses remain. The department has not developed policies or 

procedures for modifying the portfolio selection criteria to reflect changes 

to HHS mission, goals, strategies, and priorities. In addition, the HHS ITIRB 

began reviewing the IT portfolio selection criteria this year. However, the 

process for modifying portfolio selection criteria is not institutionalized 

because the process to do so was only used once and there are no 

documented policies and procedures to ensure that it will be used again. 

Until HHS defines and implements the practices required for defining the 

portfolio criteria definition, it will not have the tool it needs to select 
Page 40 GAO-06-11 HHS’s IT Investment Management Process

investments that support its mission, organizational strategies, and 

business priorities. 

Table 11 shows the rating for each key practice required to define portfolio 

selection criteria and summarizes the evidence that supports these ratings.

Table 11: Defining the Portfolio Criteria

Source: GAO.

Process for Creating a Portfolio 

Is Not Documented

At Stage 3, organizations create a portfolio of IT investments to ensure that 

IT investments are analyzed according to the organization’s portfolio 

selection criteria and to ensure that an optimal IT investment portfolio with 

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitments

1. The organization has documented 

policies and procedures for creating and 

modifying IT portfolio selection criteria.

Not 

executed

While HHS has policies and procedures for creating IT portfolio 

selection criteria, the department lacks policies and procedures for 

modifying the portfolio selection criteria.

2. Responsibility is assigned to an 

individual or group for managing the 

development and modification of the IT 

portfolio selection criteria.

Executed The HHS Lead Capital Planner is responsible for managing the 

development and modification of the IT portfolio selection criteria. 

Prerequisites 1. Adequate resources, including people, 

funding, and tools, have been committed 

for portfolio selection criteria activities.

Executed Adequate resources have been committed for portfolio selection 

criteria activities. They include the Critical Partners, portfolio 

management tool project manager, and the Office of the CIO staff.

2. A working group has been designated 

to be responsible for developing and 

modifying the IT portfolio selection 

criteria.

Executed The CPIC Reengineering/PMT Implementation Team conducts 

weekly teleconferences with HHS component agencies to 

coordinate investment management issues, including the 

development and modification of IT portfolio selection criteria. 

According to HHS IT officials, this group will evolve into the Policy 

Advisory Board, which, among other things, will formalize the IT 

portfolio selection criteria activities.

Activities 1. The enterprisewide investment board 

approves the core IT portfolio selection 

criteria, including CBSR criteria, based 

on the organization’s mission, goals, 

strategies, and priorities.

Executed The HHS ITIRB approves the core IT portfolio selection criteria, 

including cost, benefit, schedule, and risk criteria, based on the 

organization’s mission, goals, strategies, and priorities.

2. Project management personnel and 

other stakeholders are aware of the 

portfolio selection criteria.

Executed Project management personnel and other stakeholders are aware 

of the portfolio selection criteria, which are embedded into HHS’s 

portfolio management tool and contained in policies and 

procedures.

3. The enterprisewide investment board 

regularly reviews the IT portfolio 

selection criteria, using cumulative 

experience and event-driven data, and 

modifies the criteria as appropriate.

Not 

executed

The HHS ITIRB began reviewing the IT portfolio selection criteria 

this year. However, the process for modifying the portfolio 

selection criteria is not institutionalized because it was only used 

once and there are no documented policies and procedures to 

ensure that it will be used again.
Page 41 GAO-06-11 HHS’s IT Investment Management Process

manageable risks and returns is selected and funded. According to ITIM, 

creating the portfolio requires organizations to, among other things, 

document policies and procedures for analyzing, selecting, and maintaining 

the portfolio; provide adequate resources, including people, funding, and 

tools for creating the portfolio; and capture the information used to select, 

control, and evaluate the portfolio and maintain it for future reference. In 

creating the portfolio, the investment board must also (1) examine the mix 

of new and ongoing investments, and their respective data and analyses 

and select investments for funding and (2) approve or modify the 

performance expectations for the IT investments they have selected. (The 

complete list of key practices is provided in table 12.)

HHS has executed 3 of the 7 key practices associated with creating the 

portfolio. Beginning in fiscal year 2004, the department began to create a 

portfolio by using its portfolio management tool to collect cost, benefit, 

schedule, risk, strategic alignment, and enterprise architecture information 

on investments accounting for 80 percent of the dollar value of the HHS IT 

investment portfolio. Each component agency’s IT portfolio is displayed in 

priority order along with where each investment falls within the overall IT 

portfolio. Further, according to HHS IT officials, the agency has adequate 

resources for portfolio selection activities, including the Critical Partners, 

the portfolio management tool project manager, and the Office of the CIO 

staff. These officials also stated that HHS ITIRB members are also 

knowledgeable about the process of creating a portfolio.

Nevertheless, HHS has a number of significant weaknesses in the way it 

creates a portfolio. First, it does not have policies and procedures that 

sufficiently address this critical process. Although the department has 

policies and procedures for creating IT portfolio selection criteria, they 

lack policies and procedures for using these criteria to analyze, select, and 

maintain the investment portfolio. Second, even though the HHS ITIRB has 

quarterly reviews to compare project and system performance with 

expectations for enterprisewide IT investments, the board is not provided 

with information comparing the performance of component agency 

investments against expectations. In addition, the board approves or 

modifies the performance expectations for the enterprisewide IT 

investments it has selected, but does not regularly approve or modify the 

performance expectations for component agency IT investments or ensure 

that this is done. Moreover, as previously mentioned, investment 

information has not been used to fully support control and evaluate 

decisions for component agency investments. Unless HHS defines and 

implements the practices for creating a comprehensive portfolio of IT 
Page 42 GAO-06-11 HHS’s IT Investment Management Process

investments, it will not be able to determine whether it has selected the mix 

of investments that best meets its needs considering resource and funding 

constraints.

Table 12 shows the rating for each key practice required to create a 

portfolio and summarizes the evidence that supports these ratings.

Table 12: Creating the Portfolio

Source: GAO.

Criteria for Portfolio 

Performance Evaluations Are 

Not Yet Developed or Regularly 

Modified

This critical process builds upon the Stage 2 critical process, Providing 

Investment Oversight, by adding the elements of portfolio performance to 

an organization’s investment control capacity. Compared with less mature 

organizations, Stage 3 organizations will have the foundation they need to 

control the risks faced by each investment and to deliver benefits that are 

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has documented 

policies and procedures for analyzing, 

selecting, and maintaining the investment 

portfolio.

Not 

executed

While HHS has policies and procedures for creating IT portfolio 

selection criteria, the department lacks policies and procedures 

for using these criteria to analyze, select, and maintain the 

investment portfolio.

Prerequisites 1. Adequate resources, including people, 

funding, and tools, are provided for the 

process of creating the portfolio.

Executed According to HHS IT officials, adequate resources have been 

committed for portfolio selection criteria activities. They include 

the Critical Partners, portfolio management tool project manager, 

and Office of the CIO staff.

2. Board members are knowledgeable 

about the process of creating a portfolio.

Executed HHS ITIRB members are knowledgeable about the process of 

creating a portfolio; they have now gone through the process 

twice.

3. The investment board is provided with 

information comparing project and system 

performance with expectations.

Not 

executed

While the investment board is provided with information 

comparing HHS enterprisewide project and system performance 

with expectations, it is not provided with information comparing 

the performance of component agency investments against 

expectations.

Activities 1. Each IT investment board examines the 

mix of new and ongoing investments and 

their respective data and analyses and 

selects investments for funding.

Executed The ITIRB examines a mix of new and ongoing investments 

through the department’s portfolio management tool, which is 

used to analyze, prioritize, and select investments for funding. 

2. Each investment board approves or 

modifies the performance expectations for 

its selected IT investments.

Not 

executed

While the HHS ITIRB approves the performance expectations for 

its enterprisewide IT investments, it does not have a similar 

process for approving the performance expectations for 

component agency IT investments or ensuring that this is done.

3. Information used to select, control, and 

evaluate the portfolio is captured and 

maintained for future reference.

Not 

executed

Although HHS is capturing investment information, the 

information is not yet used to fully support control and evaluate 

decisions about component agency investments. 
Page 43 GAO-06-11 HHS’s IT Investment Management Process

linked to mission performance. In addition, a Stage 3 organization will have 

the benefit of performance data generated by Stage 2 processes. Executivelevel oversight of risk management outcomes and incremental benefit 

accumulation provides the organization with increased assurance that each 

IT investment will achieve the desired results. (The complete list of key 

practices is provided in table 13.)

HHS has not executed any of the seven key practices for evaluating a 

portfolio. It has yet to develop policies and procedures that address 

performance oversight from a portfolio perspective. Moreover, while the 

department annually reviews its portfolio as part of its selection process, it 

does not evaluate the investment portfolio on a continuing basis to assess 

its performance. Finally, the results of Providing Investment Oversight 

reviews from Stage 2 are important to this critical process. However, as 

previously mentioned, while the HHS ITIRB has oversight of 

enterprisewide investments, it does not regularly review a defined set of 

component agencies’ investments and maintain visibility of other 

investments. Although the department’s portfolio management tool has the 

ability to summarize performance metrics for each investment and quickly 

understand the status of each investment and any potential emerging 

problem area, the tool is currently only being used on an ad hoc basis to 

make portfolio oversight decisions. Defining and implementing processes 

to evaluate the performance of its entire portfolio would provide HHS with 

greater assurance that it is controlling the risks and achieving the benefits 

associated with the mix of investments it has selected.

Table 13 shows the rating for each key practice required to evaluate the 

portfolio and summarizes the evidence that supports these ratings.
Page 44 GAO-06-11 HHS’s IT Investment Management Process

Table 13: Evaluating the Portfolio

Source: GAO.

Process for Conducting 

Postimplementation Reviews Is 

Not Defined

The purpose of a PIR is to evaluate an investment after it has completed 

development (that is, after its transition from the implementation phase to 

the operations and maintenance phase) in order to validate actual 

investment results. This review is conducted to (1) examine differences 

between estimated and actual investment costs and benefits and possible 

ramifications for unplanned funding needs in the future and (2) extract 

“lessons learned” about the investment selection and control processes 

that can be used as the basis for management improvements. Similarly, 

PIRs should be conducted for investment projects that were terminated 

before completion, to readily identify potential management and process 

improvements. (The complete list of key practices is provided in table 14.)

HHS has not executed the six key practices for conducting PIRs. Although 

its policy calls for postimplementation reviews of IT investments that have 

recently completed implementation of the entire investment or a significant 

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has documented 

policies and procedures for reviewing, 

evaluating, and improving the 

performance of its portfolio(s).

Not 

executed

HHS does not have policies and procedures for reviewing, 

evaluating, and improving the performance of its portfolio. 

Prerequisites 1. Adequate resources, including people, 

funding, and tools have been provided for 

reviewing the investment portfolio and its 

projects.

Not 

executed

Although HHS annually reviews its portfolio as part of its selection 

process, it does not evaluate the performance on a continuing 

basis.

2. Board members are familiar with the 

process for evaluating and improving the 

portfolio’s performance.

Not 

executed

Although HHS annually reviews its portfolio as part of its selection 

process, it does not evaluate the performance on a continuing 

basis.

3. Results of relevant Providing 

Investment Oversight reviews from Stage 

2 are provided to the investment board.

Not 

executed

While the HHS ITIRB has oversight of enterprisewide investments, 

it does not effectively oversee its component agency IT 

investments. 

4. Criteria for assessing portfolio 

performance are developed, reviewed, 

and modified at regular intervals to reflect 

current performance expectations.

Not 

executed

HHS does not have criteria for assessing portfolio performance.

Activities 1. IT portfolio performance measurement 

data are defined and collected consistent 

with portfolio performance criteria.

Not 

executed

HHS does not have criteria for assessing portfolio performance.

2. Adjustments to the IT investment 

portfolio are executed in response to 

actual portfolio performance.

Not 

executed

Although HHS annually reviews its portfolio as part of its selection 

process, it does not evaluate the performance on a continuing 

basis.
Page 45 GAO-06-11 HHS’s IT Investment Management Process

phase of the investment, the department does not have specific procedures 

for conducting such reviews, including specifying who conducts and 

participates in the PIR, what information is presented in a PIR, or how 

results are to be disseminated to decision makers. To date, HHS has 

conducted closeout reviews of two enterprisewide investments following 

their implementation; however, while these reports do cover investment 

cost expectations, they cannot be considered PIRs because the reports do 

not address general conclusions, lessons learned, or schedule deviations. 

Unless PIRs are conducted on a regular basis, HHS will not be able to 

effectively evaluate the results of its IT investments to determine whether 

continuation, modification, or termination of an IT investment would be 

necessary in order to meet stated HHS mission objectives.

Table 14 shows the rating for each key practice required to conduct PIRs 

and summarizes the evidence that supports these ratings.

Table 14: Conducting Postimplementation Reviews

Source: GAO.

Type of 

practice Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has documented policies and procedures for 

conducting PIRs.

Not 

executed

Although, HHS has policy for conducting 

PIRs, the department does not have 

associated procedures for conducting 

such reviews. 

Prerequisites 1. Adequate resources, including people, funding, and tools, have 

been provided for conducting PIRs.

Not 

executed

HHS is not conducting PIRs.

2. Individuals assigned to the investment board to conduct PIRs 

should be familiar with both the policies and the procedures for 

conducting such reviews.

Not 

executed

HHS is not conducting PIRs.

Activities 1. The investment board identifies which projects will have a PIR 

conducted.

Not 

executed

HHS is not conducting PIRs.

2. Quantitative and qualitative investment data are collected, 

evaluated for reliability, and analyzed during the PIRs.

Not 

executed

HHS is not conducting PIRs.

3. Lessons learned and recommendations for improving the 

investment process are developed during the PIR, documented, 

and then distributed to all stakeholders.

Not 

executed

HHS is not conducting PIRs.
Page 46 GAO-06-11 HHS’s IT Investment Management Process

HHS Has Provided Limited 

Guidance to and Oversight of 

Component Agencies’ 

Investment Management 

Processes

The ability of a department-level CIO to effectively oversee IT investment 

management processes throughout the agency depends on the existence of 

appropriate management structures with adequate authorities and 

sufficient guidance. Under the Clinger-Cohen Act of 1996, the CIO of each 

agency is responsible for effectively managing all of the agency’s IT 

resources. To comply with the act, HHS designates its CIO to be 

responsible for ensuring that the component agencies are defining and 

implementing effective investment management processes that are 

appropriately aligned with the department’s processes. 

Although each component agency has staff responsible for gathering, 

maintaining, and analyzing IT investment information, the HHS Office of 

the CIO has the responsibility to define and implement overall HHS IT 

investment management practices, and monitor component agency 

investment management practices to ensure a cohesive departmental 

process and the capability exists to carry out the process. In accordance 

with this, the department’s investment management policies and guidelines 

state that the component agencies are to establish and manage investment 

management processes and governance structures that are aligned with the 

department’s policies and procedures. However, as mentioned in previous 

sections, the department’s investment management policies and 

procedures have several weaknesses. For example, HHS does not have a 

set of documented procedures that provide decision makers with a clear 

understanding of the selection and reselection process. 

Moreover, HHS currently has no structured mechanism in place to ensure 

that the component agencies are adhering to the department’s policies and 

procedures. According to HHS officials, the CIO has the authority to audit a 

component agencies IT investment management process. However, they 

were unable to provide us evidence of having performed any such audits. 

These officials also stated that the department’s portfolio management tool 

is another method that will enable HHS to oversee component-level 

investment management processes. However, since not all component 

agencies are using the portfolio management tool to individually make 

select, control, and evaluate decisions, its usefulness in this regard is 

limited. Until the department develops a mechanism for ensuring that 

component agencies define and implement investment management 

processes that align with those of the department, it is running the risk that 

effective processes are being institutionalized at both the department and 

the component agency level. In addition, the department will be unable to 

ensure that it is optimizing its investments in IT and effectively assessing 

and managing the risks of these investments.
Page 47 GAO-06-11 HHS’s IT Investment Management Process

HHS Does Not Have a 

Plan to Coordinate and 

Guide Improvement 

Efforts

HHS has initiated several efforts to improve its investment management 

process. Specifically, it has drafted a revised investment management guide 

that addresses the weaknesses with current guidance that we identify in 

this report. In addition, in February 2005, HHS incorporated capabilities 

into its portfolio management tool to enhance performance of control and 

evaluate functions. Specifically, the tool now has the capabilities to 

produce (1) scorecards to provide data for each investment in a portfolio, 

allowing cross investment comparisons on data elements collected; (2) 

investor maps to provide a graphical depiction of a portfolio in terms of up 

to six data categories, with the ability to show target and actual values; and 

(3) a workbook module to track the identification and resolution of issues 

that may arise regarding the management of an investment or set of 

investments. 

Although HHS has initiated these efforts, they only fully address 2 of the 14 

Stage 2 key practices the department did not execute. 

• The draft investment management guidance, when finalized, will 

address weaknesses associated with one of the key practices for 

instituting the investment board by reflecting the current management 

process, including information on the roles of key working groups 

involved in the organization’s IT investment processes, and identifying 

the manner in which investments board’s processes are to be 

coordinated with other key organizational plans and processes. The 

guidance will also address the integration of the funding and selection 

processes, a key practice the department has not executed that is 

associated with selecting an investment. 

• The enhanced portfolio management tool capabilities will enhance the 

department’s ability to oversee investments’ performance and position 

the board to perform portfolio evaluation activities, but they will not 

fully address any of the weaknesses we identify. 

HHS has not coordinated these and additional efforts that would address 

the weaknesses we identify in this report in a comprehensive plan that (1) 

specifies measurable goals, objectives, and milestones; (2) specifies 

needed resources; (3) assigns clear responsibility and accountability for 

accomplishing tasks; and (4) is approved by senior management. We have 

previously reported that such a plan is instrumental in helping agencies 

coordinate and guide improvement efforts. Until HHS develops a plan that 

would allow for the systematic prioritization, sequencing, and evaluation of 
Page 48 GAO-06-11 HHS’s IT Investment Management Process

improvement efforts, the agency risks not being able to effectively 

establish the mature investment management processes that result in 

greater certainty about the outcomes of future IT investments.

Conclusions Because of the attention that has been given to investment management, 

HHS has established several of the practices needed to effectively manage 

its investments. These practices have strengthened the department’s basic 

capabilities for selecting and controlling projects and begun to equip the 

department with the capabilities it needs to make informed decisions about 

competing investments. However, several significant weaknesses remain in 

the foundational practices needed to manage individual investments, the 

portfolio-level investments needed to manage investments as a collection, 

and in the level of guidance and oversight provided to component agency 

investment management processes. These weaknesses hamper the 

department’s ability to ensure that it is managing the mix of investments 

that will maximize returns to the organization, taking into account the 

appropriate level of risk.

Critical to HHS’s success, going forward will be the development of an 

implementation plan that (1) is based on an assessment of strengths and 

weaknesses; (2) specifies measurable goals, objectives, and milestones; (3) 

specifies needed resources; (4) assigns clear responsibility and 

accountability for accomplishing tasks; and (5) is approved by senior 

management. Although the department has initiated improvement efforts, 

it has not developed a comprehensive plan to guide these and other efforts 

needed to improve its investment management process. Without such a 

plan and procedures for implementing it, it is unlikely that the department 

will effectively establish mature investment management capability. As a 

result, HHS will continue to be challenged in its ability to make informed 

and prudent investment decisions in managing its annual multibillion-dollar 

IT budget.

Recommendations for 

Executive Action

To strengthen HHS’s investment management capability and address the 

weaknesses discussed in this report, we recommend that the Secretary of 

the Department of Health and Human Services direct the Chief Information 

Officer to develop and implement a plan for improving the department’s IT 

investment management processes. The plan should address the 

weaknesses described in this report, beginning with those we identified in 

our Stage 2 analysis and continuing with those we identified in our Stage 3 
Page 49 GAO-06-11 HHS’s IT Investment Management Process

analysis. The plan should, at a minimum, provide for accomplishing the 

following:

In Stage 2

• Develop comprehensive guidance and additional supporting guidance 

that defines and describes the complete investment management 

process, unifies existing processes enterprisewide, reflects changes in 

processes as they occur; define the operations and decision-making 

processes of the HHS investment review board and other management 

entities, such as the component agencies, involved in managing IT 

investments.

• Ensure that HHS’s investment review board’s membership includes 

business representation of its component agencies as it begins to 

execute its full range of responsibilities.

• Develop well-defined and disciplined written procedures that outline 

the process for selecting new IT proposals, reselecting ongoing IT 

investments, and integrating funding with the process of selecting an 

investment.

• Establish a process for the investment board to regularly review and 

track the performance of a defined set of component agency IT systems 

against expectations, and take corrective actions when these 

expectations are not being met; and establish a mechanism for 

maintaining visibility into other investments.

In Stage 3

• Develop and implement policies and procedures for modifying IT 

portfolio selection criteria.

• Develop policies and procedures for using the portfolio selection 

criteria to create its portfolio.

• Develop, review, and modify criteria for assessing portfolio performance 

at regular intervals to reflect current performance expectations. 

• Define and implement processes for carrying out PIRs for all IT 

investments. 
Page 50 GAO-06-11 HHS’s IT Investment Management Process

We also recommend that the HHS Secretary direct the CIO to ensure that 

the plan draws together ongoing efforts and additional efforts that are 

needed to address the weaknesses identified in this report. The plan should 

also (1) specify measurable goals, objectives, and milestones; (2) specify 

needed resources; (3) assign clear responsibility and accountability for 

accomplishing tasks; and (4) be approved by senior management. 

Finally, to improve the department oversight of its component agency 

investment management process, we are recommending that the HHS 

Secretary direct the HHS CIO to establish a mechanism for ensuring 

component agencies define and implement investment management 

processes that are aligned with those of the department.

Agency Comments The Department of Health and Human Services’s Inspector General 

provided written comments on a draft of this report (reprinted in app. II). 

In these comments, HHS generally agreed with our findings and 

recommendations and stated that the report represented a fair assessment 

of the department’s progress in IT investment management. The 

department added that it will leverage the report in its efforts to improve its 

investment management processes.

HHS expressed differing perspectives on the inclusion of component 

agency business representation on the investment review board and the 

performance of postimplementation reviews. Specifically, regarding 

business representation on the board, the department commented that it 

used a hierarchy of investment reviews (with the first review occurring at 

the component agency) combined with ITIRB members representing 

mission support areas, such as Finance, Acquisition, and Human 

Resources, to provide a structure for making the business decisions 

regarding the department’s investments. We disagree with the department 

that this arrangement provides an adequate structure for managing the 

department’s investments. Because allocating resources among major IT 

investments may require fundamental trade-offs among a multitude of 

business objectives, portfolio management decisions are essentially 

business decisions, and therefore require sufficient business 

representation on the board. CIOs and executives responsible for missionsupport functions do not constitute sufficient business representation 

because, by virtue of their responsibilities, they are not in the best position 

to make business decisions. Portfolio management decisions are better 

made by executives with business line decision-making authority.
Page 51 GAO-06-11 HHS’s IT Investment Management Process

Regarding PIRs, HHS commented that it was currently informally 

performing them by conducting closeout reviews of recently implemented 

investments and annual reviews of systems in operations and maintenance. 

PIRs are conducted to determine whether cost, benefit, schedule, and risk 

expectations that were set for investments were achieved and develop 

lessons learned about the investment selection and control processes that 

can be used as the basis for management improvements. However, neither 

the closeout reviews, nor the reviews of systems in operations and 

maintenance, are addressing all these elements. Specifically, as we stated in 

our report, the closeout reviews do not address schedule deviations, 

determine whether the benefits were achieved, or identify lessons learned. 

In addition, the reviews of projects in operations and maintenance do not 

capture the benefits realized or identify lessons learned.

Commenting on departmental-level oversight of component agency 

investments, HHS stated that it agrees with our recommendation to 

improve its oversight of component agency investments. It stated that it 

would use a number of mechanisms to do this, including performing audits 

to ensure alignment of component agency’s processes with those of the 

department, using earned value management data to identify potential 

performance problems with most investments, and directly reviewing 

investments determined to be of high priority. We agree with HHS that 

these steps would help address some of the weaknesses in project 

oversight that we identify in this report. 

As agreed with your office, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days from the 

date of this report. At that time, we will send copies to other interested 

congressional committees, the Secretary of Health and Human Services, 

and other interested parties. We will also make copies available to others 

upon request. In addition, the report will be available at no charge on the 

GAO Web site at http://www.gao.gov. 
Page 52 GAO-06-11 HHS’s IT Investment Management Process

Should you or your offices have questions on matters discussed in this 

report, please contact me at (202) 512-9286 or pownerd@gao.gov. Contact 

points for our Offices of Congressional Relations and Public Affairs may be 

found on the last page of this report. GAO staff who made major 

contributions to this report are listed in appendix III.

Sincerely yours,

David A. Powner

Director, Information Technology Management Issues
Page 53 GAO-06-11 HHS’s IT Investment Management Process

Appendix I

Objectives, Scope, and Methodology ApApenednxidexiIs

The objectives of our review were to (1) assess the Department of Health 

and Human Services’s capabilities for managing its IT investments and (2) 

determine any plans HHS might have for improving those capabilities.

To address our first objective, we reviewed the results of the department’s 

self-assessment of Stages 2 and 3 practices using our ITIM framework and 

validated and updated the results of the self-assessment through document 

reviews and interviews with officials. We reviewed written policies, 

procedures, and guidance and other documentation providing evidence of 

executed practices, including HHS’s Capital Planning and Investment 

Control Policy and Guidelines, standard operating procedures, portfolio 

management tool training manuals, and various instructional 

memorandums. We also reviewed the HHS ITIRB meeting materials, 

including quarterly status reports, meeting minutes, and records of 

decisions. We did not assess progress in establishing the capabilities found 

in Stages 4 and 5 because the department acknowledged that it had not 

executed any of the key practices in higher maturity stages. In addition, we 

conducted interviews with officials from the Office of the CIO, whose main 

responsibility is to oversee and ensure that HHS’s IT investment 

management process is implemented and followed to determine the level 

of oversight and guidance the department is providing to its component 

agencies. We also interviewed the Centers for Medicare & Medicaid’s 

Director for Investment Tracking and Assessment to determine the level of 

investment management guidance and oversight that is provided by the 

department. 

As part of our analysis, we selected two HHS enterprisewide and two 

component agency IT projects as case studies to verify that the critical 

processes and key practices were being applied. The projects selected 

(1) are recognized as major systems, (2) were in different life cycle phases, 

(3) represent a mix of headquarters and component agency investments, 

(4) support different functional areas, and (5) required different levels of 

funding. The four projects are described below: 

• HHS Public Key Infrastructure—This project supports digital signatures 

and other public key-enabled security services; it is intended to be the 

underlying architecture to support secure transmissions of electronic 

communication, such as encrypted email, by linking a digital key to a 

specific person, and issues and manages digital certificates. The intent 

of the project is to provide an identity proofing process that is both fast 

and certificate authority neutral. It is an agencywide strategic initiative 

that provides security services. The project is a major enterprisewide 
Appendix I

Objectives, Scope, and Methodology 

Page 54 GAO-06-11 HHS’s IT Investment Management Process

investment and is in the operations and maintenance phase. The project 

has a planned completion date of July 2011 and is estimated to spend 

$7.7 million for fiscal year 2006.

• HHS Enterprise Architecture Initiative—This initiative is to provide the 

overall framework for planning and managing the technology-supported 

information assets of HHS and give the department the ability to identify 

data and process redundancies and inefficiencies in its information 

systems. The program’s objectives focus on development of operational 

policies and support that enable identification, analysis and ongoing 

management of the business, and information and related technology 

architectures. It is to provide leadership, direction, and support to HHS’s 

component agencies in planning and implementing information systems 

to support required business processes. As of fiscal year 2005, the 

initiative is a major enterprisewide program investment and is estimated 

to spend $15.0 million for fiscal year 2006. 

• National Institutes of Health’s Electronic Research Administration—

This initiative is the National Institutes of Health’s infrastructure for 

conducting interactive electronic transactions for the receipt, review, 

monitoring, and administration of grant awards to biomedical 

investigators worldwide. It is to provide the technology capabilities for 

the agency to efficiently and effectively perform grants administration 

functions. The system is to provide end-to-end support of the grants 

administration process, including receipt of applications, review and 

selection of grantees, financial and progress reporting, issuance of final 

reports and grant dole-out, invention reporting, and interface with 

accounting systems. It is a major component agency investment and is 

expected to have a useful life of 13 years. The project is estimated to 

spend $42.1 million for fiscal year 2006. 

• Food and Drug Administration’s Mission Accomplishment and 

Regulatory Compliance Services—This program is a comprehensive 

redesign and reengineering of core mission-critical systems at the 

agency, including the Field Accomplishments and Compliance Tracking 

System and the Operation and Administration Support System. The first 

of these systems is to support the investigation, tracking of compliance, 

and laboratory operations related to domestic operations under the 

agency’s purview; the second is to primarily support the review and 

decision-making process of products imported into the United States. 

Both are legacy systems that execute on client-server platforms; while 

currently viable, the current systems cannot address many of the 
Appendix I

Objectives, Scope, and Methodology 

Page 55 GAO-06-11 HHS’s IT Investment Management Process

business needs due to the exponential growth in functionality on a rigid 

platform that was not designed to support the extent of change that has 

been required. The Mission Accomplishment and Regulatory 

Compliance Services is a major component agency investment and is 

expected to move to production in September 2007 and have a useful 

life of 10 years. The project is estimated to spend $10.2 million for fiscal 

year 2006. 

For these projects, we reviewed project management documentation, such 

as business cases, status reports, and meeting minutes. We also 

interviewed officials from the Office of the CIO for the two component 

agency investments and the project managers for the two HHS 

enterprisewide projects.

We compared the evidence collected from our document reviews and 

interviews to the key practices in ITIM. We rated the key practices as 

“executed” on the basis of whether the agency demonstrated (by providing 

evidence of performance) that it had met the criteria of the key practice. A 

key practice was rated as “not executed” when we found insufficient 

evidence of a practice during the review or when we determined that there 

were significant weaknesses in HHS’s execution of the key practice. In 

addition, HHS was provided the opportunity to produce evidence for key 

practices rated as “not executed.” 

To address our second objective, we obtained and evaluated documents 

showing what management actions had been taken and what initiatives had 

been planned by the agency. This documentation included the Policy 

Advisory Board charter, draft investment management policies and 

procedures, as well as procedures and guidance for control and evaluate 

functionalities within HHS’s portfolio management tool. We also 

interviewed officials from the Office of the CIO to determine efforts 

undertaken to improve IT investment management processes. 

We conducted our work at HHS headquarters in Washington, D.C., from 

January through September 2005, in accordance with generally accepted 

government auditing standards.
Page 56 GAO-06-11 HHS’s IT Investment Management Process

Appendix II

Comments from the Department of Health and 

Human Services ApendxiI
Appendix II

Comments from the Department of Health 

and Human Services

Page 57 GAO-06-11 HHS’s IT Investment Management Process
Appendix II

Comments from the Department of Health 

and Human Services

Page 58 GAO-06-11 HHS’s IT Investment Management Process
Appendix II

Comments from the Department of Health 

and Human Services

Page 59 GAO-06-11 HHS’s IT Investment Management Process
Page 60 GAO-06-11 HHS’s IT Investment Management Process

Appendix III

GAO Contact and Staff Acknowledgments ApendxiI

GAO Contact David A. Powner (202) 512-9286, pownerd@gao.gov

Staff 

Acknowledgments

In addition to the person named above, Neil Doherty, Joanne Fiorino, 

Sabine Paul, Nik Rapelje, Niti Tandon, and Amos Tevelow made key 

contributions to this report.

(310451)
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go to 

www.gao.gov and select “Subscribe to Updates.”

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to:

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional 

Relations

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548

Public Affairs Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548