GAO

United States General Accounting Office

Report to the Attorney General,

Department of Justice

December 2000 INFORMATION 

TECHNOLOGY

INS Needs to 

Strengthen Its 

Investment 

Management 

Capability

GAO-01-146

Page 1 GAO-01-146 INS’ IT Investments

Contents

Letter 5

Executive Summary 8

Chapter 1

Introduction

20

INS' Current IT Investment Efforts 20

Recent Reviews Have Identified IT Project Management Weaknesses 21

Overview of INS' Current Approach to IT Investment Management 22

Framework for Assessing Agencies' IT Investment Management 24

Objectives, Scope, and Methodology 28

Chapter 2

INS Lacks Foundation 

Capabilities Upon 

Which to Build IT 

Investment 

Management Maturity

32

INS Has Established an IRB, But Has Not Developed Policies and 

Procedures to Govern IRB Operations 34

INS Is Not Effectively Overseeing Its Ongoing IT Projects 36

INS Is Not Tracking and Using IT Asset Information for Investment 

Management Purposes 38

INS Has Not Defined Business Needs for All Its IT Projects 40

INS Has a Structured Process for Selecting New IT Proposals But Has

Not Consistently Analyzed Them According to Established Criteria 42

Chapter 3

INS Is Not Managing Its 

IT Investments as a 

Complete Portfolio

44

INS Has Not Created Useful Portfolio Selection Criteria 46

INS Does Not Analyze Its IT Investments Based on Cost, Benefit, 

Schedule, and Risk Data When Making Investment Decisions 48

INS Does Not Comparatively Assess All Its IT Projects When Making 

Selections for Funding 50

INS Does Not Oversee IT Investments' Cost, Benefit, Schedule, and 

Risk Performance 52

Chapter 4

The Department of 

Justice Is Not Guiding 

and Overseeing INS' 

Investment 

Management Approach

54
Contents

Page 2 GAO-01-146 INS’ IT Investments

Chapter 5

Conclusions, 

Recommendations, 

and Agency Comments

55

Recommendations for Executive Action 55

Agency Comments and Our Evaluation 57

Appendixes Appendix I: Comments From the Immigration and Naturalization

Service 60

Appendix II: GAO Contacts and Staff Acknowledgments 64

Tables Table 1: Summary of Stage Two Critical Process Ratings 33

Table 2: Summary of Ratings and Evidence for the IT Investment

Board Operation Critical Process 35

Table 3: Summary of Ratings and Evidence for the IT Project 

Oversight Critical Process 37

Table 4: Summary of Ratings and Evidence for the IT Asset

Tracking Critical Process 39

Table 5: Summary of Ratings and Evidence for the Business Needs 

Identification for IT Projects Critical Process 41

Table 6: Summary of Ratings and Evidence for the Proposal 

Selection Critical Process 43

Table 7: Summary of Stage Three Critical Process Ratings 45

Table 8: Summary of Ratings and Evidence for the Portfolio 

Selection Criteria Definition Critical Process 47

Table 9: Summary of Ratings and Evidence for the Investment 

Analysis Critical Process 49

Table 10: Summary of Ratings and Evidence for the Portfolio 

Development Critical Process 51

Table 11: Summary of Ratings for the Portfolio Performance 

Oversight Critical Process 53

Figures Figure 1: The Five Stages of Maturity Within ITIM 11

Figure 2: Current INS New IT Proposal Selection Process 23

Figure 3: The ITIM Stages of Maturity With Critical Processes 26

Figure 4: ITIM Component Relationships 27
Contents

Page 3 GAO-01-146 INS’ IT Investments

Abbreviations

CBSR Cost, Benefit, Schedule, and Risk 

CIO Chief Information Officer

CIPRIS Coordinated Interagency Partnership Regulating International 

Students

CIS Central Index System

CLAIMS Computer-Linked Application Information Management System

ESC Executive Steering Committee

IG Inspector General

INS Immigration and Naturalization Service

IRB Investment Review Board

ISIS Integrated Surveillance Intelligence System

IT information technology

ITIB Information Technology Investment Board

ITIM Information Technology Investment Management

LMI Logistics Management Institute

SDLC Systems Development Life Cycle
Page 4 GAO-01-146 INS’ IT Investments
Page 5 GAO-01-146 INS’ IT Investments

United States General Accounting Office

Washington, D.C. 20548 Letr

December 29, 2000

The Honorable Janet Reno

The Attorney General

Dear Madam Attorney General: 

This report addresses the Immigration and Naturalization Service's (INS) 

management of information technology (IT) investments. Each year INS 

invests hundreds of millions of dollars on IT systems and activities. We 

found that INS has established some important capabilities for managing 

these investments, but it has considerable work ahead to fully implement 

mature and effective processes. We are making recommendations to 

strengthen INS' investment management capabilities. 

We are sending copies of this report to Senator Judd Gregg, Chairman, and 

Senator Ernest F. Hollings, Ranking Minority Member, Senate 

Appropriations Subcommittee on Commerce, Justice, State, and Judiciary; 

Senator Spencer Abraham, Chairman, and Senator Edward M. Kennedy, 

Ranking Minority Member, Senate Judiciary Subcommittee on Immigration; 

Representative Harold Rogers, Chairman, and Representative Jose E. 

Serrano, Ranking Minority Member, House Appropriations Subcommittee 

on Commerce, Justice, State, and the Judiciary; Representative Lamar 

Smith, Chairman, and Representative Sheila Jackson Lee, Ranking Minority 

Member, House Judiciary Subcommittee on Immigration and Claims; the 

Honorable Jacob J. Lew, Director, Office of Management and Budget; and 

Mary Ann Wyrsch, Acting Commissioner of the Immigration and
Page 6 GAO-01-146 INS’ IT Investments

Naturalization Service. Copies will also be made available to others upon 

request.

Randolph C. Hite

Director, IT Systems Issues

David L. McClure

Director, IT Management Issues
Page 7 GAO-01-146 INS’ IT Investments
Page 8 GAO-01-146 INS’ IT Investments

Executive Summary

Purpose The Immigration and Naturalization Service (INS), an agency of the 

Department of Justice, invests hundreds of millions of dollars each year in 

information technology (IT) to carry out its core missions of (1) preventing 

aliens from entering the United States illegally and removing aliens who 

succeed in doing so and (2) providing services or benefits to facilitate entry, 

residence, employment, and naturalization of legal immigrants. 

The Clinger-Cohen Act requires agency heads to implement a process for 

maximizing the value and assessing and managing the risks of its IT 

investments. 1

 Our research of leading private and public sector 

organizations' IT management practices indicates that effective investment 

management requires the use of defined and disciplined investment 

management processes.2

 Such structured processes provide a systematic 

method for agencies to minimize risks while maximizing the return on 

investments. Given the importance of IT investment management to INS, 

GAO determined whether (1) INS is effectively managing its IT investments 

and (2) the Department of Justice is effectively promoting, guiding, and 

overseeing INS' investment management activities. 

Background Each year INS invests hundreds of millions of dollars on IT systems and 

activities. According to INS, in fiscal year 2000, it obligated about 

$327 million on IT activities, including about $94 million for development 

and deployment and the remaining amount for operations and 

maintenance, including major enhancements to existing systems. For fiscal 

year 2001, INS plans to spend about $226 million on IT for operations and 

maintenance activities.3

Recent studies have identified significant weaknesses in INS' management 

of its IT resources. In August 1998, the Logistics Management Institute 

(LMI) reported that INS did not track and manage projects to a set of cost, 

1

The fiscal year 1997 Omnibus Consolidated Appropriations Act, P. L. 104-208, renamed both 

Division D (the Federal Acquisition Reform Act) and E (the Information Technology 

Management Reform Act) of the 1996 DOD Authorization Act, P. L. 104-106, as the ClingerCohen Act of 1996.

2

Executive Guide: Improving Mission Performance Through Strategic Information 

Management and Technology (GAO/AIMD-94-115, May 1994).

3

INS has not yet decided how much it will spend in fiscal year 2001 on IT for development 

and deployment activities.
Executive Summary

Page 9 GAO-01-146 INS’ IT Investments

schedule, technical, and benefit baselines.4 LMI noted that while INS had 

defined good procedures for the development of IT projects, it did not 

consistently follow them. Similarly, in July 1999, the Justice Inspector 

General (IG) reported that INS was not adequately managing its 

information systems.5

 In particular, the IG reported that (1) estimated 

completion dates for some IT projects had been delayed without 

explanation for the delays, (2) project costs continued to spiral upward 

with no justification for how funds are spent, and (3) projects were nearing 

completion with no assurance that they will meet performance and 

functional requirements. More recently, in August 2000, GAO reported that 

INS did not have an enterprise architecture (or agencywide blueprint) to 

guide the development of its new and the evolution of its existing 

information systems, and it had not yet established the management 

structure and controls to develop one.6

 An enterprise architecture is a 

Clinger-Cohen Act requirement and a practice of successful public and 

private sector organizations. Until INS has such an architecture, it will be 

unable to fully ensure that the hundreds of millions of dollars it spends 

each year on new and existing information systems will optimally support 

mission needs. As a result, GAO recommended that INS develop a complete 

enterprise architecture, including both a current and target architecture 

and a plan for moving between the two, and that it manage the 

development of the architecture as an agencywide priority.

The Clinger-Cohen Act of 1996 was enacted to address longstanding 

problems related to federal IT management. Among other things, it requires 

agency heads to implement a process for maximizing the value and 

assessing and managing the risks of its acquisitions. A key goal of the 

Clinger-Cohen Act is that agencies have processes and information in place 

to help ensure that IT projects are being implemented at acceptable costs, 

within reasonable and expected time frames, and are contributing to 

tangible, observable, improvements in mission performance.

4

Reengineering Information Technology Management at the Immigration and 

Naturalization Service, Logistics Management Institute, August 1998. LMI is a private, 

nonprofit corporation that provides management consulting, research, and analysis to 

governments and other nonprofit organizations.

5

Follow-up Review: Immigration and Naturalization Service Management of Automation 

Programs, Office of the Inspector General, Audit Division, U.S. Department of Justice, July 

1999.

6

Information Technology: INS Need to Better Manage the Development of Its Enterprise 

Architecture (GAO/AIMD-00-212, August 1, 2000).
Executive Summary

Page 10 GAO-01-146 INS’ IT Investments

In May 2000, GAO issued an Information Technology Investment 

Management (ITIM) maturity framework, which identifies critical 

processes for successful IT investment and organizes these processes into 

a framework of increasingly mature stages. 7

 ITIM supports the 

fundamental requirements of the Clinger-Cohen Act, which calls for IT 

investment and capital planning processes and IT performance 

measurement. ITIM is intended to provide a tool for implementing these 

processes incrementally and effectively. ITIM has been favorably reviewed 

by federal Chief Information Officers (CIOs) and members of GAO's 

advisory council on IT management. 

ITIM is a hierarchical model comprising five different maturity stages. Each 

stage builds upon the lower stages and represents a step toward achieving 

both stable and effective IT investment management processes. With the 

exception of the first stage—which reflects a general absence of 

investment management processes—each maturity stage is composed of 

critical processes that must be implemented and institutionalized for the 

organization to satisfy the requirements of that stage and be able to 

advance to the next stage. These critical processes are further broken 

down into key practices. Key practices are the specific tasks and 

conditions that must be in place for an organization to effectively 

implement the necessary critical processes. Using ITIM, GAO evaluated 

relevant processes in maturity stages two and three.8

 GAO did not assess 

stages four and five because INS acknowledged that it did not have any 

stage four and five capabilities. Figure 1 shows the five ITIM stages and a 

brief description of each stage.

7

Information Technology Investment Management: A Framework for Assessing and 

Improving Process Maturity (Exposure Draft) (GAO/AIMD-10.1.23, May 2000).

8

Stage two critical processes are IT investment board operations, IT project oversight, IT 

asset tracking, business needs identification for IT projects, and proposal selection. Stage 

three critical processes that GAO reviewed are portfolio selection criteria definition, 

investment analysis, portfolio development, and portfolio performance oversight.
Executive Summary

Page 11 GAO-01-146 INS’ IT Investments

Figure 1: The Five Stages of Maturity Within ITIM

Results in Brief INS has limited capability to effectively manage its planned and ongoing IT 

investments. To its credit, INS has some important IT investment 

management capabilities in place to build upon and establish effective 

investment management processes. However, it has considerable work 

ahead to fully implement mature and effective processes. Until INS fully 

implements such processes, it will not know whether it is making the best 

investment decisions to optimize mission performance, whether its 

selected mix of investments best meets its overall mission and business 

priorities, or whether it is adequately managing the risks associated with 

these investments. 

The first major step to building a sound IT investment management process 

is to be able to measure the progress of existing IT projects to identify 

variances in cost, schedule, and performance expectations, and take 

corrective action, if appropriate, and to establish basic capabilities for 

There is little awareness of investment

management techniques. IT management

processes are ad hoc, project-centric, and

have widely variable outcomes.

Repeatable investment control techniques are in

place, and the key foundation capabilities have

been implemented focusing on cost and schedule

activities.

Comprehensive IT investment portfolio selection

and control techniques are in place that

incorporate benefit and risk criteria linked to

mission goals and strategies.

Description

Investment benchmarking and IT-enabled

change management techniques are deployed

to strategically shape business outcomes.

Process evaluation techniques focus on

improving the performance and management

of the organization's IT investment portfolio.

Enterprise

and strategic

focus

Projectcentric

Stage 4

Improving the

Investment Process

Stage 3

Developing a Complete

Investment Portfolio

Stage 2

Building the

Investment Foundation

Stage 1

Creating Investment

Awareness

Stage 5

Leveraging IT for

Strategic Outcomes

Maturity
Executive Summary

Page 12 GAO-01-146 INS’ IT Investments

selecting new IT proposals. INS has made some progress in establishing 

basic selection capabilities. For example, INS has an investment review 

board (IRB), which is comprised of both IT and business senior executives 

and functions as INS' central decision-making body for IT projects. 

However, INS has not yet implemented investment control processes and 

thus does not know if its IT projects are meeting cost, schedule, and 

performance expectations. For example, INS executives do not regularly 

track and monitor the progress of INS' projects toward achieving stated 

commitments by comparing up-to-date progress data with expectations. 

Without this information, INS executives do not have adequate assurance 

that IT projects are being developed on schedule and within budget, and 

whether INS' investments will deliver promised capabilities and benefits.

The second major step toward effective IT investment management 

requires that an organization continually assess proposed and ongoing 

projects as an integrated and competing set of investment options. This 

enables the organization to consider the relative costs, benefits, and risks 

of new proposals along with previously funded investments and identify 

the appropriate mix of IT investments that best meets its mission, 

strategies, and goals. However, INS has not yet implemented a process to 

compare both proposed and ongoing IT investments to determine priorities 

and to make decisions about what projects to fund based on their relative 

costs, benefits, schedule, and risks. As a result, INS executives are unable 

to assess and make trade-offs about the relative merits of spending funds to 

develop new systems, enhance current systems, or continue operating and 

maintaining existing systems. 

Further, the Department of Justice has a vital leadership role to play in 

ensuring that its component agencies, like INS, have effective IT 

investment management capabilities. However, Justice has not issued any 

directive to its bureaus, including INS, on the need to institutionalize 

effective IT investment management capabilities, nor has it issued 

guidance on how to accomplish this, and it has not provided oversight on 

its bureaus' investment management development efforts. During the 

course of our work, however, Justice began drafting IT investment 

management policy and guidance. Justice officials stated that they plan to 

issue the final policy by the end of December 2000 and the guidance by 

March 2001.
Executive Summary

Page 13 GAO-01-146 INS’ IT Investments

Principal Findings

INS Lacks Foundation 

Capabilities Upon Which to 

Build IT Investment 

Management Maturity

To develop overall sound IT investment management capabilities, an 

organization must first be able to control its investments so that they finish 

predictably within established schedule and budget expectations and to 

establish basic capabilities for selecting new IT proposals. To INS' credit, it 

has made some progress in establishing basic selection capabilities. For 

example, INS has established an IRB, which comprises both IT and 

business senior executives. The IRB functions as INS' central decisionmaking body for IT projects and has broad support across the agency for its 

investment decisions. In addition, the IRB has followed a structured 

process for developing and selecting new IT proposals. 

INS has not yet implemented investment control processes needed to 

adequately ensure that its IT projects meet established cost and schedule 

expectations. In particular, INS has not (1) consistently developed and 

maintained project management plans that include cost and schedule 

controls, (2) regularly tracked and monitored its IT projects' performance 

to determine whether they are meeting their cost and schedule 

expectations, and (3) acted to address identified cost and schedule 

performance problems. In addition, INS has not clearly identified business 

needs for each of its projects or trained IT project staff in business needs 

identification. 

According to INS, this lack of effective investment control capabilities 

exists because it has not viewed the need for them as an institutional 

priority. This is evidenced by the fact that INS is still experiencing some of 

the same weaknesses that were identified in earlier reviews. For example, 

INS is still not consistently developing and maintaining project 

management plans and regularly tracking its IT projects' performance to 

determine whether they are meeting their cost and schedule expectations. 

As a result, INS' limited investment control capabilities significantly 

increase the chances that its IT projects will be late, cost more than 

expected, not perform as intended, and not deliver promised business 

value.
Executive Summary

Page 14 GAO-01-146 INS’ IT Investments

INS Is Not Managing IT 

Investments as a Complete 

Portfolio

Once new proposals can be selected and developed on schedule and on 

budget, organizations need to continually assess and manage all of their IT 

projects (i.e., projects that are proposed, under development, and in 

operation) based on expected cost, benefits, schedule, and risk to create a 

complete strategic investment portfolio. Taking such a portfolio 

perspective enables the organization to consider its investments 

comprehensively, considering new proposals along with previously funded 

investments, and identifying the mix of IT investments that best meet its 

mission needs.

However, INS is not effectively managing its IT investments as a complete 

portfolio. While INS has defined portfolio categories and assigned each 

investment (including new and ongoing investments) to one of these 

categories, it has not defined the cost, benefits, schedule, and risk criteria 

to best support its mission and business priorities, and it does not use these 

criteria to select IT projects for funding. Without the use of such criteria, 

INS lacks critical information to examine the mix of new proposals and 

ongoing investments within and across its investment portfolios in order to 

select those investments that best align with mission needs and priorities. 

Further, INS executives have not monitored the performance of each of 

INS' IT investments in its portfolio by comparing actual cost, benefits, 

schedule, and risk data against expectations. According to INS, it has not 

established these investment management capabilities because IT 

investment management has not been an institutional priority. In the 

absence of such investment management capabilities, INS is unable to 

consider the relative merits of all investments, including both new and 

ongoing, to select those investments that best meet its mission needs and 

priorities. 

Justice Is Not Guiding and 

Overseeing INS' Investment 

Management Approach

The Clinger-Cohen Act requires that, among other things, the head of each 

agency implement a process for maximizing the value of the agency's IT 

investments and assessing and managing the risks of its IT investments, 

and that the agency CIO work with the agency head in implementing such a 

process. However, Justice has not provided INS, or any other Justice 

component, direction, guidance, and oversight on IT investment 

management activities. According to Justice officials, Justice had not done 

so because of other competing department priorities, even though the 

department and its components spent about $3 billion on IT in fiscal years 

1999 and 2000.
Executive Summary

Page 15 GAO-01-146 INS’ IT Investments

During the course of our work, Justice began drafting IT investment 

management policy and guidance documents in collaboration with an 

intercomponent working group. The draft policy directs Justice 

components to establish and use an IT investment management process 

and directs the Justice CIO to monitor the components' investment 

management processes through periodic briefings. A supplemental 

guidance document provides procedures for developing an investment 

management process. Justice officials stated that they plan to issue the 

final policy by the end of December 2000 and the guidance by March 2001. 

Without effective guidance and oversight, Justice does not have adequate 

assurance that INS, as well as its other components, have the necessary 

investment management processes in place to maximize the value of their 

IT investments and manage the risks associated with them. 

Recommendations for 

Executive Action

To strengthen INS' investment management capability and address the 

weaknesses discussed in this report, GAO recommends that you direct the 

Commissioner of INS to designate development and implementation of 

effective IT investment management processes as an agencywide priority 

and manage it as such. Specifically, you should direct the Commissioner to 

do the following:

• Develop a plan, within 9 months, for implementing IT investment 

management process improvements that is based on stages two and 

three critical processes and specifies measurable goals and time frames, 

ranks initiatives, defines a management structure for directing and 

controlling the improvements, establishes review milestones, and 

recognizes any direction and guidance that Justice issues. This plan 

should first focus on those critical processes in stage two of ITIM 

because, collectively, they provide the foundation for building a mature 

IT investment management process.

• Submit the plan to the Justice CIO for review and approval.

• Implement the approved plan and report to the Justice CIO, according to 

established review milestones, on progress made against the plan's 

goals and time frames.

Further, because the absence of effective investment management 

processes and an enterprise architecture9

 severely limits INS' ability to 

9

Information Technology: INS Needs to Better Manage the Development of Its Enterprise 

Architecture (GAO/AIMD-00-212, August 1, 2000).
Executive Summary

Page 16 GAO-01-146 INS’ IT Investments

effectively manage its IT investments, GAO recommends that until INS 

develops a complete enterprise architecture and implements the key 

practices associated with stages two and three critical processes, as 

described in this report, you direct the Commissioner to limit requests for 

future appropriations for IT only to efforts that

• support ongoing operations and maintenance, but not major 

enhancements, of existing systems;

• support INS efforts to develop and implement IT investment 

management processes and an enterprise architecture;

• are small, represent low technical risk, and can be delivered in a 

relatively short period of time; or 

• are congressionally mandated.

Further, to improve Justice's guidance and oversight of components' IT 

investment management process activities, GAO also recommends that you 

direct the Justice CIO to follow through on the department's plans to issue 

an IT investment management policy and guidance to the components and 

to ensure that the policy and guidance

• directs Justice components and bureaus, including INS, to develop and 

implement IT investment management processes.

• instructs Justice components and bureaus on how to develop an 

investment management process. This guidance should be based on the 

investment management guidance contained in this report and, at a 

minimum, should include component roles, responsibilities, authorities, 

and policies and procedures for developing an IT investment 

management process.

• directs the Justice CIO to monitor the components' progress in 

developing and establishing an IT investment management process and 

take appropriate action if they are not progressing sufficiently.

Agency Comments and 

GAO's Evaluation

In written comments on a draft of this report, Justice generally agreed with 

our recommendations. However, it offered minor wording modifications on 

two recommendations that it said would increase its ability to fully 

implement them. Justice also disagreed with our finding that Justice is not 

guiding and directing INS' investment management approach. 

Justice generally agreed with our recommendation that INS develop and 

submit to Justice a plan for implementing investment management process 

improvements. However, Justice suggested that the time frame for 
Executive Summary

Page 17 GAO-01-146 INS’ IT Investments

developing the plan be clarified such that INS has 6 months to develop and 

submit its plan to Justice once Justice issues its new IT investment 

management guidance. Because GAO's recommendation directed INS to 

consider any Justice guidance and direction in developing its investment 

management process improvement plan, GAO has modified the 

recommendation to include an additional 3 months to allow time for 

Justice to issue its guidance, which it plans to do in March 2001.

Justice also concurred with GAO's recommendation for INS to limit future 

appropriation requests for IT to certain investment categories because it 

lacks an enterprise architecture and effective investment management 

processes, but it suggested that GAO specify that this recommendation is in 

effect until INS completes its architecture and implements investment 

management processes. Because this is the intent of GAO's 

recommendation, GAO has clarified the recommendation to make this 

explicit.

Further, while INS agreed with GAO's recommendation for Justice to issue 

an investment management policy and guidance to its components, 

including INS, it disagreed with GAO's finding that Justice is not guiding 

and directing INS' investment management approach. According to Justice, 

it has established guidance for all aspects of IT management that its 

components are expected to follow and has a process for overseeing 

components' management of their investments. To support its position, 

Justice cited several examples, such as Justice approval authority of all 

component IT investments with life-cycle cost over $1 million, Justice 

establishment of an IT Investment Board, and Justice meetings with 

components. 

GAO does not agree with Justice's position. While GAO concurs that the 

examples cited by Justice represent important IT management functions to 

be performed in providing management oversight of individual IT 

investments, such management oversight is not the focus of GAO's 

findings, conclusions, and recommendations. Rather, GAO's report 

addresses Justice's efforts to ensure that its components, including INS, 

have each defined and implemented effective IT investment management 

processes. As such, GAO sought evidence from Justice demonstrating that 

it has directed its components to establish such processes, provided 

guidance to its components on how to develop and implement these 

processes, and monitored its components' progress to determine whether 

they are implementing such processes. However, besides the steps that 

Justice initiated during the course of GAO inquiries and plans to take, 
Executive Summary

Page 18 GAO-01-146 INS’ IT Investments

which GAO has described in this report, GAO found no such evidence. 

Moreover, Justice stated in its written comments that it agreed with GAO's 

recommendation for it to provide investment management process 

direction, guidance, and oversight to its components. 

Justice's written comments are discussed in further detail in chapter 5, and 

the full text of its comments is reproduced in appendix I.
Page 19 GAO-01-146 INS’ IT Investments
Page 20 GAO-01-146 INS’ IT Investments

Chapter 1

Introduction Chapte1r

The mission of INS, an agency of the Department of Justice, is to 

administer and enforce the immigration laws of the United States. To 

accomplish this, INS is organized into three core business areas—

enforcement, immigration services, and corporate services. Enforcement 

includes, among other things, conducting inspections of travelers entering 

the United States as they arrive at more than 300 land, sea, and air ports of 

entry; detecting and preventing the smuggling and illegal entry of aliens; 

and identifying and removing persons who have no lawful immigration 

status in the United States. Immigration services, which involve regulating 

permanent and temporary immigration to the United States, include 

granting legal permanent residence status, nonimmigrant status (e.g., 

tourists and students), and naturalization. Corporate services include 

records management, financial management, personnel management, and 

inventory management support for INS activities. 

INS' IT assets play a significant role in (1) receiving and processing 

naturalization and other benefit applications, (2) processing immigrants 

and nonimmigrants entering and leaving the United States, and 

(3) identifying and removing people who have no lawful immigration status 

in the United States. For example, the Computer-Linked Application 

Information Management System (CLAIMS 4) is a centralized case 

management tracking system, that offers support for a variety of tasks 

associated with processing and adjudicating naturalization benefits. In 

addition, the Deportable Alien Control System (DACS) automates many of 

the functions associated with tracking the location and status of illegal 

aliens in removal proceedings, including detention status.

INS' Current IT 

Investment Efforts

INS has multiple efforts underway to develop and acquire new information 

systems and to maintain existing ones. According to INS, in fiscal year 

2000, it obligated about $327 million on IT activities, including about 

$94 million for new development and the remaining amount, which 

includes enhancing existing systems, for operations and maintenance. For 

example, INS obligated $14.5 million in fiscal year 2000 to continue 

development of CLAIMS 4, which supports the processing of applications 

and petitions for immigrant benefits and is intended to fully replace 

CLAIMS 3. In addition, INS obligated about $18 million in fiscal year 2000 to 

further deploy its Integrated Surveillance Intelligence System (ISIS), which 

includes the deployment of intelligent computer aided detection systems, 

unattended ground sensors, and fixed cameras along the northern and 

southern borders to provide around-the-clock visual coverage of the 

border. For fiscal year 2001, INS plans to spend about $226 million on IT for 
Chapter 1

Introduction

Page 21 GAO-01-146 INS’ IT Investments

operations and maintenance activities.1 INS funds most of its IT efforts 

with operation and maintenance funds and currently is developing or 

maintaining 74 information systems. 

Recent Reviews Have 

Identified IT Project 

Management 

Weaknesses

Recent reviews have identified several weaknesses in INS' management of 

its IT projects. For example, in August 1998, the Logistics Management 

Institute (LMI)2

 reported that INS' Office of Information Resources 

Management (OIRM) (1) did not maintain accurate cost estimates for the 

complete life cycle of projects and (2) did not track and manage projects to 

a set of cost, schedule, technical, and benefit baselines.3

 Further, LMI noted 

that while INS' System Development Life Cycle (SDLC) manual provides a 

good model for systems development projects, OIRM did not consistently 

follow it, often bypassing key SDLC phases. 4

Similarly, in July 1999, the Justice Inspector General (IG) reported that 

(1) estimated completion dates for some INS IT projects had been delayed 

without explanation for the delays, (2) project costs continued to spiral 

upward with no justification for how funds are spent, and (3) projects were 

nearing completion with no assurance that they would meet performance 

and functional requirements. 5

Recognizing the need to address these weaknesses, INS established an 

Operational Assessment Team to analyze reported weaknesses and 

recommend specific actions to address them. The Operational Assessment 

Team validated the deficiencies identified in the LMI and Justice IG reports 

and identified additional ones. For example, the team found that system 

1

INS has not yet decided how much it will spend in fiscal year 2001 on IT for development 

and deployment activities.

2

LMI is a private, nonprofit corporation that provides management consulting, research, and 

analysis to governments and nonprofit organizations.

3

Reengineering Information Technology Management at the Immigration and 

Naturalization Service, Logistics Management Institute, August 1998.

4

“System development life cycle” is a term used to refer to the phases of a system's 

development from beginning to end (i.e., from perceived need for a system extending 

through systems design, development, implementation, operations, and maintenance). 

5

Follow-up Review: Immigration and Naturalization Service Management of Automation 

Programs, Office of the Inspector General, Audit Division, U.S. Department of Justice, July 

1999.
Chapter 1

Introduction

Page 22 GAO-01-146 INS’ IT Investments

requirements were not consistently collected, recorded, documented, 

tracked, and controlled. To illustrate, of 105 projects reviewed by the team, 

fewer than 50 percent had documented requirements and most of the 

requirements that had been documented were not current. 

Further, in August 2000, we reported that INS did not have an enterprise 

architecture to guide the development and evolution of its information 

systems.6

 An enterprise architecture is an institutional systems blueprint 

that defines in both business and technological terms the organization's 

current and target operating environments and provides a road map for 

moving from one to the other. It is required by the Clinger-Cohen Act and is 

a recognized practice of successful public and private sector organizations. 

INS had initiated some limited efforts to document its current architecture, 

but it had not yet begun developing a target architecture or a plan to move 

from the current to the target environment. Moreover, INS had not yet 

established the management structure and controls to develop the 

architecture. The absence of such an enterprise architecture increases the 

risk that the hundreds of millions of dollars INS spends each year on 

information systems will not be well integrated or compatible and will not 

effectively support mission needs and priorities.

Overview of INS' 

Current Approach to IT 

Investment 

Management 

In 1997, INS established an investment review board (IRB). The IRB 

consists of four voting members—the Deputy Commissioner (Chair) and 

INS' three Executive Associate Commissioners—and advisory or 

supporting members, including the Director of the Budget Office and the 

Acting Associate Commissioner of the Office of Information Resources 

Management. In November 1998, INS also established the Executive 

Steering Committee (ESC) to support the IRB. The ESC comprises 

portfolio managers and advisory members, which analyze investment 

proposals and make recommendations on these proposals to the IRB. 7

The IRB has established a process for selecting new IT proposals. 

According to INS officials, new proposals are developed throughout the 

6

Information Technology: INS Needs to Better Manage the Development of Its Enterprise 

Architecture (GAO/AIMD-00-212, August 1, 2000).

7

Portfolio managers are individuals who are responsible for managing a group of systems 

within a particular business area or portfolio. INS has defined eight portfolio categories: 

Biometrics, Corporate, Enforcement, Examination, Infrastructure, Inspections, IRM 

Operations, and Management.
Chapter 1

Introduction

Page 23 GAO-01-146 INS’ IT Investments

year as business needs are identified and are forwarded to the appropriate 

portfolio manager for review. After reviewing the proposal, the portfolio 

manager forwards it to the ESC for consideration for funding. The ESC 

examines the proposals submitted and determines the appropriate funding 

for each project. Once funding is determined, the ESC forwards the 

proposed funding levels to the IRB, which makes the final investment 

selections and budget formulation decisions. See figure 2 for INS' new 

proposal selection process.

Figure 2: Current INS New IT Proposal Selection Process

Source: INS.

As part of INS' annual budget execution process, the IRB considers the 

funding requests of ongoing and new projects. Project managers define 

requirements for their ongoing projects, which they submit to the 

responsible portfolio managers for review. After reviewing the 

Project proposal

Portfolio manager

Executive Steering

Committee (ESC)

Investment

Review Board

(IRB)

Justice IT

Investment

Board (ITIB)

No Yes

If required

Reevaluate

Commence

project

life cycle
Chapter 1

Introduction

Page 24 GAO-01-146 INS’ IT Investments

requirements and funding requests, each portfolio manager submits them 

to the ESC for review and to the IRB for approval. The approved funding is 

submitted to the Budget Office for inclusion into its budget execution 

process. According to INS officials, new proposals are considered for 

funding only after ongoing projects have been funded. 

Framework for 

Assessing Agencies' IT 

Investment 

Management

Several recent management reforms—including the revision to the 

Paperwork Reduction Act and the passage of the Clinger-Cohen Act of 

1996, the Government Performance and Results Act of 1993, and the Chief 

Financial Officers Act of 1990—have introduced requirements emphasizing 

the need for federal agencies to improve their management processes for 

selecting and managing IT resources. In particular, the Clinger-Cohen Act 

requires that the head of each agency implement a process for maximizing 

the value of the agency's IT investments and for assessing and managing 

the risks of its acquisitions. A key goal of the Clinger-Cohen Act is that 

agencies have processes and information in place to help ensure that 

projects are being implemented at acceptable costs within reasonable and 

expected time frames and that they are contributing to tangible, observable 

improvements in mission performance.

We and the Office of Management and Budget (OMB) have developed 

guidance to assist federal agencies in managing IT investments. One such 

guide, Assessing Risks and Returns: A Guide for Evaluating Federal 

Agencies' IT Investment Decision-making, incorporates our analysis of 

the management practices of leading private and public sector 

organizations as well as the provisions of major federal legislation (e.g., 

Clinger-Cohen Act) and executive branch guidance that address investment 

decision-making.8

 The guide provides a method for determining how well a 

federal agency is selecting and managing its IT resources and identifies 

specific areas where improvements can be made.

To enhance this guidance, we issued an Information Technology 

Investment Management (ITIM) maturity framework in May 2000.9

 ITIM 

provides a common framework for assessing IT capital planning and 

investment management practices by describing the organizational 

8

GAO/AIMD-10.1.13, February 1997.

9

Information Technology Investment Management: A Framework for Assessing and 

Improving Process Maturity (Exposure Draft) (GAO/AIMD-10.1.23, May 2000).
Chapter 1

Introduction

Page 25 GAO-01-146 INS’ IT Investments

processes, and their interrelationships that are the tenets of good 

investment management. ITIM is based on the best-practices work done as 

part of our ongoing research into the IT management practices of leading 

organizations.

ITIM is a hierarchical model comprising five maturity stages. These 

maturity stages represent steps toward achieving stable and mature 

investment management processes. As agencies advance through the 

model's stages, their capability to manage IT increases. Each stage builds 

upon the lower stages and enhances the organization's ability to manage its 

investments. With the exception of the first stage, each maturity stage is 

composed of critical processes that must be implemented and 

institutionalized for the organization to satisfy the requirements of that 

stage. These critical processes are further broken down into key practices 

that describe the types of activities that an agency should be engaged in to 

successfully implement each critical process. An organization that has 

these critical processes in place is in a better position to successfully invest 

in IT. (See figure 3 for the five stages and associated critical processes).
Chapter 1

Introduction

Page 26 GAO-01-146 INS’ IT Investments

Figure 3: The ITIM Stages of Maturity With Critical Processes

As established by the model, each critical process contains five core 

elements that indicate whether the implementation and institutionalization 

of a process can be effective and repeated. The five core elements are: 

• Purpose: This is the primary reason for engaging in the critical process 

and states the desired outcome for the critical process.

• Organizational commitment: This comprises management actions that 

ensure that the critical process is established and will endure. Key 

practices typically involve establishing organizational policies and 

engaging senior management sponsorship.

• Prerequisites: These are the conditions that must exist within an 

organization to successfully implement a critical process. This typically 

involves allocating resources, establishing organizational structures, 

and providing training.

• Activities: These are the key practices necessary to implement a critical 

process. An activity occurs over time and has recognizable results. Key 

Investment Process Benchmarking

IT-Driven Strategic Business Change

Post-Implementation Reviews and Feedback

Portfolio Performance Evaluation and Improvement

Systems and Technology Succession Management

Authority Alignment of IT Investment Boards

Portfolio Selection Criteria Definition

Investment Analysis

Portfolio Development

Portfolio Performance Oversight

IT Investment Board Operation

IT Asset Tracking

IT Project Oversight

Business Needs Identification for IT Projects

Proposal Selection

IT spending without disciplined investment

processes

Stage 4

Improving the

Investment

Process

Stage 3

Developing

a Complete

Investment Portfolio

Stage 2

Building the

Investment

Foundation

Stage 1

Creating

Investment

Awareness

Stage 5

Leveraging IT

for Strategic

Outcomes

Maturity stages

Critical processes
Chapter 1

Introduction

Page 27 GAO-01-146 INS’ IT Investments

practices typically involve establishing procedures, performing and 

tracking the work, and taking corrective actions as necessary.

• Evidence of performance: This comprises artifacts, documents, or other 

evidence that supports a contention that the key practices within a 

critical process have or are being implemented. This core element 

typically consists of the collection and verification of physical, 

documentary, or testimonial evidence and typically involves reviews by 

objective parties.

With the exception of the purpose core element, each of the other core 

elements contains key practices. The key practices are the attributes and 

activities that contribute most to the effective implementation and 

institutionalization of a critical process. (Figure 4 shows the relationship 

between the various ITIM components.)

Figure 4: ITIM Component Relationships

Maturity

stage

Maturity

stage

Critical

process

Critical

process

Core

element

Core

element

Key

practice

Key

practice

contains

contains

organized by

The five maturity stages represent the steps

toward achieving a mature, comprehensive

IT investment management process.

The five maturity stages represent the steps

toward achieving a mature, comprehensive

IT investment management process.

Key practices are the tasks within a core

element that must be performed by an

organization in order to effectively implement

and institutionalize a critical process.

Key practices are the tasks within a core

element that must be performed by an

organization in order to effectively implement

and institutionalize a critical process.

The core elements provide the common framework

for each critical process. The five types of core

elements are purpose, organizational commitment,

prerequisites, activities, and evidence of performance.

The core elements provide the common framework

for each critical process. The five types of core

elements are purpose, organizational commitment,

prerequisites, activities, and evidence of performance.

With the exception of Stage 1, each maturity

stage is composed of critical processes, which

must be implemented to attain that stage.

With the exception of Stage 1, each maturity

stage is composed of critical processes, which

must be implemented to attain that stage.
Chapter 1

Introduction

Page 28 GAO-01-146 INS’ IT Investments

Objectives, Scope, and 

Methodology

Our objectives were to determine whether (1) INS is effectively managing 

its IT investments and (2) the Department of Justice is effectively 

promoting, guiding, and overseeing INS' investment management activities. 

To determine whether INS is effectively managing its investments, we 

applied our ITIM framework and the associated assessment method. As 

part of the ITIM assessment method, INS conducted a self-assessment of its 

IT investment management activities using the ITIM framework. In its selfassessment, INS indicated whether it executed each of the key practices in 

stages two through five. INS asserted that it executed many of the key 

practices within stages two and three but only four key practices in all of 

stages four and five. Accordingly, we did not include ITIM stages four and 

five in the scope of our review. Also, we did not evaluate the key practices 

within stages two and three that INS stated it had not executed. 

We evaluated INS against 9 of the 10 critical processes in stages two and 

three. We did not evaluate INS against the stage three critical process 

Authority Alignment of IT Investment Boards. This critical process is only 

relevant if an organization has more than one IT investment board and INS 

has only one. The nine critical processes we examined focus primarily on 

INS' ability to effectively select and control its IT investments. 

To determine whether INS had implemented these nine critical processes, 

we evaluated policies, procedures, and guidance related to INS' IT 

investment management activities. In particular, we analyzed the following: 

organizational charters, INS' System Development Life Cycle manual, 

requirements management process guide, and administrative manuals (e.g., 

Personal Property Handbook). We also reviewed documentation 

associated with specific investment management activities, such as IRB 

and ESC meeting minutes, project management plans, system deployment 

plans, budget formulation and execution plans, quarterly reports to Justice, 

and contractor statements of work. 

In addition, we reviewed four IT projects to verify the execution of INSdefined processes, procedures, and practices. The four projects were 

selected based on the following criteria: (1) the projects should represent 

different life cycle phases (e.g., requirements definition, design, operations 

and maintenance), (2) the projects should support different INS business 

areas (e.g., Examinations, Enforcement), (3) at least one project should be 

considered high risk, and (4) at least one project should have been 
Chapter 1

Introduction

Page 29 GAO-01-146 INS’ IT Investments

reviewed by Justice's Information Technology Investment Board (ITIB). 

The projects we evaluated are:

• Coordinated Interagency Partnership Regulating International 

Students (CIPRIS): CIPRIS is an Internet-based system that is intended 

to modernize and streamline the current process for collecting 

information relating to nonimmigrant foreign students and other 

exchange program participants. It is intended to enable U. S. 

universities, schools, and cultural exchange programs to report and 

share information electronically with INS and other government 

regulatory agencies. INS has implemented an operational prototype of 

CIPRIS at 21 educational institutions. CIPRIS is a concept exploration 

project that supports the Examinations business area within INS. INS 

has designated CIPRIS as a high-risk project and it has been reviewed by 

Justice's ITIB. According to INS, it obligated about $3.1 million for 

CIPRIS in fiscal year 2000.

• Computer-Linked Application Information Management System 

(CLAIMS) 4.0: According to INS, CLAIMS 4 is intended to improve 

delivery of naturalization services by fully automating INS' case 

management system. According to INS, CLAIMS 4 supports the 

Immigration Services Program within INS and is currently operational at 

59 sites. According to INS, it obligated $14.5 million for CLAIMS 4 in 

fiscal year 2000.

• Integrated Surveillance Intelligence System (ISIS): ISIS was 

established to detect and deter illegal intruders and to safely apprehend 

illegal aliens on the U.S.-Mexico and U.S.-Canada borders. ISIS is 

designed to provide all-weather sensor and video surveillance of the 

U.S. borders 24 hours a day, 7 days a week. The major components of 

ISIS are the Intelligent Computer-Assisted Detection system, ground 

sensors, and the Remote Video Surveillance system. ISIS supports the 

Enforcement program area within INS and has been reviewed by 

Justice's ITIB. According to INS, it obligated about $18 million for ISIS 

in fiscal year 2000 to further deploy the system. 

• Central Index System (CIS): CIS provides INS with information about 

persons of interest to the INS. According to INS, CIS also interacts with 

various INS databases to provide the data necessary for INS operations. 

CIS currently maintains approximately 45 million detailed records on 

individuals of interest to INS. CIS supports the INS' Corporate business 

area and is in the operations and maintenance phase of its life cycle. 

According to INS, it obligated about $2.6 million for CIS in fiscal year 

2000.
Chapter 1

Introduction

Page 30 GAO-01-146 INS’ IT Investments

We did not validate INS' IT spending obligations for fiscal year 2000 and IT 

spending estimates for fiscal year 2001.

To supplement our document reviews, we interviewed senior INS officials, 

including the Deputy Commissioner, who chairs the IRB, and the Executive 

Associate Commissioner for Management, who is the Chief Information 

Officer (CIO) and an IRB member. We also interviewed the Acting 

Associate Commissioner for Information Resources Management, who 

chairs the ESC; the Director of INS' Investment Management Team; 

portfolio managers; the Director of the Office of Strategic Information and 

Technology Development; IT project managers; program managers; Office 

of Budget representatives; and officials involved with the development and 

maintenance of INS' asset tracking systems. 

We compared the evidence collected from our document review and 

interviews to the key practices and critical processes in ITIM. Because 

ITIM is a hierarchical framework, the rating of each critical process is 

dependent on the key practices below it. Therefore, we first rated the key 

practices. In accordance with the ITIM assessment method, we rated a key 

practice as “executed” when we determined, by consensus, that INS was 

executing the key aspects of the practice. A key practice was rated as “not 

executed” when we determined that there were significant weaknesses in 

INS' execution of the key practice and INS offered no adequate alternative, 

or when the team found no evidence of a practice during the review. 

Once the key practices were rated, we rated each of the nine critical 

processes we reviewed. A critical process was rated as “implemented” if all 

of the underlying key practices were rated as being executed. A critical 

process was rated as “not implemented, but improvements underway” if 

over half, but not all, of its underlying key practices were rated as being 

executed. A critical process was rated as “not implemented” when there 

were significant weaknesses (i.e., fewer than 50 percent of the key 

practices had been implemented) in INS' implementation of the underlying 

key practices and no adequate alternative was in place.

To determine whether the Department of Justice is effectively promoting, 

guiding, and overseeing INS' investment management activities, we 

interviewed officials within the Office of Information Management and 

Security Staff, the organization that plays a leading role in Justice's 

investment management activities. We also reviewed Justice's January 2000 

investment management guidance, draft policy and guidance documents, 

INS project proposals, ITIB review and decision documentation, and 
Chapter 1

Introduction

Page 31 GAO-01-146 INS’ IT Investments

quarterly briefing documents. We also discussed Justice's oversight 

activities with various officials within INS.

We conducted our work at INS and Justice headquarters in Washington, 

D.C., from May 2000 through October 2000 in accordance with generally 

accepted government auditing standards. Justice's Assistant Attorney 

General for Administration provided written comments of a draft of this 

report. These comments are presented in chapter 5 and are reprinted in 

appendix I. 
Page 32 GAO-01-146 INS’ IT Investments

Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity Chapte2r

The primary purpose of ITIM stage two maturity is to attain repeatable, 

successful IT project-level investment control processes and basic 

selection processes. For an organization to develop an overall sound IT 

investment management process, it must first be able to control its 

investments so that it can identify expectation gaps early and correct them. 

According to ITIM, stage two maturity includes (1) defining IRB operations, 

(2) developing a basic process for selecting new IT proposals, 

(3) developing project-level investment control processes, (4) creating an 

IT asset inventory, and (5) identifying the business needs for each IT 

project.

INS has not fully implemented any of the critical processes associated with 

stage two; however, it has improvements underway and is close to fully 

implementing two of these processes. INS has (1) established an IRB, 

which comprises both IT and business senior executives and functions as 

INS' central decision-making body for IT projects, and (2) the IRB has 

followed a structured process for developing and selecting new IT 

proposals and making initial funding decisions for these proposals. 

However, INS has not yet developed some of the capabilities necessary to 

build a sound IT investment management process. For example, INS has 

not (1) established basic project-level control processes to ensure that its 

IT projects are performing as expected, (2) created an IT asset inventory 

for investment management, and (3) defined business needs for all of its IT 

projects. According to INS, it lacks these critical investment capabilities 

because it has not yet made IT investment management an institutional 

priority. Table 1 summarizes INS' stage two maturity. 
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 33 GAO-01-146 INS’ IT Investments

Table 1: Summary of Stage Two Critical Process Ratings

INS' capabilities for each of the stage two critical processes are discussed 

below.

Critical process Rating Key practices

Key practices

executed

IT Investment Board 

Operation

Not implemented, but 

improvements underway

6 4

IT Project Oversight Not implemented 11 2

IT Asset Tracking Not implemented 8 2

Business Needs 

Identification for IT 

Projects

Not implemented 8 4

Proposal Selection Not implemented, but 

improvements underway

6 5

Total 39 17
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 34 GAO-01-146 INS’ IT Investments

INS Has Established an 

IRB, But Has Not 

Developed Policies and 

Procedures to Govern 

IRB Operations

The purpose of this critical process is to define and establish the governing 

board or boards responsible for selecting, controlling, and evaluating IT 

investments. This includes defining the membership, guiding policies, 

operations, roles and responsibilities, and authorities for the investment 

board and, if appropriate, each board's support staff. These policies, roles 

and responsibilities, and authorities also provide the basis for the board's 

investment selection, control, and evaluation activities. 

According to ITIM, effective IT investment board operations require, 

among other things, that (1) the board membership include both IT and 

business knowledge, (2) the organization's executives and line managers 

support and carry out board decisions, (3) the organization create an 

organization-specific process guide that includes policies and procedures 

to direct the board's operations, and (4) the IRB operate according to these 

written policies and procedures.

INS is executing many of the practices in this critical process. For example, 

INS has an IRB that functions as a central decision-making body for IT 

investments and is composed of senior executives from both INS' IT and 

business areas. During our discussions with agency officials, we found 

broad support within the organization for the IRB's decisions. For example, 

three of the four program/project managers we interviewed acknowledged 

the IRB's role in investment decision-making. The IRB is chaired by the 

Deputy Commissioner and includes INS' three Executive Associate 

Commissioners. The IRB is supported by an ESC, which is comprised of 

senior representatives who manage INS' eight IT portfolios. The ESC 

reviews and analyzes IT investments and makes recommendations to the 

IRB for final approval. This senior level involvement and the breadth of 

representation help to demonstrate executive sponsorship of the process 

and support for the projects selected.

While INS has an IRB, it is not functioning according to written policies and 

procedures. Instead, the IRB operates according to undocumented 

procedures for selecting new IT proposals. According to the Director of 

INS' Investment Management Team, INS has begun developing written 

policies and procedures and plans to complete them about March 2001. 

However, until INS develops and implements these policies and 

procedures, key IT investment activities may not be done consistently, if at 

all. Table 2 summarizes the ratings for each key practice and the specific 

findings supporting the ratings.
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 35 GAO-01-146 INS’ IT Investments

Table 2: Summary of Ratings and Evidence for the IT Investment Board Operation Critical Process

Key practice Rating Summary of evidence

Organizational 

commitment

1. An organization-specific IT 

investment process guide is created 

to direct each board's operations.

Not executed INS does not have an organization-specific IT investment 

process guide to direct the board's operations. According 

to the Director of INS' Investment Management Team, 

INS is developing a process guide and plans to complete 

it by March 2001.

2. Organization executives and line 

managers support and carry out IT 

investment board decisions. 

Executed INS executives and line managers support the IRB's 

decisions. Three of the four program/project managers 

we interviewed acknowledged the IRB's role in 

investment decision-making.

Prerequisites 1. Adequate resources are provided 

for operating each IT investment 

board. 

Executed INS has adequate resources for operating the investment 

board. Resources include both internal staff support and 

contractor-provided support. Also, the IRB has been 

operating as a decisional body since the Fall of 1998.

2. Board members understand the 

investment board's policies and 

procedures and exhibit core 

competencies in using the IT 

investment approach via training, 

education, or experience.

Executed IRB members understand the IRB's informal practices for 

investing in IT and exhibit competencies in using the IT 

investment approach.

Activities 1. Each investment board is created 

and defined with board membership 

integrating both IT and business 

knowledge. 

Executed IRB membership includes representatives from both IT 

and business areas within INS.

2. Each IT investment board 

operates according to written 

policies and procedures in the 

organization-specific IT investment 

process guide. 

Not executed The IRB does not operate according to written 

procedures. INS does not have an organization-specific 

IT investment process guide (See organizational 

commitment 1). However, the IRB operates according to 

undocumented, established procedures for selecting new 

IT proposals.
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 36 GAO-01-146 INS’ IT Investments

INS Is Not Effectively 

Overseeing Its Ongoing 

IT Projects

The purpose of project oversight is to ensure that the IRB provides 

effective oversight for its ongoing IT projects throughout all phases of their 

life cycle. Under stage 2 maturity, the IRB should review each project's 

progress toward predefined cost and schedule expectations, using 

established criteria, and take corrective actions when cost estimates and 

project milestones are not achieved. Implementing this critical process 

provides the basis for evolving the organization's IT investment control 

activities. 

According to ITIM, effective project oversight requires, among other things, 

(1) having written polices and procedures for project management, 

(2) developing and maintaining an approved project management plan for 

each IT project, (3) having written policies and procedures for oversight of 

IT projects, (4) making up-to-date cost and schedule data for each project 

available to the IRB, (5) reviewing each project's performance by 

comparing actual cost and schedule data to expectations regularly, and 

(6) ensuring that corrective actions for each underperforming project are 

defined, implemented, and tracked until the desired outcome is achieved. 

INS is not effectively overseeing its IT projects. While INS has documented 

policies and procedures for project management in its System 

Development Life Cycle (SDLC) manual, it is not following its own 

procedures. For example, INS has not developed and maintained project 

management plans that include cost and schedule controls for each of its IT 

projects, an SDLC requirement. In fact, only two of the four projects that 

we reviewed had current project management plans. 

Furthermore, INS does not have written polices and procedures for 

oversight of its IT projects. Without written polices and procedures, INS 

increases the risk that project oversight activities will not be performed 

effectively. For example, the IRB does not (1) receive up-to-date cost and 

schedule data for each project, (2) oversee each project's performance 

regularly by comparing actual cost and schedule data to expectations, and 

(3) ensure that corrective actions are implemented and tracked for 

underperforming projects. In the absence of effective oversight, INS 

executives do not have adequate assurance that IT projects are being 

developed on schedule and within budget. Table 3 summarizes the ratings 

for each key practice and the specific findings supporting the ratings.
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 37 GAO-01-146 INS’ IT Investments

Table 3: Summary of Ratings and Evidence for the IT Project Oversight Critical Process

Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has written 

policies and procedures for project 

management.

Executed INS' SDLC manual contains policies and procedures for 

project management. 

2. The organization has written 

policies and procedures for 

management oversight of IT 

projects.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.” 

Prerequisites 1. Adequate resources are provided 

to assist the board(s) in overseeing 

IT projects.

Not executed According to INS, it does not have adequate resources to 

assist the board in overseeing IT projects. 

2. Each IT project has and 

maintains an approved project 

management plan that includes cost 

and schedule controls.

Not executed According to INS' officials, not all projects have project 

management plans. Two of the four case study projects 

we reviewed did not have project management plans.

3. An IT investment board is 

operating.

Executed The IRB is functioning as the central decision-making 

body for IT projects, although it is not operating 

according to written policies and procedures (See IT 

Investment Board Operation: organizational commitment 

1).

4. Information from the IT asset 

inventory is used by the IT 

investment board as applicable.

Not executed The IRB does not use information from an IT asset 

inventory.

Activities 1. Each project's up-to-date cost 

and schedule data are provided to 

the appropriate IT investment board.

Not executed Up-to-date cost and schedule data are not provided to 

the IRB for each project. Three of the four projects we 

reviewed did not provide up-to-date cost and schedule 

data to the IRB.

2. Using established criteria, the IT 

investment board oversees each IT 

project's performance regularly by 

comparing actual cost and schedule 

data to expectations.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

3. The IT investment board performs 

special reviews of projects that have 

not met predetermined performance 

standards.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

4. Appropriate corrective actions for 

each under performing project are 

defined, documented, and agreed to 

by the IT investment board and the 

project manager.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

5. Corrective actions are 

implemented and tracked until the 

desired outcome is achieved.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 38 GAO-01-146 INS’ IT Investments

INS Is Not Tracking 

and Using IT Asset 

Information for 

Investment 

Management Purposes

The purpose of the asset tracking critical process is to create and maintain 

an IT asset inventory to assist in managerial decision-making. To make 

good investment decisions, an organization must know where its IT assets 

(i.e., personnel, systems, applications, hardware, software licenses, etc.) 

are located and how funds are being expended toward acquiring, 

maintaining, and deploying them. This critical process identifies IT assets 

within the organization and creates a comprehensive inventory of them. 

This inventory can take many forms, but regardless of form, the inventory 

should identify each asset and its associated components. 

Beyond identifying IT assets, this process is used to support other ITIM 

critical processes by serving as an investment information and data 

repository that contains such items as the list of systems and projects and 

data on each project's progress toward achieving its plans. To support 

investment decision-making, this inventory should also be accessible 

where it is of the most value to decisionmakers.

According to ITIM, effectively tracking IT assets requires, among other 

things, (1) making investment information available on demand to 

decisionmakers, (2) developing and maintaining an IT asset inventory 

according to written procedures, (3) overseeing the development and 

maintenance of the asset tracking process, and (4) assigning responsibility 

for managing this tracking process.

INS has not implemented an effective IT asset tracking process for 

investment management. While investment information from various 

sources has been available to the IRB on an ad hoc basis, it is not available 

on demand and INS has not developed and maintained an inventory for 

investment management purposes according to written policies and 

procedures. In addition, the IRB does not oversee IT asset tracking 

activities and has not assigned responsibility for managing this tracking 

process to support investment decision-making. In the absence of standard, 

documented procedures for developing and maintaining the inventory, INS 

executives do not have adequate assurance that timely, complete, and 

consistent asset data are available to them. Table 4 summarizes the ratings 

for each key practice and the specific findings supporting the ratings.
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 39 GAO-01-146 INS’ IT Investments

Table 4: Summary of Ratings and Evidence for the IT Asset Tracking Critical Process

IT Asset Tracking Rating Summary of evidence

Organizational 

commitment

1. The organization has written 

policies and procedures for 

developing and maintaining an IT 

asset inventory. 

Not executed INS does not have written policies and procedures for 

developing and maintaining an IT asset inventory to 

support investment management. 

2. An official is assigned responsibility 

for managing the IT asset tracking 

process.

Not executed INS has not assigned responsibility for managing the IT 

asset tracking process to support IT investment decisionmaking.

Prerequisites 1. Adequate resources are provided 

for performing the IT asset tracking 

activities.

Executed According to INS, it has adequate resources for 

performing IT asset-tracking activities. 

2. An IT investment board exists and 

oversees the development and 

maintenance of IT asset tracking 

activities.

Not executed The IRB does not oversee the development and 

maintenance of IT asset-tracking activities.

Activities 1. The organization's IT asset 

inventory is developed and 

maintained according to a written 

procedure.

Not executed INS has not developed an IT asset inventory to support IT 

investment decision-making. 

2. IT asset inventory changes are 

maintained according to a written 

procedure.

Not executed INS has not developed an IT asset inventory to support IT 

investment decision-making.

3. Investment information is available 

on demand to decisionmakers and 

other affected parties.

Executed Investment information is available to decisionmakers on 

an ad hoc basis from various repositories. 

4. Historical IT asset inventory 

records are maintained for future 

selections and assessments.

Not executed INS has not developed an IT asset inventory to support IT 

investment decision-making.
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 40 GAO-01-146 INS’ IT Investments

INS Has Not Defined 

Business Needs for All 

Its IT Projects 

The purpose of defining business needs for each IT project is to ensure that 

each project supports the organization's business needs and meets users' 

needs. Thus, this critical process creates the link between the 

organization's business objectives and its IT management strategy. 

According to ITIM, effectively identifying business needs requires, among 

other things, (1) defining the organization's business needs or stated 

mission goals, (2) identifying users for each project who will participate in 

the project ‘s development and implementation, (3) defining business 

needs for each project, and (4) training IT staff in business needs 

identification.

INS has executed some of the key practices associated with effectively 

defining business needs for IT projects. For example, INS has (1) defined 

its business needs and mission goals in its annual performance plan and 

(2) identified users for its projects who participate in the project ‘s 

development and implementation. However, INS has not clearly defined 

specific business needs for each project. In addition, only one of the four 

project managers that we interviewed stated that he or she had been 

trained in business needs identification. In the absence of documented 

business needs, the IRB cannot ensure that it is selecting IT investments 

that meet its mission needs and priorities. Table 5 summarizes the ratings 

for each key practice and the specific findings supporting the ratings.
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 41 GAO-01-146 INS’ IT Investments

Table 5: Summary of Ratings and Evidence for the Business Needs Identification for IT Projects Critical Process

Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has written 

policies and procedures for identifying 

the business needs (and the 

associated users) of each IT project.

Executed The SDLC manual and a requirements management 

process guide contain written policies and procedures 

for identifying the business needs and associated users 

of each IT project. The Technical Bulletin for Chartering 

User Groups for Automation Projects defines procedures 

for identifying the associated users of IT projects. 

However, three of the four program managers whom we 

interviewed were not aware of these policies and 

procedures.

Prerequisites 1. Adequate resources are provided 

for identifying business needs and 

associated users.

Not executed According to INS, it does not have adequate resources 

to identify business needs and associated users. While 

the Office of Strategic Information and Technology 

Development has been assigned responsibility for doing 

this, it is not fully staffed.

2. The organization has defined 

business needs or stated mission 

goals.

Executed INS' mission goals are defined in its annual performance 

plan. 

3. IT staff are trained in business 

needs identification.

Not executed IT staff are not consistently trained in business needs 

identification. Only one of the four IT project managers 

whom we interviewed stated that he or she was trained 

in business needs identification.

4. All IT projects are identified in the IT 

asset inventory.

Not executed INS has a list of IT projects. However, it is not maintained 

as part an IT asset inventory.

Activities 1. The business needs for each IT 

project are clearly identified and 

defined.

Not executed According to INS officials, business needs for each IT 

project are not always identified and identified business 

needs are not always clear. Two of the four case study 

projects we reviewed did not have identified business 

needs.

2. Specific users are identified for 

each IT project.

Executed The four case study projects we reviewed did have 

identified users. 

3. Identified users participate in 

project management throughout a 

project's life cycle.

Executed For the four case study projects we reviewed, identified 

users do participate in project management throughout 

the life cycle.
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 42 GAO-01-146 INS’ IT Investments

INS Has a Structured 

Process for Selecting 

New IT Proposals But 

Has Not Consistently 

Analyzed Them 

According to 

Established Criteria 

The purpose of proposal selection is to establish a structured process for 

selecting new IT proposals. According to ITIM, effective proposal selection 

requires, among other things, (1) designating an official to manage the 

proposal selection process, (2) using a structured process to develop new 

IT proposals, (3) making funding decisions for new IT proposals according 

to an established selection process, and (4) analyzing and ranking new IT 

proposals according to established selection criteria, including cost and 

schedule criteria.

INS has established a structured process for selecting new IT proposals. 

The Deputy Commissioner, as the Chair of the IRB, is designated to manage 

INS' proposal selection process. In addition, INS uses a structured process 

to develop new proposals and makes initial funding decisions for these 

proposals. However, INS has not consistently analyzed and ranked these 

proposals according to established selection criteria. Established selection 

criteria would assist IT managers in creating proposals that best meet the 

needs and priorities of INS. Table 6 summarizes the ratings for each key 

practice and the specific findings supporting the ratings.
Chapter 2

INS Lacks Foundation Capabilities Upon 

Which to Build IT Investment Management 

Maturity

Page 43 GAO-01-146 INS’ IT Investments

Table 6: Summary of Ratings and Evidence for the Proposal Selection Critical Process

Key practice Rating Summary of evidence

Organizational 

commitment

1. Executives and managers follow 

an established selection process.

Executed INS' IRB follows an established process for submitting 

and selecting IT proposals.

2. An official is designated to 

manage the proposal selection 

process.

Executed INS' Deputy Commissioner, as Chair of the IRB, is 

designated to manage the proposal selection process.

Prerequisites 1. Adequate resources are provided 

for proposal selection activities.

Executed According to INS, it has adequate resources for proposal 

selection activities. These resources include the ESC 

members, IRB support staff, and contractor support.

Activities 1. The organization uses a 

structured process to develop new 

IT proposals.

Executed INS uses a structured process to develop new IT 

proposals. This process involves submitting new 

proposal templates (IRB presentation packages) to 

portfolio managers who evaluate them and forward them 

to the ESC. The ESC recommends proposals to the IRB 

for review and approval. 

2. Executives analyze and prioritize 

new IT proposals according to 

established selection criteria.

Not executed Executives do not consistently analyze and rank new IT 

proposals according to established selection criteria.

3. Executives make funding 

decisions for new IT proposals 

according to an established 

process.

Executed An established process has been used to make funding 

decisions. 
Page 44 GAO-01-146 INS’ IT Investments

Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio Chapte3r

An IT investment portfolio is a collection of investments that are assessed 

and managed based on common criteria. While an organization may have 

more than one level of investment portfolios, it should always have an 

enterprisewide portfolio. Managing investments as a portfolio is a 

conscious, continuous, and proactive approach to expending limited 

resources on all competing initiatives in light of the relative beneficial 

effects of these investments. Taking an enterprisewide portfolio 

perspective enables an organization to consider its investments 

comprehensively so that the investments address its mission, strategic 

goals, and objectives. A portfolio approach also allows an organization to 

determine priorities and make decisions about which projects to fund 

based on analyses of the relative costs, benefits, and risks of all projects, 

including projects that are proposed, under development, and in operation.

The purpose of ITIM stage three maturity is to create and manage IT 

investments as a complete enterprise investment portfolio. Once ongoing 

projects can be implemented on schedule and within budget as is 

emphasized in stage two, the organization is capable of managing its 

projects as an investment portfolio. According to ITIM, stage three maturity 

includes (1) defining portfolio selection criteria, (2) engaging in projectlevel investment analysis, (3) developing a complete portfolio based on the 

investment analysis, and (4) maintaining oversight over the investment 

performance of the portfolio.

INS has not implemented any of the critical processes in stage three. In 

general, INS has not created the associated policies and procedures to 

initiate or perpetuate any of the critical processes, and as a result, it has not 

systematically collected and analyzed the data needed to make sound and 

informed decisions about competing investment choices, which 

consciously consider value and risk. In addition, while INS has established 

eight portfolio categories, it has not established an enterprisewide 

investment portfolio. Therefore, decisions may be made between 

competing investments within a business area, but INS cannot make tradeoffs between investments across the enterprise to determine which 

projects contribute most to the agency mission and priorities. According to 

INS officials, INS has not yet made IT investment management an 

institutional priority. Table 7 summarizes INS' stage three maturity. 
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 45 GAO-01-146 INS’ IT Investments

Table 7: Summary of Stage Three Critical Process Ratings

INS' capabilities for each of the stage three critical processes are discussed 

below.

Critical process Rating Key practices

Key practices

executed

Portfolio Selection 

Criteria Definition

Not implemented 6 1

Investment Analysis Not implemented 7 1

Portfolio Development Not implemented 9 4

Portfolio Performance 

Oversight

Not implemented 9 0

Total 31 6
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 46 GAO-01-146 INS’ IT Investments

INS Has Not Created 

Useful Portfolio 

Selection Criteria 

Portfolio selection criteria make up a necessary part of an IT investment 

management process. Developing an enterprisewide investment portfolio 

involves defining appropriate investment cost, benefit, schedule, and risk 

criteria to ensure that the selected investments will best support the 

organization's strategic goals, objectives, and mission. Thus, portfolio 

selection criteria need to reflect the enterprisewide and strategic focus of 

the organization. In addition, the criteria should (1) include cost, benefit, 

schedule, and risk elements, which serve to create a common set of criteria 

that are used to compare projects of different types to one another and 

(2) be clearly communicated to project managers throughout the 

organization so that these managers can take the criteria into account 

when developing proposals. Without portfolio selection criteria, projects 

may be selected on the basis of isolated business needs, the type and 

availability of funds, or the receptivity of management to a specific project 

proposal. 

Thus, according to ITIM, developing portfolio selection criteria requires, 

among other things, that (1) an investment board approve the criteria, 

including cost, benefit, schedule, and risk criteria; (2) the criteria be 

distributed throughout the organization; (3) adequate resources be 

provided for selection criteria definition activities; and (4) a working group 

be responsible for creating and modifying the criteria.

INS developed criteria for selecting new proposals; however, the criteria 

had not been approved by the IRB and did not consistently include cost, 

schedule, benefit, and risk criteria. Furthermore, INS had not distributed 

the criteria throughout INS. For example, none of the IT project and 

program managers that we interviewed were aware of the selection criteria 

that had been developed. In addition, while INS indicated that it has 

adequate resources to develop complete portfolio selection criteria, it has 

not designated a working group to create and modify the criteria. Without 

useful selection criteria, INS is missing a critical means of ensuring that 

selected investments best support the organization's mission and priorities. 

Table 8 summarizes the ratings for each key practice and the specific 

findings supporting the ratings.
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 47 GAO-01-146 INS’ IT Investments

Table 8: Summary of Ratings and Evidence for the Portfolio Selection Criteria Definition Critical Process

Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has written 

policies and procedures for creating 

and modifying IT portfolio selection 

criteria.

Not executed INS does not have written policies and procedures for 

creating and modifying IT portfolio selection criteria.

Prerequisites 1. Adequate resources are provided 

for selection criteria definition 

activities.

Executed According to INS, it has adequate resources and staff for 

selection criteria definition activities. These resources 

include the ESC members.

2. A working group is designated to 

be responsible for creating and 

modifying the IT portfolio selection 

criteria.

Not executed Responsibility for creating and modifying IT portfolio 

selection criteria is not designated. 

Activities 1. The enterprisewide IT investment 

board approves the core IT portfolio 

selection criteria, including cost, 

benefit, schedule, and risk (CBSR) 

criteria, based on the organization's 

mission, goals, strategies, and 

priorities.

Not executed INS has not developed selection criteria that consistently 

included CBSR criteria and that have been applied to all 

IT investments, including ongoing investments. 

2. The IT portfolio selection criteria 

are distributed throughout the 

organization. 

Not executed Selection criteria have not been distributed throughout 

the organization. None of the four project managers who 

we interviewed were aware of criteria used by the IRB to 

select IT investments.

3. The IT portfolio selection criteria 

are reviewed using cumulative 

experience and event-driven data 

and modified, as appropriate.

Not executed INS has reviewed its IT portfolio selection criteria. 

However, the criteria have not been developed for all 

investments (See activity 1).
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 48 GAO-01-146 INS’ IT Investments

INS Does Not Analyze 

Its IT Investments 

Based on Cost, Benefit, 

Schedule, and Risk 

Data When Making 

Investment Decisions 

The purpose of investment analysis is to ensure that all IT investments are 

consistently analyzed and prioritized according to the organization's 

portfolio selection criteria, which should include cost, benefit, schedule, 

and risk criteria. According to ITIM, effective investment analysis includes, 

among other things, that (1) portfolio selection criteria have been 

developed; (2) the IRB ensures that cost, benefit, schedule, and risk data 

are assessed and validated for each investment; (3) the IRB compares each 

investment against the organization's portfolio selection criteria; and 

(4) the IRB creates a ranked list of investments using the portfolio selection 

criteria. 

INS' IRB does not analyze and rank proposed and ongoing investments 

based on their expected cost, benefit, schedule, and risk. As mentioned 

previously, INS has not developed selection criteria that include these 

elements, nor has it ensured that cost, benefit, schedule, and risk data are 

assessed and validated for each IT investment. For example, none of the 

four projects we reviewed provided cost, benefit, schedule, or risk data to 

INS' IRB for consideration during the selection process. Instead, the IRB 

focused on the near-term cost (e.g., annual budget dollars) of each project 

and the perceived importance of the project to INS' mission. In the absence 

of portfolio selection criteria and good investment-related data (i.e., cost, 

benefit, schedule, and risk data), the IRB cannot compare and analyze its 

investments based on their cost, benefit, schedule, and risk expectations 

and create a ranked list of investments that best align with mission 

improvement goals and organizational direction. As a result, INS is missing 

critical information for making sound IT investment decisions. Table 9 

summarizes the ratings for each key practice and the specific findings 

supporting the ratings.
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 49 GAO-01-146 INS’ IT Investments

Table 9: Summary of Ratings and Evidence for the Investment Analysis Critical Process

Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has written 

policies and procedures for 

analyzing IT investments.

Not executed INS does not have written policies and procedures for 

analyzing IT investments. 

Prerequisites 1. Adequate resources are provided 

for investment analysis activities.

Executed According to INS, it has adequate resources for 

investment analysis activities. These resources include 

the ESC members.

2. IT investment portfolio selection 

criteria have been developed. 

Not executed IT investment portfolio selection criteria have not been 

developed for all investments (See the Portfolio Selection 

Criteria Definition critical process).

3. Information from the IT asset 

inventory is used by the IT 

investment board.

Not executed The IRB does not use information from an IT asset 

inventory.

Activities 1. Each IT investment board 

ensures that the CBSR data and 

other required data are validated for 

each investment within its span of 

control.

Not executed The IRB does not ensure that cost, benefit, schedule, or 

risk data are validated. None of the four project 

managers that we interviewed provided this data to the 

IRB.

2. Each IT investment board 

assesses each of its IT investments 

with respect to the IT portfolio 

selection criteria. 

Not executed The IRB does not assess each of its IT investments with 

respect to IT portfolio selection criteria (See prerequisite 

2).

3. Each IT investment board 

prioritizes its full portfolio of IT 

investments using the portfolio 

selection criteria.

Not executed The IRB does not prioritize its full portfolio of IT 

investments using portfolio selection criteria (See 

prerequisite 2).
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 50 GAO-01-146 INS’ IT Investments

INS Does Not 

Comparatively Assess 

All Its IT Projects 

When Making 

Selections for Funding

The purpose of the portfolio development process is to ensure that the IRB 

analyzes and compares all IT investments to select and fund those with 

manageable risks and returns and that best address the strategic business 

direction and priorities of the organization. Once this is accomplished, 

investments can be compared to one another within and across the 

portfolio categories and the best overall portfolio can then be selected for 

funding.

According to ITIM, portfolio development requires, among other things, 

(1) defining common portfolio categories and assigning each investment to 

a portfolio category; (2) ensuring that investments have been analyzed and 

their cost, benefit, schedule, and risk data validated; and (3) examining the 

mix of investments across the portfolio categories in making funding 

decisions.

INS does not assess all its IT projects in making selections for funding. 

While INS has defined common portfolio categories, it is not using them to 

manage its investments. INS has created eight portfolio categories and 

assigned all of its investments to one of the portfolios. However, the IRB 

has not analyzed these investments, including both proposed and ongoing 

projects, based on validated cost, benefit, schedule, and risk data. Without 

these meaningful data, the IRB cannot compare its investments across 

portfolio categories. As a result, the IRB cannot make trade-offs between 

investment alternatives, determine which projects contribute most to 

agency performance, or eliminate redundant systems. Table 10 summarizes 

the ratings for each key practice and the specific findings supporting the 

ratings.
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 51 GAO-01-146 INS’ IT Investments

Table 10: Summary of Ratings and Evidence for the Portfolio Development Critical Process

Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has written 

policies and procedures for 

establishing and maintaining the 

portfolio development process.

Not executed INS does not have written policies or procedures for 

establishing and maintaining the portfolio development 

process.

Prerequisites 1. Adequate resources are provided 

for executing the portfolio 

development process.

Executed According to INS, it has adequate resources available to 

execute the portfolio development process. These 

resources include the ESC members.

2. Board members exhibit core 

competencies in portfolio 

development.

Not executed The IRB/ESC members do not collectively analyze all IT 

investments using portfolio selection criteria and thus 

have not exhibited core competencies in portfolio 

development.

3. Individual IT investments have 

been analyzed and their CBSR data 

have been validated.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

4. The organization has defined its 

common portfolio categories. 

Executed INS has defined the following eight portfolio categories. 

They are Enforcement, Inspections, Examinations, 

Corporate, Management, Infrastructure, Biometrics, and 

IRM Operations. 

Activities 1. Each IT investment board assigns 

investment proposals to a portfolio 

category.

Executed Each IT project is assigned to a portfolio category.

2. Each IT investment board 

examines the mix of proposals and 

investments across the common 

portfolio categories and makes 

selections for funding.

Not executed INS does not examine the mix of investments across all 

portfolio categories.

3. Each IT investment board 

approves or modifies the annual 

CBSR expectations for each of its 

selected IT investments.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

4. A repository of portfolio 

development information is 

established, updated, and 

maintained.

Executed A repository of portfolio information has been created 

and is being maintained.
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 52 GAO-01-146 INS’ IT Investments

INS Does Not Oversee 

IT Investments' Cost, 

Benefit, Schedule, and 

Risk Performance 

The purpose of the portfolio performance oversight critical process is to 

ensure that each IT investment achieves its cost, benefit, schedule, and risk 

expectations. This critical process builds upon the IT Project Oversight 

critical process by adding the elements of benefit measurement and risk 

management to an organization's investment control capacity. Executivelevel oversight of project-level risk and benefit management activities 

provides the organization with increased assurance that each investment 

will achieve the desired cost, benefit, schedule, and risk results.

According to ITIM, effective portfolio performance oversight requires, 

among other things, that the IRB (1) have access to up-to-date cost, benefit, 

schedule, and risk data; (2) monitor the performance of each investment in 

its portfolio by comparing actual project-level cost, benefit, schedule, and 

risk data to the predefined expectations for the project; and (3) correct 

poorly performing projects.

INS does not monitor its investments' performance to ensure that they are 

meeting cost, benefit, schedule, and risk performance expectations. As 

mentioned previously, up-to-date cost, benefit, schedule, and risk data are 

not available. Without these data, the IRB is unable to monitor the 

performance of its investments to ensure that they are achieving their cost, 

benefit, schedule, and risk expectations and to act when performance 

problems arise. Table 11 summarizes the ratings for each key practice and 

the specific findings supporting the ratings.
Chapter 3

INS Is Not Managing Its IT Investments as a 

Complete Portfolio

Page 53 GAO-01-146 INS’ IT Investments

Table 11: Summary of Ratings for the Portfolio Performance Oversight Critical Process

Key practice Rating Summary of evidence

Organizational 

commitment

1. The organization has written 

policies and procedures for 

monitoring and controlling portfolio 

performance.

Not executed INS has no policies and procedures for monitoring and 

controlling portfolio performance.

Prerequisites 1. Adequate resources are provided 

for monitoring and controlling the 

portfolio's performance.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

2. Annual CBSR expectations are 

agreed upon for each IT investment.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

3. The IT investment board has 

access to up-to-date actual and 

expected CBSR data in the 

repository.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

Activities 1. Each IT investment board monitors 

the performance of each investment 

in its portfolio by comparing actual 

CBSR data to expectations.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

2. Using established criteria, the IT 

investment board identifies its 

investments that have not met 

predetermined CBSR performance 

expectations. 

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

3. The IT investment board and the 

project manager determine the root 

cause of the poor performance.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

4. The IT investment board and the 

project manager develop an action 

plan designed to remedy the 

identified cause(s) of poor 

performance.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”

5. Corrective actions are initiated and 

outcomes are tracked.

Not executed INS indicated in its self-assessment that this key practice 

was “not executed.”
Page 54 GAO-01-146 INS’ IT Investment

Chapter 4

The Department of Justice Is Not Guiding and 

Overseeing INS' Investment Management 

Approach Chapte4r

The Clinger-Cohen Act of 1996 imposed rigor and structure on how 

agencies approach the selection and management of IT projects. 1

 Among 

other things, it requires the head of each agency to implement a process for 

maximizing the value of the agency's IT investments and assess and 

manage the risks of its IT investments. It also requires that the agency CIO 

work with the agency head in implementing this process. As such, Justice is 

responsible for ensuring that its bureaus and components, including INS, 

implement an effective IT investment management process. 

Justice has not provided INS, or any other Justice component, sufficient 

direction, guidance, and oversight of IT investment management activities. 

While Justice issued guidance in January 2000 describing its high-level 

investment management process, the guidance does not address the need 

or requirement for Justice's components to implement an IT investment 

management process. Specifically, this guidance does not instruct the 

components to establish IT investment management processes nor does it 

establish expectations for doing so. According to Justice officials, Justice 

had not established these processes because of other competing 

department priorities, even though the department and its components 

spent about $3 billion on IT in fiscal years 1999 and 2000.

During the course of our work, Justice began drafting IT investment 

management policy and guidance documents in collaboration with an 

intercomponent working group. The draft policy directs Justice 

components to establish and use an IT investment management process 

and directs the Justice CIO to monitor the components' investment 

management processes through periodic briefings. A supplemental 

guidance document provides procedures for developing an investment 

management process. Justice officials stated that they plan to issue the 

final policy by the end of December 2000 and the guidance by March 2001. 

Until Justice issues its policy and guidance and begins monitoring its 

components' progress, it has no assurance that it has the necessary 

investment management processes in place to maximize the value of its IT 

investments and manage the risks associated with them. 

1

The fiscal year 1997 Omnibus Consolidated Appropriations Act, P. L. 104-208, renamed both 

Division D (the Federal Acquisition Reform Act) and E (the Information Technology 

Management Reform Act) of the 1996 DOD Authorization Act, P. L. 104-106, as the ClingerCohen Act of 1996.
Page 55 GAO-01-146 INS’ IT Investment

Chapter 5

Conclusions, Recommendations, and Agency 

Comments Chapte5r

IT is critical to INS' ability to provide vital services, such as granting 

naturalization benefits and detecting and preventing the illegal entry of 

aliens into the United States. Effectively and efficiently managing IT 

requires, among other things, a structured approach for minimizing the risk 

and maximizing the return on IT investments. However, INS executives are 

making investment decisions involving hundreds of millions of dollars 

without vital data about these investments' relative costs, benefits, and 

risks. As a result, INS cannot adequately know whether it is making the 

right investment decisions, whether it has selected the mix of investments 

that best meets its overall mission and business priorities, or whether these 

investments are living up to expectations.

INS has initiated efforts to establish an IT investment management 

foundation. However, it is lacking many important foundational investment 

management capabilities, particularly those relating to controlling projects 

against predetermined expectations and addressing variances. As a result, 

it runs the serious risk that its IT projects will be late, cost more than 

expected, and not perform as intended.

INS' use of portfolio categories and portfolio managers provides some 

structure to its portfolio development process and provides each business 

area the opportunity to identify the projects that it determines to be the 

most important to its performance. However, INS' lack of performance data 

from ongoing projects handicaps the IRB's ability to perform its portfolio 

oversight function. In addition, the absence of any project-to-project 

comparison limits the IRB's ability to judge whether its mix of investments 

best meets its mission needs and priorities. As a result, INS can have little 

confidence that its chosen mix of IT investments best meets mission goals 

and priorities and that these investments will be developed within an 

acceptable level of risk, on time, and within budget.

Further, Justice has a statutory role under the Clinger-Cohen Act to ensure 

that its component agencies, including INS, have effective investment 

management processes. Until Justice fulfills this role, it has little assurance 

that INS, or its other components, are investing the department's limited IT 

resources to maximize return on investment, minimize risk, and best 

support mission needs. 

Recommendations for 

Executive Action

To strengthen INS' investment management capability and address the 

weaknesses discussed in this report, we recommend that you direct the 

Commissioner of the Immigration and Naturalization Service to designate 
Chapter 5

Conclusions, Recommendations, and Agency 

Comments

Page 56 GAO-01-146 INS’ IT Investment

development and implementation of effective IT investment management 

processes as an agencywide priority and manage it as such. Specifically, 

you should direct the Commissioner to do the following:

• Develop a plan, within 9 months, for implementing IT investment 

management process improvements that is based on stages two and 

three critical processes and specifies measurable goals and time frames, 

ranks initiatives, defines a management structure for directing and 

controlling the improvements, establishes review milestones, and 

recognizes any direction and guidance that Justice issues. This plan 

should first focus on those critical processes in stage two of ITIM 

because, collectively, they provide the foundation for building a mature 

IT investment management process.

• Submit the plan to the Justice CIO for review and approval.

• Implement the approved plan and report to the Justice CIO, according to 

established review milestones, on progress made against the plan's 

goals and time frames.

Further, because the absence of effective investment management 

processes and an enterprise architecture1

 severely limits INS' ability to 

effectively manage its IT investments, we recommend that until INS 

develops a complete enterprise architecture and implements the key 

practices associated with stages two and three critical processes, as 

described in this report, you direct the Commissioner to limit requests for 

future appropriations for IT only to efforts that

• support ongoing operations and maintenance, but not major 

enhancements, of existing systems;

• support INS efforts to develop and implement IT investment 

management processes and an enterprise architecture;

• are small, represent low technical risk, and can be delivered in a 

relatively short period of time; or 

• are congressionally mandated.

Further, to improve Justice's guidance and oversight of components' IT 

investment management process activities, we also recommend that you 

direct the Justice CIO to follow through on the department's plans to issue 

1 Information Technology: INS Needs to Better Manage the Development of Its Enterprise 

Architecture (GAO/AIMD-00-212, August 1, 2000).
Chapter 5

Conclusions, Recommendations, and Agency 

Comments

Page 57 GAO-01-146 INS’ IT Investment

an IT investment management policy and guidance to the components and 

to ensure that the policy and guidance:

• Directs Justice components and bureaus, including INS, to develop and 

implement IT investment management processes.

• Instructs Justice components and bureaus on how to develop an 

investment management process. This guidance should be based on the 

investment management guidance contained in this report and, at a 

minimum, should include component roles, responsibilities, authorities, 

and policies and procedures for developing an IT investment 

management process.

• Directs the Justice CIO to monitor the components' progress in 

developing and establishing an IT investment management process and 

take appropriate action if they are not progressing sufficiently.

Agency Comments and 

Our Evaluation

In written comments on a draft of this report, Justice's Assistant Attorney 

General for Administration generally agreed with our recommendations, 

although he offered minor wording modifications on two recommendations 

that he said would increase Justice's ability to fully implement them. The 

Assistant Attorney General for Administration also disagreed with our 

finding that Justice is not guiding and directing INS' investment 

management approach.

Justice generally agreed with our recommendation that INS develop and 

submit to Justice a plan for implementing investment management process 

improvements. However, Justice suggested that the time frame for 

developing the plan be clarified such that INS has 6 months to develop and 

submit its plan to Justice once Justice issues its new IT investment 

management guidance. Because our recommendation directed INS to 

consider any Justice guidance and direction in developing its investment 

management process improvement plan, we modified the recommendation 

to include an additional 3 months to allow time for Justice to issue its 

guidance, which it plans to do in March 2001.

Justice also concurred with our recommendation that INS limit future 

appropriation requests for IT to certain investment categories because it 

lacks an enterprise architecture and effective investment management 

processes, but suggested that we specify that this recommendation is in 

effect until INS completes its architecture and implements investment 

management processes. Because this is the intent of our recommendation, 

we clarified the recommendation to make this explicit.
Chapter 5

Conclusions, Recommendations, and Agency 

Comments

Page 58 GAO-01-146 INS’ IT Investment

Also in its comments, Justice agreed that, while INS has some important 

investment management capabilities, INS still needs to develop effective 

investment management processes. Further, Justice agreed with our 

recommendation for Justice to issue an investment management policy and 

guidance to its components, including INS, that (1) directs components to 

develop and implement IT investment management processes, (2) instructs 

components on how to develop and implement these processes based on 

the investment management framework in our report, and (3) ensures that 

components' progress in doing so is monitored. Moreover, Justice stated, 

which we note in our report, that it is now working with its components to 

develop an IT investment management policy and process, and it has made 

this a department priority for this year.

However, Justice stated that our draft report fails to recognize the extent of 

Justice's oversight of INS' IT investment management process. Further, it 

disagreed with our finding that Justice is not guiding and directing INS' 

investment management approach. Justice stated that it has established 

guidance for all aspects of IT management that its components are 

expected to follow and has a process for overseeing components' 

management of their investments. Justice cited six examples to illustrate 

its point, such as Justice approval authority of all component IT 

investments with life-cycle cost over $1 million, Justice establishment of an 

IT investment board, Justice meetings with components, including 

Attorney General meetings with the INS Commissioner, and Justice 

forwarding of OMB budget requirements to components.

We do not agree with Justice's position. While we concur that the examples 

cited by Justice represent important IT management functions to be 

performed in providing management oversight of individual IT 

investments, such management oversight is not the focus of our findings, 

conclusions, and recommendations. Rather, our report addresses Justice's 

efforts to ensure that its components, including INS, have each defined and 

implemented effective IT investment management processes. As such, we 

sought evidence from Justice demonstrating that it has directed its 

components to establish such processes, provided guidance to its 

components on how to develop and implement these processes, and 

monitored its components' progress to determine whether they are 

implementing such processes. However, besides the steps that Justice 

initiated during the course of our inquiries and plans to take, which we 

have described in this report, we found no such evidence. Moreover, 

Justice stated in its written comments that it agreed with our 
Chapter 5

Conclusions, Recommendations, and Agency 

Comments

Page 59 GAO-01-146 INS’ IT Investment

recommendation for it to provide investment management process 

direction, guidance, and oversight to its components.

Justice's written comments and our evaluation of them are presented in 

appendix I.
Page 60 GAO-01-146 INS’ IT Investments

Appendix I

Comments From the Immigration and Apendxies

Naturalization Service ApendxiI

Note: GAO comments 

supplementing those in 

the report text appear at 

the end of this appendix.

See comment 1.
Appendix I

Comments From the Immigration and 

Naturalization Service

Page 61 GAO-01-146 INS’ IT Investments

See comment 2.
Appendix I

Comments From the Immigration and 

Naturalization Service

Page 62 GAO-01-146 INS’ IT Investments

See comment 3.
Appendix I

Comments From the Immigration and 

Naturalization Service

Page 63 GAO-01-146 INS’ IT Investments

The following are GAO's comments on the Department of Justice's letter 

dated November 16, 2000.

GAO Comments 1. We do not agree with Justice's statement that it has established 

guidance for all aspects of IT management that its components are 

expected to follow and has a process for overseeing components' 

management of their investments. While we concur that the examples 

cited by Justice represent important IT management functions to be 

performed in providing management oversight of individual IT 

investments, such management oversight is not the focus of our 

findings, conclusions, and recommendations. Rather, our report 

addresses Justice's efforts to ensure that its components, including 

INS, have each defined and implemented effective IT investment 

management processes. To this end, we sought evidence from Justice 

demonstrating that it has directed its components to establish such 

processes, provided guidance to its components on how to develop and 

implement these processes, and monitored its components' progress to 

determine whether they are implementing such processes. Besides the 

steps that Justice initiated during the course of our inquiries and plans 

to take, which we have described in our report, we found no such 

evidence. Moreover, Justice stated in its written comments that it 

agrees with our recommendation for it to provide investment 

management process direction, guidance, and oversight to its 

components. 

2. Because our recommendation directed INS to consider any Justice 

guidance and direction in developing its investment management 

process improvement plan, we have modified our recommendation to 

incorporate Justice's suggestion that INS have 6 months to develop and 

submit its plan to Justice after Justice issues its new IT investment 

management guidance. 

3. It was our intent that INS limit its future appropriation requests for IT 

to certain investment categories only until it completes its architecture 

and implements investment management processes. As a result, we 

have clarified the recommendation to make this explicit.
Page 64 GAO-01-146 INS’ IT Investment

Appendix II

GAO Contacts and Staff Acknowledgments ApendxiI

GAO Contacts Randolph C. Hite, (202) 512-3870

David L. McClure, (202) 512-6408

Acknowledgments Deborah Davis, Lester Diamond, Tamra Goldstein, Kelly Hlavka, Sabine 

Paul, and John Rehberger made key contributions to this report.

(511705) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of 

reports are $2 each. A check or money order should be made out to 

the Superintendent of Documents. VISA and MasterCard credit 

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are 

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and 

testimony. To receive facsimile copies of the daily list or any list 

from the past 30 days, please call (202) 512-6000 using a touchtone 

phone. A recorded menu will provide information on how to obtain 

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet, 

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at: 

http://www.gao.gov

To Report Fraud, 

Waste, or Abuse in 

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00