Information Technology: INS Needs to Strengthen Its Investment Management Capability
Published: 2000-12-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Immigration and Naturalization Service (INS) invests hundreds of millions of dollars each year in information technology (IT) to help (1) prevent aliens from entering the United States illegally and remove aliens who succeed in doing so and (2) provide services or benefits to facilitate entry, residence, employment, and naturalization to legal immigrants. The Clinger-Cohen Act requires agency heads to implement a process for maximizing the value and assessing and managing the risks of its IT investments. GAO examined leading private and public sector IT management practices to determine whether INS is effectively managing its IT investments and whether the Department of Justice (DOJ) is effectively promoting, guiding, and overseeing INS' investment management activities.
section_title: What GAO Found
paragraphs: GAO found that INS lacks the basic capabilities upon which to build IT investment management maturity. Furthermore, INS is not managing IT investments as a complete portfolio. By managing its IT investments as individual projects, INS will not be able to determine which investments contribute most to the agency mission. GAO also found that DOJ is not guiding and overseeing INS' investment management approach.

section_title: Letter
paragraphs: This report addresses the Immigration and Naturalization Service's (INS)  management of information technology (IT) investments. Each year INS  invests hundreds of millions of dollars on IT systems and activities. We  found that INS has established some important capabilities for managing  these investments, but it has considerable work ahead to fully implement  mature and effective processes. We are making recommendations to  strengthen INS' investment management capabilities.
We are sending copies of this report to Senator Judd Gregg, Chairman, and  Senator Ernest F. Hollings, Ranking Minority Member, Senate  Appropriations Subcommittee on Commerce, Justice, State, and Judiciary;  Senator Spencer Abraham, Chairman, and Senator Edward M. Kennedy,  Ranking Minority Member, Senate Judiciary Subcommittee on Immigration;  Representative Harold Rogers, Chairman, and Representative Jose E.  Serrano, Ranking Minority Member, House Appropriations Subcommittee  on Commerce, Justice, State, and the Judiciary; Representative Lamar  Smith, Chairman, and Representative Sheila Jackson Lee, Ranking Minority  Member, House Judiciary Subcommittee on Immigration and Claims; the  Honorable Jacob J. Lew, Director, Office of Management and Budget; and  Mary Ann Wyrsch, Acting Commissioner of the Immigration and Naturalization Service. Copies will also be made available to others upon  request.
subsections: section_title: Introduction
paragraphs: The mission of INS, an agency of the Department of Justice, is to  administer and enforce the immigration laws of the United States. To  accomplish this, INS is organized into three core business areas— enforcement, immigration services, and corporate services. Enforcement  includes, among other things, conducting inspections of travelers entering  the United States as they arrive at more than 300 land, sea, and air ports of  entry; detecting and preventing the smuggling and illegal entry of aliens;  and identifying and removing persons who have no lawful immigration  status in the United States. Immigration services, which involve regulating  permanent and temporary immigration to the United States, include  granting legal permanent residence status, nonimmigrant status (e.g.,  tourists and students), and naturalization. Corporate services include  records management, financial management, personnel management, and  inventory management support for INS activities.
INS' IT assets play a significant role in (1) receiving and processing  naturalization and other benefit applications, (2) processing immigrants  and nonimmigrants entering and leaving the United States, and  (3) identifying and removing people who have no lawful immigration status  in the United States. For example, the Computer-Linked Application  Information Management System (CLAIMS 4) is a centralized case  management tracking system, that offers support for a variety of tasks  associated with processing and adjudicating naturalization benefits. In  addition, the Deportable Alien Control System (DACS) automates many of  the functions associated with tracking the location and status of illegal  aliens in removal proceedings, including detention status.
subsections: section_title: INS' Current IT Investment Efforts
paragraphs: INS has multiple efforts underway to develop and acquire new information  systems and to maintain existing ones. According to INS, in fiscal year  2000, it obligated about $327 million on IT activities, including about  $94 million for new development and the remaining amount, which  includes enhancing existing systems, for operations and maintenance. For  example, INS obligated $14.5 million in fiscal year 2000 to continue  development of CLAIMS 4, which supports the processing of applications  and petitions for immigrant benefits and is intended to fully replace  CLAIMS 3. In addition, INS obligated about $18 million in fiscal year 2000 to  further deploy its Integrated Surveillance Intelligence System (ISIS), which  includes the deployment of intelligent computer aided detection systems,  unattended ground sensors, and fixed cameras along the northern and  southern borders to provide around-the-clock visual coverage of the  border. For fiscal year 2001, INS plans to spend about $226 million on IT for  operations and maintenance activities. INS funds most of its IT efforts  with operation and maintenance funds and currently is developing or  maintaining 74 information systems.
section_title: Recent Reviews Have Identified IT Project Management Weaknesses
paragraphs: Recent reviews have identified several weaknesses in INS' management of  its IT projects. For example, in August 1998, the Logistics Management  Institute (LMI) reported that INS' Office of Information Resources  Management (OIRM) (1) did not maintain accurate cost estimates for the  complete life cycle of projects and (2) did not track and manage projects to  a set of cost, schedule, technical, and benefit baselines. Further, LMI noted  that while INS' System Development Life Cycle (SDLC) manual provides a  good model for systems development projects, OIRM did not consistently  follow it, often bypassing key SDLC phases.
Similarly, in July 1999, the Justice Inspector General (IG) reported that  (1) estimated completion dates for some INS IT projects had been delayed  without explanation for the delays, (2) project costs continued to spiral  upward with no justification for how funds are spent, and (3) projects were  nearing completion with no assurance that they would meet performance  and functional requirements.
Recognizing the need to address these weaknesses, INS established an  Operational Assessment Team to analyze reported weaknesses and  recommend specific actions to address them. The Operational Assessment  Team validated the deficiencies identified in the LMI and Justice IG reports  and identified additional ones. For example, the team found that system  requirements were not consistently collected, recorded, documented,  tracked, and controlled. To illustrate, of 105 projects reviewed by the team,  fewer than 50 percent had documented requirements and most of the  requirements that had been documented were not current.
Further, in August 2000, we reported that INS did not have an enterprise  architecture to guide the development and evolution of its information  systems. An enterprise architecture is an institutional systems blueprint  that defines in both business and technological terms the organization's  current and target operating environments and provides a road map for  moving from one to the other. It is required by the Clinger-Cohen Act and is  a recognized practice of successful public and private sector organizations.  INS had initiated some limited efforts to document its current architecture,  but it had not yet begun developing a target architecture or a plan to move  from the current to the target environment. Moreover, INS had not yet  established the management structure and controls to develop the  architecture. The absence of such an enterprise architecture increases the  risk that the hundreds of millions of dollars INS spends each year on  information systems will not be well integrated or compatible and will not  effectively support mission needs and priorities.
section_title: Overview of INS' Current Approach to IT Investment Management
paragraphs: In 1997, INS established an investment review board (IRB). The IRB  consists of four voting members—the Deputy Commissioner (Chair) and  INS' three Executive Associate Commissioners—and advisory or  supporting members, including the Director of the Budget Office and the  Acting Associate Commissioner of the Office of Information Resources  Management. In November 1998, INS also established the Executive  Steering Committee (ESC) to support the IRB. The ESC comprises  portfolio managers and advisory members, which analyze investment  proposals and make recommendations on these proposals to the IRB.
The IRB has established a process for selecting new IT proposals.  According to INS officials, new proposals are developed throughout the  year as business needs are identified and are forwarded to the appropriate  portfolio manager for review. After reviewing the proposal, the portfolio  manager forwards it to the ESC for consideration for funding. The ESC  examines the proposals submitted and determines the appropriate funding  for each project. Once funding is determined, the ESC forwards the  proposed funding levels to the IRB, which makes the final investment  selections and budget formulation decisions. See figure 2 for INS' new  proposal selection process.
As part of INS' annual budget execution process, the IRB considers the  funding requests of ongoing and new projects. Project managers define  requirements for their ongoing projects, which they submit to the  responsible portfolio managers for review. After reviewing the  requirements and funding requests, each portfolio manager submits them  to the ESC for review and to the IRB for approval. The approved funding is  submitted to the Budget Office for inclusion into its budget execution  process. According to INS officials, new proposals are considered for  funding only after ongoing projects have been funded.
section_title: Framework for Assessing Agencies' IT Investment Management
paragraphs: Several recent management reforms—including the revision to the  Paperwork Reduction Act and the passage of the Clinger-Cohen Act of  1996, the Government Performance and Results Act of 1993, and the Chief  Financial Officers Act of 1990—have introduced requirements emphasizing  the need for federal agencies to improve their management processes for  selecting and managing IT resources. In particular, the Clinger-Cohen Act  requires that the head of each agency implement a process for maximizing  the value of the agency's IT investments and for assessing and managing  the risks of its acquisitions. A key goal of the Clinger-Cohen Act is that  agencies have processes and information in place to help ensure that  projects are being implemented at acceptable costs within reasonable and  expected time frames and that they are contributing to tangible, observable  improvements in mission performance.
We and the Office of Management and Budget (OMB) have developed  guidance to assist federal agencies in managing IT investments. One such  guide, Assessing Risks and Returns: A Guide for Evaluating Federal  Agencies' IT Investment Decision-making, incorporates our analysis of  the management practices of leading private and public sector  organizations as well as the provisions of major federal legislation (e.g.,  Clinger-Cohen Act) and executive branch guidance that address investment  decision-making. The guide provides a method for determining how well a  federal agency is selecting and managing its IT resources and identifies  specific areas where improvements can be made.
To enhance this guidance, we issued an Information Technology  Investment Management (ITIM) maturity framework in May 2000. ITIM  provides a common framework for assessing IT capital planning and  investment management practices by describing the organizational  processes, and their interrelationships that are the tenets of good  investment management. ITIM is based on the best-practices work done as  part of our ongoing research into the IT management practices of leading  organizations.
ITIM is a hierarchical model comprising five maturity stages. These  maturity stages represent steps toward achieving stable and mature  investment management processes. As agencies advance through the  model's stages, their capability to manage IT increases. Each stage builds  upon the lower stages and enhances the organization's ability to manage its  investments. With the exception of the first stage, each maturity stage is  composed of critical processes that must be implemented and  institutionalized for the organization to satisfy the requirements of that  stage. These critical processes are further broken down into key practices  that describe the types of activities that an agency should be engaged in to  successfully implement each critical process. An organization that has  these critical processes in place is in a better position to successfully invest  in IT. (See figure 3 for the five stages and associated critical processes).
As established by the model, each critical process contains five core  elements that indicate whether the implementation and institutionalization  of a process can be effective and repeated. The five core elements are:
Purpose: This is the primary reason for engaging in the critical process  and states the desired outcome for the critical process.
Organizational commitment: This comprises management actions that  ensure that the critical process is established and will endure. Key  practices typically involve establishing organizational policies and  engaging senior management sponsorship.
Prerequisites: These are the conditions that must exist within an  organization to successfully implement a critical process. This typically  involves allocating resources, establishing organizational structures,  and providing training.
Activities: These are the key practices necessary to implement a critical  process. An activity occurs over time and has recognizable results. Key  practices typically involve establishing procedures, performing and  tracking the work, and taking corrective actions as necessary.
Evidence of performance: This comprises artifacts, documents, or other  evidence that supports a contention that the key practices within a  critical process have or are being implemented. This core element  typically consists of the collection and verification of physical,  documentary, or testimonial evidence and typically involves reviews by  objective parties.
With the exception of the purpose core element, each of the other core  elements contains key practices. The key practices are the attributes and  activities that contribute most to the effective implementation and  institutionalization of a critical process. (Figure 4 shows the relationship  between the various ITIM components.)
section_title: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine whether (1) INS is effectively managing  its IT investments and (2) the Department of Justice is effectively  promoting, guiding, and overseeing INS' investment management activities.
To determine whether INS is effectively managing its investments, we  applied our ITIM framework and the associated assessment method. As  part of the ITIM assessment method, INS conducted a self-assessment of its  IT investment management activities using the ITIM framework. In its self- assessment, INS indicated whether it executed each of the key practices in  stages two through five. INS asserted that it executed many of the key  practices within stages two and three but only four key practices in all of  stages four and five. Accordingly, we did not include ITIM stages four and  five in the scope of our review. Also, we did not evaluate the key practices  within stages two and three that INS stated it had not executed.
We evaluated INS against 9 of the 10 critical processes in stages two and  three. We did not evaluate INS against the stage three critical process  Authority Alignment of IT Investment Boards. This critical process is only  relevant if an organization has more than one IT investment board and INS  has only one. The nine critical processes we examined focus primarily on  INS' ability to effectively select and control its IT investments.
To determine whether INS had implemented these nine critical processes,  we evaluated policies, procedures, and guidance related to INS' IT  investment management activities. In particular, we analyzed the following:  organizational charters, INS' System Development Life Cycle manual,  requirements management process guide, and administrative manuals (e.g.,  Personal Property Handbook). We also reviewed documentation  associated with specific investment management activities, such as IRB  and ESC meeting minutes, project management plans, system deployment  plans, budget formulation and execution plans, quarterly reports to Justice,  and contractor statements of work.
In addition, we reviewed four IT projects to verify the execution of INS- defined processes, procedures, and practices. The four projects were  selected based on the following criteria: (1) the projects should represent  different life cycle phases (e.g., requirements definition, design, operations  and maintenance), (2) the projects should support different INS business  areas (e.g., Examinations, Enforcement), (3) at least one project should be  considered high risk, and (4) at least one project should have been  reviewed by Justice's Information Technology Investment Board (ITIB).  The projects we evaluated are:
Coordinated Interagency Partnership Regulating International  Students (CIPRIS): CIPRIS is an Internet-based system that is intended  to modernize and streamline the current process for collecting  information relating to nonimmigrant foreign students and other  exchange program participants. It is intended to enable U. S.  universities, schools, and cultural exchange programs to report and  share information electronically with INS and other government  regulatory agencies. INS has implemented an operational prototype of  CIPRIS at 21 educational institutions. CIPRIS is a concept exploration  project that supports the Examinations business area within INS. INS  has designated CIPRIS as a high-risk project and it has been reviewed by  Justice's ITIB. According to INS, it obligated about $3.1 million for  CIPRIS in fiscal year 2000.
Computer-Linked Application Information Management System  (CLAIMS) 4.0: According to INS, CLAIMS 4 is intended to improve  delivery of naturalization services by fully automating INS' case  management system. According to INS, CLAIMS 4 supports the  Immigration Services Program within INS and is currently operational at  59 sites. According to INS, it obligated $14.5 million for CLAIMS 4 in  fiscal year 2000.
Integrated Surveillance Intelligence System (ISIS): ISIS was  established to detect and deter illegal intruders and to safely apprehend  illegal aliens on the U.S.-Mexico and U.S.-Canada borders. ISIS is  designed to provide all-weather sensor and video surveillance of the  U.S. borders 24 hours a day, 7 days a week. The major components of  ISIS are the Intelligent Computer-Assisted Detection system, ground  sensors, and the Remote Video Surveillance system. ISIS supports the  Enforcement program area within INS and has been reviewed by  Justice's ITIB. According to INS, it obligated about $18 million for ISIS  in fiscal year 2000 to further deploy the system.
Central Index System (CIS): CIS provides INS with information about  persons of interest to the INS. According to INS, CIS also interacts with  various INS databases to provide the data necessary for INS operations.  CIS currently maintains approximately 45 million detailed records on  individuals of interest to INS. CIS supports the INS' Corporate business  area and is in the operations and maintenance phase of its life cycle.  According to INS, it obligated about $2.6 million for CIS in fiscal year  2000.
We did not validate INS' IT spending obligations for fiscal year 2000 and IT  spending estimates for fiscal year 2001.
To supplement our document reviews, we interviewed senior INS officials,  including the Deputy Commissioner, who chairs the IRB, and the Executive  Associate Commissioner for Management, who is the Chief Information  Officer (CIO) and an IRB member. We also interviewed the Acting  Associate Commissioner for Information Resources Management, who  chairs the ESC; the Director of INS' Investment Management Team;  portfolio managers; the Director of the Office of Strategic Information and  Technology Development; IT project managers; program managers; Office  of Budget representatives; and officials involved with the development and  maintenance of INS' asset tracking systems.
We compared the evidence collected from our document review and  interviews to the key practices and critical processes in ITIM. Because  ITIM is a hierarchical framework, the rating of each critical process is  dependent on the key practices below it. Therefore, we first rated the key  practices. In accordance with the ITIM assessment method, we rated a key  practice as “executed” when we determined, by consensus, that INS was  executing the key aspects of the practice. A key practice was rated as “not  executed” when we determined that there were significant weaknesses in  INS' execution of the key practice and INS offered no adequate alternative,  or when the team found no evidence of a practice during the review.
Once the key practices were rated, we rated each of the nine critical  processes we reviewed. A critical process was rated as “implemented” if all  of the underlying key practices were rated as being executed. A critical  process was rated as “not implemented, but improvements underway” if  over half, but not all, of its underlying key practices were rated as being  executed. A critical process was rated as “not implemented” when there  were significant weaknesses (i.e., fewer than 50 percent of the key  practices had been implemented) in INS' implementation of the underlying  key practices and no adequate alternative was in place.
To determine whether the Department of Justice is effectively promoting,  guiding, and overseeing INS' investment management activities, we  interviewed officials within the Office of Information Management and  Security Staff, the organization that plays a leading role in Justice's  investment management activities. We also reviewed Justice's January 2000  investment management guidance, draft policy and guidance documents,  INS project proposals, ITIB review and decision documentation, and  quarterly briefing documents. We also discussed Justice's oversight  activities with various officials within INS.
We conducted our work at INS and Justice headquarters in Washington,  D.C., from May 2000 through October 2000 in accordance with generally  accepted government auditing standards. Justice's Assistant Attorney  General for Administration provided written comments of a draft of this  report. These comments are presented in chapter 5 and are reprinted in  appendix I.
section_title: INS Lacks Foundation Capabilities Upon Which to Build IT Investment Management Maturity
paragraphs: The primary purpose of ITIM stage two maturity is to attain repeatable,  successful IT project-level investment control processes and basic  selection processes. For an organization to develop an overall sound IT  investment management process, it must first be able to control its  investments so that it can identify expectation gaps early and correct them.  According to ITIM, stage two maturity includes (1) defining IRB operations,  (2) developing a basic process for selecting new IT proposals,  (3) developing project-level investment control processes, (4) creating an  IT asset inventory, and (5) identifying the business needs for each IT  project.
INS has not fully implemented any of the critical processes associated with  stage two; however, it has improvements underway and is close to fully  implementing two of these processes. INS has (1) established an IRB,  which comprises both IT and business senior executives and functions as  INS' central decision-making body for IT projects, and (2) the IRB has  followed a structured process for developing and selecting new IT  proposals and making initial funding decisions for these proposals.
However, INS has not yet developed some of the capabilities necessary to  build a sound IT investment management process. For example, INS has  not (1) established basic project-level control processes to ensure that its  IT projects are performing as expected, (2) created an IT asset inventory  for investment management, and (3) defined business needs for all of its IT  projects. According to INS, it lacks these critical investment capabilities  because it has not yet made IT investment management an institutional  priority. Table 1 summarizes INS' stage two maturity.
INS' capabilities for each of the stage two critical processes are discussed  below.
subsections: section_title: INS Has Established an IRB, But Has Not Developed Policies and Procedures to Govern IRB Operations
paragraphs: The purpose of this critical process is to define and establish the governing  board or boards responsible for selecting, controlling, and evaluating IT  investments. This includes defining the membership, guiding policies,  operations, roles and responsibilities, and authorities for the investment  board and, if appropriate, each board's support staff. These policies, roles  and responsibilities, and authorities also provide the basis for the board's  investment selection, control, and evaluation activities.
According to ITIM, effective IT investment board operations require,  among other things, that (1) the board membership include both IT and  business knowledge, (2) the organization's executives and line managers  support and carry out board decisions, (3) the organization create an  organization-specific process guide that includes policies and procedures  to direct the board's operations, and (4) the IRB operate according to these  written policies and procedures.
INS is executing many of the practices in this critical process. For example,  INS has an IRB that functions as a central decision-making body for IT  investments and is composed of senior executives from both INS' IT and  business areas. During our discussions with agency officials, we found  broad support within the organization for the IRB's decisions. For example,  three of the four program/project managers we interviewed acknowledged  the IRB's role in investment decision-making. The IRB is chaired by the  Deputy Commissioner and includes INS' three Executive Associate  Commissioners. The IRB is supported by an ESC, which is comprised of  senior representatives who manage INS' eight IT portfolios. The ESC  reviews and analyzes IT investments and makes recommendations to the  IRB for final approval. This senior level involvement and the breadth of  representation help to demonstrate executive sponsorship of the process  and support for the projects selected.
While INS has an IRB, it is not functioning according to written policies and  procedures. Instead, the IRB operates according to undocumented  procedures for selecting new IT proposals. According to the Director of  INS' Investment Management Team, INS has begun developing written  policies and procedures and plans to complete them about March 2001.  However, until INS develops and implements these policies and  procedures, key IT investment activities may not be done consistently, if at  all. Table 2 summarizes the ratings for each key practice and the specific  findings supporting the ratings.
section_title: INS Is Not Effectively Overseeing Its Ongoing IT Projects
paragraphs: The purpose of project oversight is to ensure that the IRB provides  effective oversight for its ongoing IT projects throughout all phases of their  life cycle. Under stage 2 maturity, the IRB should review each project's  progress toward predefined cost and schedule expectations, using  established criteria, and take corrective actions when cost estimates and  project milestones are not achieved. Implementing this critical process  provides the basis for evolving the organization's IT investment control  activities.
According to ITIM, effective project oversight requires, among other things,  (1) having written polices and procedures for project management,  (2) developing and maintaining an approved project management plan for  each IT project, (3) having written policies and procedures for oversight of  IT projects, (4) making up-to-date cost and schedule data for each project  available to the IRB, (5) reviewing each project's performance by  comparing actual cost and schedule data to expectations regularly, and  (6) ensuring that corrective actions for each underperforming project are  defined, implemented, and tracked until the desired outcome is achieved.
INS is not effectively overseeing its IT projects. While INS has documented  policies and procedures for project management in its System  Development Life Cycle (SDLC) manual, it is not following its own  procedures. For example, INS has not developed and maintained project  management plans that include cost and schedule controls for each of its IT  projects, an SDLC requirement. In fact, only two of the four projects that  we reviewed had current project management plans.
Furthermore, INS does not have written polices and procedures for  oversight of its IT projects. Without written polices and procedures, INS  increases the risk that project oversight activities will not be performed  effectively. For example, the IRB does not (1) receive up-to-date cost and  schedule data for each project, (2) oversee each project's performance  regularly by comparing actual cost and schedule data to expectations, and  (3) ensure that corrective actions are implemented and tracked for  underperforming projects. In the absence of effective oversight, INS  executives do not have adequate assurance that IT projects are being  developed on schedule and within budget. Table 3 summarizes the ratings  for each key practice and the specific findings supporting the ratings.
section_title: INS Is Not Tracking and Using IT Asset Information for Investment Management Purposes
paragraphs: The purpose of the asset tracking critical process is to create and maintain  an IT asset inventory to assist in managerial decision-making. To make  good investment decisions, an organization must know where its IT assets  (i.e., personnel, systems, applications, hardware, software licenses, etc.)  are located and how funds are being expended toward acquiring,  maintaining, and deploying them. This critical process identifies IT assets  within the organization and creates a comprehensive inventory of them.  This inventory can take many forms, but regardless of form, the inventory  should identify each asset and its associated components.
Beyond identifying IT assets, this process is used to support other ITIM  critical processes by serving as an investment information and data  repository that contains such items as the list of systems and projects and  data on each project's progress toward achieving its plans. To support  investment decision-making, this inventory should also be accessible  where it is of the most value to decisionmakers.
According to ITIM, effectively tracking IT assets requires, among other  things, (1) making investment information available on demand to  decisionmakers, (2) developing and maintaining an IT asset inventory  according to written procedures, (3) overseeing the development and  maintenance of the asset tracking process, and (4) assigning responsibility  for managing this tracking process.
INS has not implemented an effective IT asset tracking process for  investment management. While investment information from various  sources has been available to the IRB on an ad hoc basis, it is not available  on demand and INS has not developed and maintained an inventory for  investment management purposes according to written policies and  procedures. In addition, the IRB does not oversee IT asset tracking  activities and has not assigned responsibility for managing this tracking  process to support investment decision-making. In the absence of standard,  documented procedures for developing and maintaining the inventory, INS  executives do not have adequate assurance that timely, complete, and  consistent asset data are available to them. Table 4 summarizes the ratings  for each key practice and the specific findings supporting the ratings.
section_title: INS Has Not Defined Business Needs for All Its IT Projects
paragraphs: The purpose of defining business needs for each IT project is to ensure that  each project supports the organization's business needs and meets users'  needs. Thus, this critical process creates the link between the  organization's business objectives and its IT management strategy.  According to ITIM, effectively identifying business needs requires, among  other things, (1) defining the organization's business needs or stated  mission goals, (2) identifying users for each project who will participate in  the project ‘s development and implementation, (3) defining business  needs for each project, and (4) training IT staff in business needs  identification.
INS has executed some of the key practices associated with effectively  defining business needs for IT projects. For example, INS has (1) defined  its business needs and mission goals in its annual performance plan and  (2) identified users for its projects who participate in the project ‘s  development and implementation. However, INS has not clearly defined  specific business needs for each project. In addition, only one of the four  project managers that we interviewed stated that he or she had been  trained in business needs identification. In the absence of documented  business needs, the IRB cannot ensure that it is selecting IT investments  that meet its mission needs and priorities. Table 5 summarizes the ratings  for each key practice and the specific findings supporting the ratings.
section_title: INS Has a Structured Process for Selecting New IT Proposals But Has Not Consistently Analyzed Them According to Established Criteria
paragraphs: The purpose of proposal selection is to establish a structured process for  selecting new IT proposals. According to ITIM, effective proposal selection  requires, among other things, (1) designating an official to manage the  proposal selection process, (2) using a structured process to develop new  IT proposals, (3) making funding decisions for new IT proposals according  to an established selection process, and (4) analyzing and ranking new IT  proposals according to established selection criteria, including cost and  schedule criteria.
INS has established a structured process for selecting new IT proposals.  The Deputy Commissioner, as the Chair of the IRB, is designated to manage  INS' proposal selection process. In addition, INS uses a structured process  to develop new proposals and makes initial funding decisions for these  proposals. However, INS has not consistently analyzed and ranked these  proposals according to established selection criteria. Established selection  criteria would assist IT managers in creating proposals that best meet the  needs and priorities of INS. Table 6 summarizes the ratings for each key  practice and the specific findings supporting the ratings.
section_title: INS Is Not Managing Its IT Investments as a Complete Portfolio
paragraphs: An IT investment portfolio is a collection of investments that are assessed  and managed based on common criteria. While an organization may have  more than one level of investment portfolios, it should always have an  enterprisewide portfolio. Managing investments as a portfolio is a  conscious, continuous, and proactive approach to expending limited  resources on all competing initiatives in light of the relative beneficial  effects of these investments. Taking an enterprisewide portfolio  perspective enables an organization to consider its investments  comprehensively so that the investments address its mission, strategic  goals, and objectives. A portfolio approach also allows an organization to  determine priorities and make decisions about which projects to fund  based on analyses of the relative costs, benefits, and risks of all projects,  including projects that are proposed, under development, and in operation.
The purpose of ITIM stage three maturity is to create and manage IT  investments as a complete enterprise investment portfolio. Once ongoing  projects can be implemented on schedule and within budget as is  emphasized in stage two, the organization is capable of managing its  projects as an investment portfolio. According to ITIM, stage three maturity  includes (1) defining portfolio selection criteria, (2) engaging in project- level investment analysis, (3) developing a complete portfolio based on the  investment analysis, and (4) maintaining oversight over the investment  performance of the portfolio.
INS has not implemented any of the critical processes in stage three. In  general, INS has not created the associated policies and procedures to  initiate or perpetuate any of the critical processes, and as a result, it has not  systematically collected and analyzed the data needed to make sound and  informed decisions about competing investment choices, which  consciously consider value and risk. In addition, while INS has established  eight portfolio categories, it has not established an enterprisewide  investment portfolio. Therefore, decisions may be made between  competing investments within a business area, but INS cannot make trade- offs between investments across the enterprise to determine which  projects contribute most to the agency mission and priorities. According to  INS officials, INS has not yet made IT investment management an  institutional priority. Table 7 summarizes INS' stage three maturity.
INS' capabilities for each of the stage three critical processes are discussed  below.
subsections: section_title: INS Has Not Created Useful Portfolio Selection Criteria
paragraphs: Portfolio selection criteria make up a necessary part of an IT investment  management process. Developing an enterprisewide investment portfolio  involves defining appropriate investment cost, benefit, schedule, and risk  criteria to ensure that the selected investments will best support the  organization's strategic goals, objectives, and mission. Thus, portfolio  selection criteria need to reflect the enterprisewide and strategic focus of  the organization. In addition, the criteria should (1) include cost, benefit,  schedule, and risk elements, which serve to create a common set of criteria  that are used to compare projects of different types to one another and  (2) be clearly communicated to project managers throughout the  organization so that these managers can take the criteria into account  when developing proposals. Without portfolio selection criteria, projects  may be selected on the basis of isolated business needs, the type and  availability of funds, or the receptivity of management to a specific project  proposal.
Thus, according to ITIM, developing portfolio selection criteria requires,  among other things, that (1) an investment board approve the criteria,  including cost, benefit, schedule, and risk criteria; (2) the criteria be  distributed throughout the organization; (3) adequate resources be  provided for selection criteria definition activities; and (4) a working group  be responsible for creating and modifying the criteria.
INS developed criteria for selecting new proposals; however, the criteria  had not been approved by the IRB and did not consistently include cost,  schedule, benefit, and risk criteria. Furthermore, INS had not distributed  the criteria throughout INS. For example, none of the IT project and  program managers that we interviewed were aware of the selection criteria  that had been developed. In addition, while INS indicated that it has  adequate resources to develop complete portfolio selection criteria, it has  not designated a working group to create and modify the criteria. Without  useful selection criteria, INS is missing a critical means of ensuring that  selected investments best support the organization's mission and priorities.  Table 8 summarizes the ratings for each key practice and the specific  findings supporting the ratings.
section_title: INS Does Not Analyze Its IT Investments Based on Cost, Benefit, Schedule, and Risk Data When Making Investment Decisions
paragraphs: The purpose of investment analysis is to ensure that all IT investments are  consistently analyzed and prioritized according to the organization's  portfolio selection criteria, which should include cost, benefit, schedule,  and risk criteria. According to ITIM, effective investment analysis includes,  among other things, that (1) portfolio selection criteria have been  developed; (2) the IRB ensures that cost, benefit, schedule, and risk data  are assessed and validated for each investment; (3) the IRB compares each  investment against the organization's portfolio selection criteria; and  (4) the IRB creates a ranked list of investments using the portfolio selection  criteria.
INS' IRB does not analyze and rank proposed and ongoing investments  based on their expected cost, benefit, schedule, and risk. As mentioned  previously, INS has not developed selection criteria that include these  elements, nor has it ensured that cost, benefit, schedule, and risk data are  assessed and validated for each IT investment. For example, none of the  four projects we reviewed provided cost, benefit, schedule, or risk data to  INS' IRB for consideration during the selection process. Instead, the IRB  focused on the near-term cost (e.g., annual budget dollars) of each project  and the perceived importance of the project to INS' mission. In the absence  of portfolio selection criteria and good investment-related data (i.e., cost,  benefit, schedule, and risk data), the IRB cannot compare and analyze its  investments based on their cost, benefit, schedule, and risk expectations  and create a ranked list of investments that best align with mission  improvement goals and organizational direction. As a result, INS is missing  critical information for making sound IT investment decisions. Table 9  summarizes the ratings for each key practice and the specific findings  supporting the ratings.
section_title: INS Does Not Comparatively Assess All Its IT Projects When Making Selections for Funding
paragraphs: The purpose of the portfolio development process is to ensure that the IRB  analyzes and compares all IT investments to select and fund those with  manageable risks and returns and that best address the strategic business  direction and priorities of the organization. Once this is accomplished,  investments can be compared to one another within and across the  portfolio categories and the best overall portfolio can then be selected for  funding.
According to ITIM, portfolio development requires, among other things,  (1) defining common portfolio categories and assigning each investment to  a portfolio category; (2) ensuring that investments have been analyzed and  their cost, benefit, schedule, and risk data validated; and (3) examining the  mix of investments across the portfolio categories in making funding  decisions.
INS does not assess all its IT projects in making selections for funding.  While INS has defined common portfolio categories, it is not using them to  manage its investments. INS has created eight portfolio categories and  assigned all of its investments to one of the portfolios. However, the IRB  has not analyzed these investments, including both proposed and ongoing  projects, based on validated cost, benefit, schedule, and risk data. Without  these meaningful data, the IRB cannot compare its investments across  portfolio categories. As a result, the IRB cannot make trade-offs between  investment alternatives, determine which projects contribute most to  agency performance, or eliminate redundant systems. Table 10 summarizes  the ratings for each key practice and the specific findings supporting the  ratings.
section_title: INS Does Not Oversee IT Investments' Cost, Benefit, Schedule, and Risk Performance
paragraphs: The purpose of the portfolio performance oversight critical process is to  ensure that each IT investment achieves its cost, benefit, schedule, and risk  expectations. This critical process builds upon the IT Project Oversight  critical process by adding the elements of benefit measurement and risk  management to an organization's investment control capacity. Executive- level oversight of project-level risk and benefit management activities  provides the organization with increased assurance that each investment  will achieve the desired cost, benefit, schedule, and risk results.
According to ITIM, effective portfolio performance oversight requires,  among other things, that the IRB (1) have access to up-to-date cost, benefit,  schedule, and risk data; (2) monitor the performance of each investment in  its portfolio by comparing actual project-level cost, benefit, schedule, and  risk data to the predefined expectations for the project; and (3) correct  poorly performing projects.
INS does not monitor its investments' performance to ensure that they are  meeting cost, benefit, schedule, and risk performance expectations. As  mentioned previously, up-to-date cost, benefit, schedule, and risk data are  not available. Without these data, the IRB is unable to monitor the  performance of its investments to ensure that they are achieving their cost,  benefit, schedule, and risk expectations and to act when performance  problems arise. Table 11 summarizes the ratings for each key practice and  the specific findings supporting the ratings.
section_title: The Department of Justice Is Not Guiding and Overseeing INS' Investment Management Approach
paragraphs: The Clinger-Cohen Act of 1996 imposed rigor and structure on how  agencies approach the selection and management of IT projects.  Among  other things, it requires the head of each agency to implement a process for  maximizing the value of the agency's IT investments and assess and  manage the risks of its IT investments. It also requires that the agency CIO  work with the agency head in implementing this process. As such, Justice is  responsible for ensuring that its bureaus and components, including INS,  implement an effective IT investment management process.
Justice has not provided INS, or any other Justice component, sufficient  direction, guidance, and oversight of IT investment management activities.  While Justice issued guidance in January 2000 describing its high-level  investment management process, the guidance does not address the need  or requirement for Justice's components to implement an IT investment  management process. Specifically, this guidance does not instruct the  components to establish IT investment management processes nor does it  establish expectations for doing so. According to Justice officials, Justice  had not established these processes because of other competing  department priorities, even though the department and its components  spent about $3 billion on IT in fiscal years 1999 and 2000.
During the course of our work, Justice began drafting IT investment  management policy and guidance documents in collaboration with an  intercomponent working group. The draft policy directs Justice  components to establish and use an IT investment management process  and directs the Justice CIO to monitor the components' investment  management processes through periodic briefings. A supplemental  guidance document provides procedures for developing an investment  management process. Justice officials stated that they plan to issue the  final policy by the end of December 2000 and the guidance by March 2001.  Until Justice issues its policy and guidance and begins monitoring its  components' progress, it has no assurance that it has the necessary  investment management processes in place to maximize the value of its IT  investments and manage the risks associated with them.
section_title: Conclusions, Recommendations, and Agency Comments
paragraphs: IT is critical to INS' ability to provide vital services, such as granting  naturalization benefits and detecting and preventing the illegal entry of  aliens into the United States. Effectively and efficiently managing IT  requires, among other things, a structured approach for minimizing the risk  and maximizing the return on IT investments. However, INS executives are  making investment decisions involving hundreds of millions of dollars  without vital data about these investments' relative costs, benefits, and  risks. As a result, INS cannot adequately know whether it is making the  right investment decisions, whether it has selected the mix of investments  that best meets its overall mission and business priorities, or whether these  investments are living up to expectations.
INS has initiated efforts to establish an IT investment management  foundation. However, it is lacking many important foundational investment  management capabilities, particularly those relating to controlling projects  against predetermined expectations and addressing variances. As a result,  it runs the serious risk that its IT projects will be late, cost more than  expected, and not perform as intended.
INS' use of portfolio categories and portfolio managers provides some  structure to its portfolio development process and provides each business  area the opportunity to identify the projects that it determines to be the  most important to its performance. However, INS' lack of performance data  from ongoing projects handicaps the IRB's ability to perform its portfolio  oversight function. In addition, the absence of any project-to-project  comparison limits the IRB's ability to judge whether its mix of investments  best meets its mission needs and priorities. As a result, INS can have little  confidence that its chosen mix of IT investments best meets mission goals  and priorities and that these investments will be developed within an  acceptable level of risk, on time, and within budget.
Further, Justice has a statutory role under the Clinger-Cohen Act to ensure  that its component agencies, including INS, have effective investment  management processes. Until Justice fulfills this role, it has little assurance  that INS, or its other components, are investing the department's limited IT  resources to maximize return on investment, minimize risk, and best  support mission needs.
subsections: section_title: Recommendations for Executive Action
paragraphs: To strengthen INS' investment management capability and address the  weaknesses discussed in this report, we recommend that you direct the  Commissioner of the Immigration and Naturalization Service to designate  development and implementation of effective IT investment management  processes as an agencywide priority and manage it as such. Specifically,  you should direct the Commissioner to do the following:
Develop a plan, within 9 months, for implementing IT investment  management process improvements that is based on stages two and  three critical processes and specifies measurable goals and time frames,  ranks initiatives, defines a management structure for directing and  controlling the improvements, establishes review milestones, and  recognizes any direction and guidance that Justice issues. This plan  should first focus on those critical processes in stage two of ITIM  because, collectively, they provide the foundation for building a mature  IT investment management process.
Submit the plan to the Justice CIO for review and approval.
Implement the approved plan and report to the Justice CIO, according to  established review milestones, on progress made against the plan's  goals and time frames.
Further, because the absence of effective investment management  processes and an enterprise architecture severely limits INS' ability to  effectively manage its IT investments, we recommend that until INS  develops a complete enterprise architecture and implements the key  practices associated with stages two and three critical processes, as  described in this report, you direct the Commissioner to limit requests for  future appropriations for IT only to efforts that support ongoing operations and maintenance, but not major  enhancements, of existing systems; support INS efforts to develop and implement IT investment  management processes and an enterprise architecture; are small, represent low technical risk, and can be delivered in a  relatively short period of time; or are congressionally mandated.
Further, to improve Justice's guidance and oversight of components' IT  investment management process activities, we also recommend that you  direct the Justice CIO to follow through on the department's plans to issue  an IT investment management policy and guidance to the components and  to ensure that the policy and guidance:
Directs Justice components and bureaus, including INS, to develop and  implement IT investment management processes.
Instructs Justice components and bureaus on how to develop an  investment management process. This guidance should be based on the  investment management guidance contained in this report and, at a  minimum, should include component roles, responsibilities, authorities,  and policies and procedures for developing an IT investment  management process.
Directs the Justice CIO to monitor the components' progress in  developing and establishing an IT investment management process and  take appropriate action if they are not progressing sufficiently.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, Justice's Assistant Attorney  General for Administration generally agreed with our recommendations,  although he offered minor wording modifications on two recommendations  that he said would increase Justice's ability to fully implement them. The  Assistant Attorney General for Administration also disagreed with our  finding that Justice is not guiding and directing INS' investment  management approach.
Justice generally agreed with our recommendation that INS develop and  submit to Justice a plan for implementing investment management process  improvements. However, Justice suggested that the time frame for  developing the plan be clarified such that INS has 6 months to develop and  submit its plan to Justice once Justice issues its new IT investment  management guidance. Because our recommendation directed INS to  consider any Justice guidance and direction in developing its investment  management process improvement plan, we modified the recommendation  to include an additional 3 months to allow time for Justice to issue its  guidance, which it plans to do in March 2001.
Justice also concurred with our recommendation that INS limit future  appropriation requests for IT to certain investment categories because it  lacks an enterprise architecture and effective investment management  processes, but suggested that we specify that this recommendation is in  effect until INS completes its architecture and implements investment  management processes. Because this is the intent of our recommendation,  we clarified the recommendation to make this explicit.
Also in its comments, Justice agreed that, while INS has some important  investment management capabilities, INS still needs to develop effective  investment management processes. Further, Justice agreed with our  recommendation for Justice to issue an investment management policy and  guidance to its components, including INS, that (1) directs components to  develop and implement IT investment management processes, (2) instructs  components on how to develop and implement these processes based on  the investment management framework in our report, and (3) ensures that  components' progress in doing so is monitored. Moreover, Justice stated,  which we note in our report, that it is now working with its components to  develop an IT investment management policy and process, and it has made  this a department priority for this year.
However, Justice stated that our draft report fails to recognize the extent of  Justice's oversight of INS' IT investment management process. Further, it  disagreed with our finding that Justice is not guiding and directing INS'  investment management approach. Justice stated that it has established  guidance for all aspects of IT management that its components are  expected to follow and has a process for overseeing components'  management of their investments. Justice cited six examples to illustrate  its point, such as Justice approval authority of all component IT  investments with life-cycle cost over $1 million, Justice establishment of an  IT investment board, Justice meetings with components, including  Attorney General meetings with the INS Commissioner, and Justice  forwarding of OMB budget requirements to components.
We do not agree with Justice's position. While we concur that the examples  cited by Justice represent important IT management functions to be  performed in providing management oversight of individual IT  investments, such management oversight is not the focus of our findings,  conclusions, and recommendations. Rather, our report addresses Justice's  efforts to ensure that its components, including INS, have each defined and  implemented effective IT investment management processes. As such, we  sought evidence from Justice demonstrating that it has directed its  components to establish such processes, provided guidance to its  components on how to develop and implement these processes, and  monitored its components' progress to determine whether they are  implementing such processes. However, besides the steps that Justice  initiated during the course of our inquiries and plans to take, which we  have described in this report, we found no such evidence. Moreover,  Justice stated in its written comments that it agreed with our  recommendation for it to provide investment management process  direction, guidance, and oversight to its components.
Justice's written comments and our evaluation of them are presented in  appendix I.