Homeland Security: Visitor and Immigrant Status Program Operating, but Management Improvements Are Still Needed
Released: 2006-01-25T00:00:00

Letter

Background

US-VISIT is a governmentwide program intended to enhance the security of U.S. citizens and visitors, facilitate legitimate travel and trade, ensure the integrity of the U.S. immigration system, and protect the privacy of our visitors. The scope of the program includes the pre-entry, entry, status, and exit of hundreds of millions of foreign national travelers who enter and leave the United States at over 300 air, sea, and land ports of entry, as well as analytical capabilities spanning this overall process.

To achieve its goals, US-VISIT uses biometric information (digital fingerscans and photographs) to verify identity and screen persons against watch lists. In many cases, the US-VISIT process begins overseas, at U.S. consular offices, which collect biometric information from applicants for visas, and check this information against a database of known criminals and suspected terrorists. When a visitor arrives at a port of entry, the biometric information is used to verify that the visitor is the person who was issued the visa or other travel documents. Ultimately, visitors are to confirm their departure by having their visas or passports scanned and undergoing fingerscanning. (Currently, at a few pilot sites, departing visitors are asked to undergo these exit procedures.) The exit confirmation is added to the visitor’s travel records to demonstrate compliance with the terms of admission to the United States.

Other key US-VISIT functions include ● collecting, maintaining, and sharing information on certain foreign nationals who enter and exit the United States; identifying foreign nationals who (1) have overstayed or violated the terms of their admission; (2) may be eligible to receive, extend, or adjust their immigration status; or (3) should be apprehended or detained by law enforcement officials; ● detecting fraudulent travel documents, verifying traveler identity, and determining traveler admissibility through the use of biometrics; and ● facilitating information sharing and coordination within the immigration and border management community.

In July 2003, DHS established a program office with responsibility for managing the acquisition, deployment, operation, and sustainment of the US-VISIT system and its associated supporting people (e.g., Customs and Border Protection officers), processes (e.g., entry/exit policies and procedures), and facilities (e.g., inspection booths and lanes).

As of October 2005, about $1.4 billion has been appropriated for the program, and according to program officials, about $962 million has been obligated to acquire, develop, deploy, operate, and maintain US-VISIT entry capabilities, and to test and evaluate exit capability options.

Acquisition and Implementation Strategy

DHS plans to deliver US-VISIT capability in four increments, with Increments 1 through 3 being interim, or temporary, solutions that fulfill legislative mandates to deploy an entry/exit system, and Increment 4 being the implementation of a long-term vision that is to incorporate improved business processes, new technology, and information sharing to create an integrated border management system for the future. In Increments 1 through 3, the program is building interfaces among existing (“legacy”) systems, enhancing the capabilities of these systems, and deploying these capabilities to air, sea, and land ports of entry. These first three increments are to be largely acquired and implemented through existing system contracts and task orders.

In May 2004, DHS awarded an indefinite-delivery/indefinite-quantity prime contract to Accenture and its partners. According to the contract, the prime contractor will help support the integration and consolidation of processes, functionality, and data, and it will develop a strategy to build on the technology and capabilities already available to produce the strategic solution, while also assisting the program office in leveraging existing systems and contractors in deploying the interim solutions.

US-VISIT Is Being Implemented in Four Increments

Increment 1 concentrates on establishing capabilities at air and sea ports of entry. It is divided into two parts—1A and 1B. ● Increment 1A (air and sea entry) includes the electronic capture and matching of biographic and biometric information (two digital index fingerscans and a digital photograph) for selected foreign nationals, including those from visa waiver countries. Increment 1A was deployed on January 5, 2004, through the modification of pre- existing systems. These modifications accommodated the collection and maintenance of additional data fields and established interfaces required to share data among DHS systems in support of entry processing at 115 airports and 14 seaports. ● Increment 1B (air and sea exit) involves the testing of exit devices to collect biometric exit data for select foreign nationals. Three exit alternatives were pilot tested at 11 air and sea ports of entry. These alternatives are as follows. ● Kiosk—A self-service device (including a touch screen interface, document scanner, finger scanner, digital camera, and receipt printer) that captures a digital photograph and fingerprint and prints out an encoded receipt. ● Mobile device—A hand-held device that is operated by a workstation attendant and includes a document scanner, finger scanner, digital camera, and receipt printer to capture a digital photograph and fingerprint. ● Validator—A hand-held device that is used to capture a digital photograph and fingerprint, which are then matched to the photograph and fingerprint captured via the kiosk and encoded in the receipt.

Increment 2 focuses primarily on extending US-VISIT to land ports of entry. It is divided into three parts—2A, 2B, and 2C. ● Increment 2A (air, sea, and land entry) includes the capability to biometrically compare and authenticate valid machine-readable visas and other travel and entry documents at all ports of entry. Increment 2A was deployed on October 23, 2005, according to program officials. It also includes the deployment by October 26, 2006, of the capability to read biometrically enabled passports from visa waiver countries. ● Increment 2B (land entry) redesigned the Increment 1 entry solution and expanded it to the 50 busiest land ports of entry. The process for issuing entry/exit forms was redesigned to enable the electronic capture of biographic, biometric (unless the traveler is exempt), and related travel documentation for arriving travelers. This increment was deployed to the busiest 50 U.S. land border ports of entry on December 29, 2004. Before Increment 2B, all information on the entry/exit forms was hand written. The redesigned process provides for electronically capturing the biographic data on the entry/exit form. In some cases, Customs and Border Protection (CBP) officers enter the data electronically and then print the completed form. ● Increment 2C (land entry and exit) is to provide the capability to automatically, passively, and remotely record the entry and exit of covered individuals using radio frequency (RF) technology tags at primary inspection and exit lanes. This tag includes a unique ID number that is to be embedded in each entry/exit form, thus associating a unique number with a US-VISIT record for the person holding that form. One of DHS’s goals in using this technology is to improve the ability to collect entry and exit information. In August 2005, the program office deployed the technology to three land ports of entry to verify the feasibility of using passive RF technology to record traveler entries and exits from the number embedded in the entry/exit form. The results of this demonstration are to be reported in February 2006.

Increment 3 extended Increment 2B (land entry) capabilities to 104 land ports of entry; this increment was essentially completed as of December 19, 2005.

Increment 4 is the strategic US-VISIT program capability, which program officials stated will likely consist of a further series of incremental releases or mission capability enhancements that will support business outcomes. The program reports that it has worked with its prime contractor and partners to develop this overall vision for the immigration and border management enterprise.

All increments before Increment 4 depend on the interfacing and integration of existing systems, including the following: ● The Arrival and Departure Information System (ADIS) stores ● noncitizen traveler arrival and departure data received from air and sea carrier manifests, ● arrival data captured by CBP officers at air and sea ports of ● I-94 issuance data captured by CBP officers at Increment 2B land ● departure information captured at US-VISIT biometric departure pilot (air and sea) locations, ● pedestrian arrival information and pedestrian and vehicle departure information captured at Increment 2C port of entry locations, and ● status update information provided by SEVIS and CLAIMS 3 (described below).

ADIS provides record matching, query, and reporting functions. ● The passenger processing component of the Treasury Enforcement Communications System (TECS) includes two systems: Advance Passenger Information System (APIS), a system that captures arrival and departure manifest information provided by air and sea carriers, and the Interagency Border Inspection System, a system that maintains lookout data and interfaces with other agencies’ databases. CBP officers use these data as part of the admission process. The results of the admission decision are recorded in TECS and ADIS. ● The Automated Biometric Identification System (IDENT) collects and stores biometric data about foreign visitors. ● The Student and Exchange Visitor Information System (SEVIS) and the Computer Linked Application Information Management System (CLAIMS 3) contain information on foreign students and foreign nationals who request benefits, such as change of status or extension of stay.

Some of these systems, such as IDENT, are managed by the program office, while some systems are managed by other organizational entities within DHS. For example, TECS is managed by CBP, SEVIS is managed by Immigration and Customs Enforcement, CLAIMS 3 is under United States Citizenship and Immigration Services, and ADIS is jointly managed by CBP and US-VISIT.

US-VISIT also interfaces with other, non-DHS systems for relevant purposes, including watch list updates and checks to determine whether a visa applicant has previously applied for a visa or currently has a valid U.S. visa. In particular, US-VISIT receives biographic and biometric information from the Department of State’s Consular Consolidated Database as part of the visa application process, and returns fingerscan information and watch list changes.

US-VISIT Capability Is Operating at Ports of Entry

Over the last 3 years, US-VISIT program officials and supporting contractor staff have worked to meet challenging legislative time frames, as well as a DHS-imposed requirement to use biometric identifiers. Under law, for example, DHS was to create an electronic entry and exit system to screen and monitor the stay of foreign nationals who enter and leave the United States and implement the system at (1) air and sea ports of entry by December 31, 2003, (2) the 50 highest-volume land ports of entry by December 31, 2004, and (3) the remaining ports of entry by December 31, 2005. It was also to provide the means to collect arrival/departure data from biometrically enabled and machine-readable travel documents at all ports of entry.

To the program office’s credit, it has largely met its obligations relative to an entry capability. For example, on January 5, 2004, it deployed and began operating most aspects of its planned entry capability at 115 airports and 14 seaports, and added the remaining aspects in February 2005. During 2004, it also deployed and began operating this entry capability in the secondary inspection areas of the 50 highest volume land ports of entry. As of December 19, 2005, it had deployed and begun operating its entry capability at all but 1 of the remaining 104 land ports of entry. The program has also been working to define feasible and cost-effective exit solutions, including technology feasibility testing at 3 land ports of entry and operational performance evaluations at 11 air and sea ports of entry.

Moreover, the development and deployment of this entry capability has occurred during a period of considerable organizational change, starting with the creation of DHS from 23 separate agencies in early 2003, followed by the establishment of a US-VISIT program office shortly thereafter—which was only about 5 months before it had to meet its first legislative milestone. Compounding these program challenges was the fact that the systems that were to be used in building and deploying an entry capability were managed and operated by a number of the separate agencies that had been merged to form the new department, each of which was governed by different policies, procedures, and standards.

As a result of the program’s efforts to deploy and operate an entry capability, DHS reports that it has been able to apprehend and prevent the entry of hundreds of criminal aliens: as of March 2005, DHS reported that more than 450 people with records of criminal or immigration violations have been prevented from entering. For example, its biometric screening prevented the reentry of a convicted felon, previously deported, who was attempting to enter under an alias; standard biographic record checks using only names and birth dates would have likely cleared the individual.

Another potential consequence, although difficult to demonstrate, is the deterrent effect of having an operational entry capability. Although deterrence is not an expressly stated goal of the program, officials have cited it as a potential byproduct of having a publicized capability at the border to screen entry on the basis of identity verification and matching against watch lists of known and suspected terrorists. Accordingly, the deterrent potential of the knowledge that unwanted entry may be thwarted and the perpetrators caught is arguably a layer of security that should not be overlooked.

DHS Has Yet to Demonstrate that US-VISIT as Defined Is the Right Solution

A prerequisite for prudent investment in programs is having reasonable assurance that a proposed course of action is the right thing to do, meaning that it properly fits within the larger context of an agency’s strategic plans and related operational and technology environments, and that the program will produce benefits in excess of costs over its useful life. We have made recommendations to DHS aimed at ensuring that this is in fact the case for US-VISIT, and the department has taken steps intended to address our recommendations. These steps, however, have yet to produce sufficient analytical information to demonstrate that US-VISIT as defined is the right solution. Without this knowledge, investment in the program cannot be fully justified.

Operational and Technological Context Are Still Being Defined

Agency programs need to properly fit within a common strategic context or frame of reference governing key aspects of program operations—e.g., what functions are to be performed by whom, when and where they are to be performed, what information is to be used to perform them, and what rules and standards will govern the application of technology to support them. Without a clear operational context for US-VISIT, the risk is increased that the program will not interoperate with related programs and thus not cost-effectively meet mission needs.

In September 2003 we reported that DHS had not defined key aspects of the larger homeland security environment in which US- VISIT would need to operate. For example, certain policy and standards decisions had not been made, such as whether official travel documents would be required for all persons who enter and exit the country—including U.S. and Canadian citizens—and how many fingerprints would be collected. Nonetheless, program officials were making assumptions and decisions at that time that, if they turned out to be inconsistent with subsequent policy or standards decisions, would require US-VISIT rework. To minimize the impact of these changes, we recommended that DHS clarify the context in which US-VISIT is to operate.

About 28 months later, defining this operational context remains a work in progress. For example, the program’s relationships and dependencies with other closely allied initiatives and programs are still unclear. According to the US-VISIT Chief Strategist, an immigration and border management strategic plan was drafted in March 2005 that shows how US-VISIT is aligned with DHS’s organizational mission and that defines an overall vision for immigration and border management. According to this official, the vision provides for an immigration and border management enterprise that unifies multiple internal departmental and other external stakeholders with common objectives, strategies, processes, and infrastructures. As of December 2005, however, we were told that this strategic plan has not been approved.

In addition, since the plan was drafted, DHS has reported that other relevant initiatives have been undertaken. For example: ● The DHS Security and Prosperity Partnership of North America is to, among other things, establish a common approach to securing the countries of North America—the United States, Canada, and Mexico—by, for example, implementing a border facilitation strategy to build capacity and improve the legitimate flow of people and cargo at our shared borders. ● The DHS Secure Border Initiative is to implement a comprehensive approach to securing our borders and combating illegal immigration.

According to the Chief Strategist, portions of the strategic plan are being incorporated into these initiatives, but these initiatives and their relationships with US-VISIT are still being defined.

Similarly, the mission and operational environment of US-VISIT are related to those of another major DHS program—the Automated Commercial Environment (ACE), which is a new trade processing system that is planned to support the movement of legitimate imports and exports and to strengthen border security. In addition, both US-VISIT and ACE could potentially use common IT infrastructures and services. As we reported in February 2005, the program office recognized these similarities, but managing the relationship between the two programs had not been a priority matter. Accordingly, we recommended that DHS give priority to understanding the relationships and dependencies between the US- VISIT and ACE programs.

Since our recommendation, the US-VISIT and ACE managers have formed an integrated project team to, among other things, ensure that the two programs are programmatically and technically aligned. Program officials stated that the team has met three times since April 2005 and plans to meet on a quarterly basis going forward. The team has discussed potential areas of focus and agreed to three areas: RF technology, program control, and data governance. However, it does not have an approved charter, and it has not developed explicit plans or milestone dates for identifying the dependencies and relationships between the two programs.

It is important that DHS define the operational context for US-VISIT, as well as its relationships and dependencies with closely allied initiatives and such programs as ACE. The more time it takes to settle these issues, the more likely that extensive and expensive rework will be needed at a later date.

Return on Investment Has Yet to be Determined

Prudent investment also requires that an agency have reasonable assurance that a proposed program will produce mission value commensurate with expected costs and risks. Thus far, DHS has yet to develop an adequate basis for knowing that this is the case for its early US-VISIT increments. Without this knowledge, it cannot adequately ensure that these increments are justified.

Assessments of costs and benefits are extremely important, because the decision to invest in any capability should be based on reliable analyses of return on investment. According to OMB guidance, individual increments of major systems are to be individually supported by analyses of benefits, cost, and risk. In addition, OMB guidance on the analysis needed to justify investments states that such analysis should meet certain criteria to be considered reasonable. These criteria include, among other things, comparing alternatives on the basis of net present value and conducting uncertainty analyses of costs and benefits. (DHS has also issued guidance on such economic analyses, which is consistent with that of OMB.) Without reliable analyses, an organization cannot be reasonably assured that a proposed investment is a prudent and justified use of resources.

In September 2003, we reported that the program had not assessed the costs and benefits of Increment 1. Accordingly, we recommended that DHS perform such assessments for future increments. In February 2005, we reported that although the program office had developed a cost-benefit analysis for Increment 2B (which provides the capability for electronic collection of traveler information at land ports of entry), it had again not justified the investment, because its treatment of both benefits and costs was unclear and insufficient. Further, we reported that the cost estimates on which the cost-benefit analysis was based were of questionable reliability, because effective cost-estimating practices were not followed. Accordingly, we recommended that DHS follow certain specified practices for estimating the costs of future increments.

Since our February 2005 report, the program has developed a cost- benefit analysis for Increment 1B (which is to provide exit capabilities at air and sea ports of entry). The latest version of this analysis, dated June 23, 2005, identifies potential costs and benefits for three exit solutions at air and sea ports of entry and provides a general rationale for the viability of the three alternatives described. This latest analysis meets some but not all the OMB criteria for economic analyses. For example, it explains why the investment was needed, and it shows that at least two alternatives to the status quo were considered. However, it does not include, for example, a complete uncertainty analysis for the three exit alternatives evaluated. That is, it does not include a sensitivity analysis for the three alternatives, which is a major part of an uncertainly analysis. (A sensitivity analysis is a quantitative assessment of the effect that a change in a given assumption—such as unit labor cost—will have on net present value.) A complete analysis of uncertainty is important because it provides decision makers with a perspective on the potential variability of the cost and benefit estimates should the facts, circumstances, and assumptions change.

In addition, the quality of a cost-benefit analysis is dependent on the quality of the cost assessments on which it is based. However, the cost estimate associated with the June 2005 cost-benefit analysis for the three exit solutions (Increment 1B) did not meet key criteria for reliable cost estimating. For example, it did not include a detailed work breakdown structure. A work breakdown structure serves to organize and define the work to be performed, so that associated costs can be identified and estimated. Thus, it provides a reliable basis for ensuring that the estimates include all relevant costs.

Program officials stated that they recognize the importance of developing reliable cost estimates and have initiated actions to more reliably estimate the costs of future increments. For example, the program has chartered a cost analysis process action team, which is to develop, document, and implement a cost analysis policy, process, and plan for the program. Program officials also stated that they have hired additional contracting staff with cost-estimating experience.

Strengthening the program’s cost-estimating capability is extremely important. The absence of reliable cost estimates impedes, among other things, both the development of reliable economic justification for program decisions and the effective measurement of performance.

Analysis of Program Impacts and Options Is Being Performed

Program decisions and planning depend on adequate analyses and assessments of program impacts and options. The department has begun to develop such analyses, but some of these, such as its analyses of the operational impact of Increment 2B and of the options for its exit capability, do not yet provide an adequate basis for investment and deployment decisions.

We reported in May 2004 that the program had not assessed its workforce and facility needs for Increment 2B (which provides the capability for electronic collection of traveler information at land ports of entry). Because of this, we questioned the validity of the program’s assumptions and plans concerning workforce and facilities, since the program lacked a basis for determining whether its assumptions were correct and thus whether its plans were adequate. Accordingly, we recommended that DHS assess the full impact of Increment 2B on workforce levels and facilities at land ports of entry, including performing appropriate modeling exercises.

Seven months later, the program office evaluated Increment 2B operational performance, with the stated purpose of determining the effectiveness of Increment 2B performance at the 50 busiest land ports of entry. For this evaluation, the program office established a baseline for comparing the average times to issue and process entry/exit forms at 3 of these 50 ports of entry. The program office then conducted two evaluations of the processing times at the three ports, first after Increment 2B was deployed as a pilot, and next 3 months later, after it was deployed to all 50 ports of entry. The evaluation results showed that the average processing times decreased for all three sites. Program officials concluded that these results supported their workforce and facilities planning assumptions that no additional staff was required to support deployment of Increment 2B and that minimal modifications were required at the facilities.

However, the scope of the evaluations is not sufficient to satisfy the evaluations’ stated purpose or our recommendation for assessing the full impact of 2B. For example, the selection of the three sites, according to program officials, was based on a number of factors, including whether the sites already had sufficient staff to support the pilot. Selecting sites based on this factor could affect the results, and it presupposes that not all ports of entry have the staff needed to support 2B. In addition, evaluation conditions were not always held constant: specifically, fewer workstations were used to process travelers in establishing the baseline processing times at two of the ports of entry than were used during the pilot evaluations.

Moreover, CBP officials from a land port of entry that was not an evaluation site (San Ysidro) told us that US-VISIT deployment has not reduced but actually lengthened processing times. (San Ysidro processes the highest volume of travelers of all land ports of entry.) Although these officials did not provide specific data to support their statement, their perception nevertheless raises questions about the potential impact of Increment 2B on the 47 sites that were not evaluated.

Similarly, in February 2005, we reported that US-VISIT had not adequately planned for evaluating the alternatives for Increment 1B (which provides exit capabilities at air and sea ports of entry) because the scope and timeline of its exit pilot evaluation were compressed. Accordingly, we recommended that DHS reassess plans for deploying an exit capability to ensure that the scope of the exit pilot provides for adequate evaluation of alternative solutions.

Over the last 11 months, the program office has taken actions to expand the scope and time frames of the pilot. For example, it increased the number of ports of entry in the pilot from 5 to 11, and it also extended the time frame by about 7 months. Further, according to program officials, they were able to achieve the target sample sizes necessary to have a 95 percent confidence level in their results.

Nevertheless, questions remain about whether the exit alternatives have been adequately evaluated to permit selection of the best exit solution for national deployment. For example, one of the criteria against which the alternatives were evaluated was the rate of traveler compliance with US-VISIT exit policies (that is, foreign travelers providing information as they exit the United States). However, across the three alternatives, the average compliance with these policies was only 24 percent, which raises questions as to their effectiveness. The evaluation report cites several reasons for the low compliance rate, including that compliance during the pilot was voluntary. The report further concludes that national deployment of the exit solution will not meet the desired compliance rate unless the exit process incorporates an enforcement mechanism, such as not allowing persons to reenter the United States if they do not comply with the exit process. Although an enforcement mechanism might indeed improve compliance, program officials stated that no formal evaluation has been conducted of enforcement mechanisms or their possible effect on compliance. The program director agreed that additional evaluation is needed to assess the impact of implementing potential enforcement mechanisms and plans to do such evaluation.

DHS Is Still Establishing Needed Program Management Capabilities

Establishing effective program management capabilities is important to ensure that an organization is going about delivering a program in the right way. Accordingly, we have made recommendations to establish specific people and process management capabilities. While DHS is making progress in implementing many of our recommendations in this area, this progress has often been slow.

One area in which DHS has made good progress is in implementing our recommendations to establish the human capital capabilities necessary to manage US-VISIT. In September 2003, we reported that the US-VISIT program had not fully staffed or adequately funded its program office or defined specific roles and responsibilities for program office staff. Our prior experience with major acquisitions like US-VISIT shows that to be successful, they need, among other things, to have adequate resources, and program staff need to understand what they are to do, how they relate to each other, and how they fit in their organization. In addition, prior research and evaluations of organizations show that effective human capital management can help agencies establish and maintain the workforce they need to accomplish their missions. Accordingly, we recommended that DHS ensure that human capital and financial resources are provided to establish a fully functional and effective program office, and that the department define program office positions, roles, and responsibilities. We also recommended that DHS develop and implement a human capital strategy for the program office that provides for staffing positions with individuals who have the appropriate knowledge, skills, and abilities.

DHS has implemented our recommendation that it define program office positions, roles, and responsibilities, and it has partially completed our two other people-related recommendations. It has filled most of its planned government positions and is on the way to filling the rest, and it has filled all of its planned contractor positions. However, the program completed a workforce analysis in February 2005 and requested additional positions based on the results. Securing these necessary resources will be a continuing challenge.

In addition, as we reported in February 2005, the program office, working with the Office of Personnel Management, developed a draft human capital plan that employed widely accepted human capital planning tools and principles (for example, it included an action plan that identified activities, their proposed completion dates, and the office responsible for the action). In addition, the program office had completed some of the activities in the plan. Since then, the program office has finalized the human capital plan, completed more activities, and formulated plans to complete others (for example, according to the program office, it has completed an analysis of its workforce to determine diversity trends, retirement and attrition rates, and mission-critical and leadership competency gaps, and it has plans to complete an analysis of workforce data to maintain strategic focus on preserving the skills, knowledge, and leadership abilities required for the US-VISIT program’s success).

Program officials also said that the reason they have not completed several activities in the plan is that these activities are related to the department’s new human capital initiative, MAXHR. Because this initiative is to include the development of departmentwide competencies, program officials told us that it could potentially affect ongoing program activities related to competencies. As a result, these officials said that they are coordinating these activities closely with the department as it develops and implements this new initiative, which is currently being reviewed by the DHS Deputy Secretary.

DHS’s progress in implementing our human capital recommendations should help ensure that it has sufficient staff with the right skills and abilities to successfully execute the program. Having such staff has been and will be particularly important in light of the program’s more limited progress to date in establishing program management process capabilities.

DHS’s progress in establishing effective processes governing how program managers and staff are to perform their respective roles and responsibilities has generally been slow. In our experience, weak process management controls typically result in programs falling short of expectations. From September 2003, we have made numerous recommendations aimed at enabling the program to strengthen its process controls in such areas as acquisition management, test management, risk management, configuration management, capacity management, security, privacy, and independent verification and validation (IV&V). DHS has not yet completed the implementation of any of our recommendations in these areas, with one exception. It has ensured that the program office’s IV&V contractor was independent of the products and processes that it was verifying and validating, as we recommended. In July 2005, the program office issued a new contract for IV&V services after following steps to ensure the contractor’s independence (for example, IV&V contract bidders were to be independent of the development and integration contractors and are prohibited from soliciting, proposing, or being awarded work for the program other than IV&V services). If effectively implemented, these steps should adequately ensure that verification and validation activities are performed in an objective manner, and thus should provide valuable assistance to program managers and decision makers.

In the other management areas, DHS has partially completed or has only begun to address our recommendations, and more remains to be done. For example, DHS has not completed the development and implementation of key acquisition controls. We reported in September 2003 that the program office had not defined key acquisition management controls to support the acquisition of US- VISIT, increasing the risk that the program would not satisfy system requirements or meet benefit expectations on time and within budget. Accordingly, we recommended that DHS develop and implement a plan for satisfying key acquisition management controls in accordance with best practices.

The program office has recently taken steps to lay the foundation for establishing key acquisition management controls. For example, it has developed a process improvement plan to define and implement these controls that includes a governance structure for overseeing improvement activities. In addition, the program office has recently completed a self-assessment of its acquisition process maturity, and it plans to use the assessment results to establish a baseline of its acquisition process maturity as a benchmark for improvement. According to program officials, the assessment included key process areas that are generally consistent with the process areas cited in our recommendation. The program has ranked these process areas and plans to focus on those with highest priority. (Some of these high-priority process areas are also areas in which we have made recommendations, such as configuration management and risk management.)

The improvement plan is currently being updated to reflect the results of the baseline assessment and to include a work breakdown structure, process prioritization, and resource estimates. According to a program official, the goal is to conduct a formal appraisal to assess the capability level of some or all of the high-priority process areas by October 2006.

These recent steps provide a foundation for progress, but fully and effectively implementing key acquisition management controls takes considerable time, and DHS is still in the early stages of the process. Therefore, it is important that these improvement efforts stay on track. Until these controls are effectively implemented, US-VISIT will be at risk of not delivering promised capabilities on time and within budget.

Another management area of high importance to a complex program like US-VISIT is test management. The purpose of system testing is to identify and correct system defects before the system is deployed. To be effective, testing activities should be planned and implemented in a structured and disciplined fashion. Among other things, this includes developing effective test plans to guide the testing activities and ensuring that test plans are developed and approved before test execution.

In this area also, DHS’s progress responding to our recommendation has been limited. We reported in May 2004, and again in February 2005, that system testing was not based on well-defined test plans, and thus the quality of testing being performed was at risk. Because DHS test plans were not sufficiently well-defined to be effective, we recommended that before testing begins, DHS develop and approve test plans that meet the criteria that relevant systems development guidance prescribes for effective test plans: namely, that they (1) specify the test environment; (2) describe each test to be performed, including test controls, inputs, and expected outputs; (3) define the test procedures to be followed in conducting the tests; and (4) provide traceability between the test cases and the requirements to be verified by the testing.

About 20 months later, the quality of the system test plans, and thus system testing, is still a challenge. To the program’s credit, the test plans for the Proof of Concept for Increment 2C, dated June 28, 2005 (which introduces RF technology to automatically record the entry and exit of covered individuals), satisfied part of our recommendation. Specifically, the test plan for this increment was approved on June 30, 2005, before testing began (according to program officials, it began on July 5, 2005). Further, the test plan described, for example, the scope, complexity, and completeness of the test environment; it described the tests to be performed, including a high-level description of controls, inputs, and outputs; and it identified the test procedures to be performed.

However, the test plan did not adequately trace between test cases and the requirements to be verified by testing. For example, about 70 percent of the requirements that we analyzed did not have specific references to test cases. Further, we identified traceability inconsistencies, such as one requirement that was mapped to over 50 test cases, even though none of the 50 cases referenced the requirement.

Time and resource constraints were identified as the reasons that test plans have not been complete. Specifically, program officials stated that milestones do not permit existing testing/quality personnel the time required to adequately review testing documents. According to these officials, even when the start of testing activities is delayed because, for example, requirements definition or product development takes longer than anticipated, testing milestones are not extended.

Without complete test plans, the program does not have adequate assurance that the system is being fully tested, and thus unnecessarily assumes the risk of system defects not being detected and addressed before the system is deployed. This means that the system may not perform as intended when deployed, and defects will not be addressed until late in the systems development cycle, when they are more difficult and time-consuming to fix. This has in fact happened already: postdeployment system interface problems surfaced for Increment 1, and manual work-arounds had to be implemented after the system was deployed.

Until process management weaknesses such as these are addressed, the program will continue to be overly dependent on the exceptional performance of individuals to produce results. Such dependence increases the risk of the US-VISIT program falling short of expectations.

DHS Has Yet to Fully Establish Program Accountability Mechanisms

To better ensure that US-VISIT and DHS meet expectations, we made recommendations related to measuring and disclosing progress against program commitments. Thus far, such performance and accountability mechanisms have yet to be fully established. Measurements of the operational performance of the system are necessary to ensure that the system adequately supports mission operations, and measurements of program progress and outcomes are important for demonstrating that the program is on track and is producing results. Without such measurements, program performance and accountability can suffer.

As we reported in September 2003, the operational performance of initial system increments was largely dependent on the performance of existing systems that were to be interfaced to create these increments. For example, we said that the performance of an increment would be constrained by the availability and downtime of the existing systems, some of which had known problems in these areas. Accordingly, we recommended that DHS define performance standards for each increment that are measurable and that reflect the limitations imposed by this reliance on existing systems. In February 2005, we reported that several technical performance standards for increments 1 and 2B had been defined, but that it was not clear that these standards reflected the limitations imposed by the reliance on existing systems. Since then, the program office has defined certain other technical performance standards for the next increment (Increment 2C, Phase 1), including standards for availability. Consistent with what we reported, the functional requirements document states that these performance standards are largely dependent upon those of the current systems, and for system availability, it sets an aggregated availability standard for Increment 2C components. However, the document does not contain sufficient information for a determination of whether these performance standards actually reflect the limitations imposed by reliance on existing systems. Unless the program defines performance standards that do this, it will be unable to identify and effectively address performance shortfalls.

Similarly, as we observed in June 2003, to permit meaningful program oversight, it is important that expenditure plans describe how well DHS is progressing against the commitments made in prior expenditure plans. The expenditure plan for fiscal year 2005 (the fourth US-VISIT expenditure plan) does not describe progress against commitments made in the previous plans. For example, according to the fiscal year 2004 plan, US-VISIT was to analyze, field test, and begin deploying alternative approaches for capturing biometrics during the exit process. However, according to the fiscal year 2005 plan, US-VISIT was to expand its exit pilot sites during the summer and fall of 2004, and it would not deploy the exit solution until fiscal year 2005. The plan does not explain the reason for this change from its previous commitment nor its potential impact. Nor does it describe the status of the exit pilot testing or deployment, such as whether the program has met its target schedule or whether the schedule has slipped.

Additionally, the fiscal year 2004 plan stated that $45 million in fiscal year 2004 was to be used for exit activities. However, in the fiscal year 2005 plan, the figure for exit activities was $73 million in fiscal year 2004 funds. The plan does not highlight this difference or address the reason for the change in amounts. Also, although the fiscal year 2005 expenditure plan includes benefits stated in the fiscal year 2004 plan, it does not describe progress in addressing those benefits, even though in the earlier plan, US-VISIT stated that it was developing metrics for measuring the projected benefits, including baselines by which progress could be assessed. The fiscal year 2005 plan again states that performance measures are under development.

Figure 1 provides our analysis of the commitments made in the fiscal year 2003 and 2004 plans, compared with progress reported and planned in February 2005.

The deployment of an exit capability, an important aspect of the program that was to result from the exit pilots shown in the figure, further illustrates missed commitments that need to be reflected in the next expenditure plan. In the fiscal year 2005 expenditure plan, the program committed to deploying an exit capability to air and sea ports of entry by September 30, 2005. Although US-VISIT has completed its evaluation of exit solutions at 11 pilot sites (9 airports and 2 seaports), no decision has yet been made on when an exit capability will be deployed. According to program officials, deployment to further sites would take at least 6 months from the time of the decision. This means that the program office will not meet its commitment.

Another accountability mechanism that we recommended in May 2004 is for the program to develop a plan, including explicit tasks and milestones, for implementing all our open recommendations, and report on progress, including reasons for delays, both to department leadership (the DHS Secretary and Under Secretary) in periodic reports and to the Congress in all future expenditure plans. The department has taken action to address this recommendation, but the initial report does not disclose enough information for a complete assessment of progress. The program office did assign responsibility to specific individuals for preparing the implementation plan, and it developed a report identifying the person responsible for each recommendation and summarizing progress. This report was provided for the first time to the DHS Deputy Secretary on October 3, 2005, and the program office plans to forward subsequent reports every 6 months. However, some of the report’s progress descriptions are inconsistent with our assessment. For example, the report states that the impact of Increment 2B on workforce levels and facilities at land ports of entry has been fully assessed. However, as mentioned earlier, evaluation conditions were not always held constant—that is, fewer workstations were used to process travelers in establishing the baseline processing times at two of the ports of entry than were used during the pilot evaluations.

In addition, the report does not specifically describe progress against most of our recommendations. For example, we recommended that the program reassess plans for deploying an exit capability to ensure that the scope of the exit pilot provides for adequate evaluation of alternative solutions. With regard to the exit evaluation, the report states that the program office has completed exit testing and has forwarded the exit evaluation report to the Deputy Secretary for a decision. However, it does not state whether the program office had expanded the scope or time frames of the pilot.

In closing, I would emphasize that the program has met many of the demanding requirements in law for deployment of an entry-exit system, owing, in large part, to the hard work and dedication of the program office and its contractors, as well as the close oversight and direction of the House and Senate Appropriations Committees. Nevertheless, core capabilities, such as exit, have yet to be established and implemented, and fundamental questions about the program’s fit within the larger homeland security context and its return on investment remain unanswered. Moreover, the program is overdue in establishing the means to effectively manage the delivery of future capabilities. The longer the program proceeds without these, the greater the risk that the program will not meet its commitments. Measuring and disclosing the extent to which these commitments are being met are also essential to holding the department accountable, and thus are an integral aspect of effective program management. Our recommendations provide a comprehensive framework for addressing each of these important areas and thus ensuring that the program as defined is the right solution, that delivery of this solution is being managed in the right way, and that accountability for both is in place. We look forward to continuing to work constructively with the program to better ensure the program’s success.

Mr. Chairman, this concludes my statement. I would be happy to answer any questions that you or members of the committee may have at this time.

Contact and Acknowledgement

If you should have any questions about this testimony, please contact Randolph C. Hite at (202) 512-3439 or hiter@gao.gov. Other major contributors to this testimony included Tonia Brown, Barbara Collier, Deborah Davis, James Houtz, Scott Pettis, and Dan Wexler.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.