Homeland Security: Visitor and Immigrant Status Program Operating, but Management Improvements Are Still Needed
Published: 2006-01-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Homeland Security (DHS) has established a program--the U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)--to collect, maintain, and share information, including biometric identifiers, on selected foreign nationals who enter and exit the United States. US-VISIT uses these biometric identifiers (digital fingerscans and photographs) to screen persons against watch lists and to verify that a visitor is the person who was issued a visa or other travel document. Visitors are also to confirm their departure by having their visas or passports scanned and undergoing fingerscanning at selected air and sea ports of entry. GAO was asked to testify on (1) the status of US-VISIT and (2) DHS progress in implementing recommendations that GAO made as part of its prior reviews of US-VISIT annual expenditure plans. The testimony is based on GAO's prior reports as well as ongoing work for the House Committee on Homeland Security. GAO's recommendations are directed at helping the department improve its capabilities to deliver US-VISIT capability and benefit expectations on time and within budget. According to DHS, the recommendations have made US-VISIT a stronger program.
section_title: What GAO Found
paragraphs: The US-VISIT program has met a number of demanding requirements that were mandated in legislation. A pre-entry screening capability is in place in overseas visa issuance offices, and an entry identification capability is operating at 115 airports, 14 seaports, and 154 land ports of entry. This has been accomplished during a period of DHS-wide change, and has resulted in preventing criminal aliens from entering the country and potentially deterring others from even attempting to do so. Nevertheless, DHS has more to do to implement GAO recommendations aimed at better ensuring that US-VISIT is maximizing its potential for success and holding itself accountable for results. DHS has taken steps to address those GAO recommendations intended to ensure that US-VISIT as defined is the "right thing." For example, it is clarifying the strategic context within which US-VISIT is to operate, having drafted a strategic plan to show how US-VISIT is aligned with DHS's mission goals and operations and to provide an overall vision for immigration and border management. However, the plan has yet to be approved, causing its integration with other departmentwide border security initiatives to remain unclear. In addition, the department has analyzed the program's costs, benefits, and risks, but its analyses do not yet demonstrate that the program is producing or will produce mission value commensurate with expected costs and risks. In particular, the department's return-on-investment analyses for exit options do not demonstrate that these solutions will be cost-effective. DHS has also taken steps to address those GAO recommendations aimed at ensuring that the program is executed in the "right way." The department has made good progress in establishing the program's human capital capabilities, which should help ensure that it has sufficient staff with the necessary skills and abilities. This is particularly important in light of the program's more limited progress in establishing capabilities in certain program management process areas, such as test management. For example, a test plan used in a recent system acceptance test did not adequately trace between test cases and the requirements to be verified by testing. Incomplete test plans reduce assurance that systems will perform as intended once they are deployed. DHS also has begun addressing GAO's recommendations to establish accountability for program performance and results, but more needs to be done. For example, DHS's expenditure plans have not described progress against commitments made in previous plans. Unless performance against commitments is measured and disclosed, the ability to manage and oversee the program will suffer. The longer the program proceeds without fully addressing GAO's recommendations, the greater the risk that it will not deliver promised capabilities and benefits on time and within budget.

section_title: Letter
paragraphs: We appreciate the opportunity to participate in the Subcommittee’s  hearing on US-VISIT (the United States Visitor and Immigrant Status  Indicator Technology), a multibillion-dollar program of the  Department of Homeland Security (DHS) that is intended to achieve  a daunting set of goals: to enhance the security of our citizens and  visitors and ensure the integrity of the U.S. immigration system, and  at the same time to facilitate legitimate trade and travel and protect  privacy. To achieve these goals, US-VISIT is to record the entry into  and exit from the United States of selected travelers, verify their  identity, and determine their compliance with the terms of their  admission and stay.
Since fiscal year 2002, the House and Senate Appropriations  Committees have provided valuable oversight and direction to DHS  on US-VISIT by legislatively directing it to submit annual  expenditure plans for committee approval. This legislation also  directed us to review these plans. Our reviews have produced four  reports that, among other things, described DHS progress against  legislatively mandated milestones and identified fundamental  challenges that the department faced in delivering promised  program capabilities and benefits on time and within cost. For  example, we reported in September 2003 that the program office did  not have the human capital and acquisition process discipline  needed to effectively manage the program. In light of the challenges  that we identified, we concluded that the program carries an  appreciable level of risk, meaning that it must be managed  effectively if it is to be successful.
Managing US-VISIT effectively requires high levels of capability and  expertise. Fundamentally, it entails being able to respond  affirmatively to two basic questions. First, are we doing the right  thing? To be sure that a program is doing the right thing, it needs to  be justified by sufficient fact-based and verifiable analysis to show  that the program as defined will properly fit within the larger  homeland security operational and technological environments and  that it will produce mission value commensurate with expected  costs and risks. The second question is, are we doing it the right  way? To be done the right way, a program needs to be executed in a  rigorous and disciplined manner, which means that it needs to  employ the necessary mix of people, processes, and tools to  reasonably ensure that promised program capabilities and expected  mission value are delivered on time and within budget. Beyond  these two questions, effective program management also means that  the program is held accountable for results, which involves  measuring and disclosing performance relative to explicitly defined  program goals, outcomes, and commitments.
Over the last 4 years, our reports have provided recommendations  to DHS to ensure that these questions are answered and used as the  basis for informed decision making about US-VISIT. They have also  provided recommendations to promote DHS accountability for the  program. These recommendations have been aimed at helping the  department to ensure that this program fulfills expectations: in  other words, that the program is doing the right thing in the right  way, and that it is holding itself accountable for doing so. According  to DHS, the recommendations have made US-VISIT a stronger  program. Further, they concur with the need to implement them  with due speed and diligence.
My statement will describe the status of US-VISIT and where the  department now stands in implementing these recommendations  and thus in addressing the challenges that it faces. It is based on our  aforementioned reports to the Appropriations Committees and our  ongoing work for the House Committee on Homeland Security. All  work on which this testimony is based was performed in  accordance with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: US-VISIT is a governmentwide program intended to enhance the  security of U.S. citizens and visitors, facilitate legitimate travel and  trade, ensure the integrity of the U.S. immigration system, and  protect the privacy of our visitors. The scope of the program  includes the pre-entry, entry, status, and exit of hundreds of millions  of foreign national travelers who enter and leave the United States  at over 300 air, sea, and land ports of entry, as well as analytical  capabilities spanning this overall process.
To achieve its goals, US-VISIT uses biometric information (digital  fingerscans and photographs) to verify identity and screen persons  against watch lists. In many cases, the US-VISIT process begins  overseas, at U.S. consular offices, which collect biometric  information from applicants for visas, and check this information  against a database of known criminals and suspected terrorists.  When a visitor arrives at a port of entry, the biometric information is  used to verify that the visitor is the person who was issued the visa  or other travel documents. Ultimately, visitors are to confirm their  departure by having their visas or passports scanned and  undergoing fingerscanning. (Currently, at a few pilot sites, departing  visitors are asked to undergo these exit procedures.) The exit  confirmation is added to the visitor’s travel records to demonstrate  compliance with the terms of admission to the United States.
Other key US-VISIT functions include  ●  collecting, maintaining, and sharing information on certain foreign  nationals who enter and exit the United States;  identifying foreign nationals who (1) have overstayed or violated the  terms of their admission; (2) may be eligible to receive, extend, or  adjust their immigration status; or (3) should be apprehended or  detained by law enforcement officials;  ●  detecting fraudulent travel documents, verifying traveler identity,  and determining traveler admissibility through the use of biometrics;  and  ●  facilitating information sharing and coordination within the  immigration and border management community.
In July 2003, DHS established a program office with responsibility  for managing the acquisition, deployment, operation, and  sustainment of the US-VISIT system and its associated supporting  people (e.g., Customs and Border Protection officers), processes  (e.g., entry/exit policies and procedures), and facilities (e.g.,  inspection booths and lanes).
As of October 2005, about $1.4 billion has been appropriated for the  program, and according to program officials, about $962 million has  been obligated to acquire, develop, deploy, operate, and maintain  US-VISIT entry capabilities, and to test and evaluate exit capability  options.
subsections: section_title: Acquisition and Implementation Strategy
paragraphs: DHS plans to deliver US-VISIT capability in four increments, with  Increments 1 through 3 being interim, or temporary, solutions that  fulfill legislative mandates to deploy an entry/exit system, and  Increment 4 being the implementation of a long-term vision that is to  incorporate improved business processes, new technology, and  information sharing to create an integrated border management  system for the future. In Increments 1 through 3, the program is  building interfaces among existing (“legacy”) systems, enhancing  the capabilities of these systems, and deploying these capabilities to  air, sea, and land ports of entry. These first three increments are to  be largely acquired and implemented through existing system  contracts and task orders.
In May 2004, DHS awarded an indefinite-delivery/indefinite-quantity  prime contract to Accenture and its partners. According to the  contract, the prime contractor will help support the integration and  consolidation of processes, functionality, and data, and it will  develop a strategy to build on the technology and capabilities  already available to produce the strategic solution, while also  assisting the program office in leveraging existing systems and  contractors in deploying the interim solutions.
section_title: US-VISIT Is Being Implemented in Four Increments
paragraphs: Increment 1 concentrates on establishing capabilities at air and sea  ports of entry. It is divided into two parts—1A and 1B.  ●  Increment 1A (air and sea entry) includes the electronic capture and  matching of biographic and biometric information (two digital index  fingerscans and a digital photograph) for selected foreign nationals,  including those from visa waiver countries. Increment 1A was  deployed on January 5, 2004, through the modification of pre- existing systems. These modifications accommodated the collection  and maintenance of additional data fields and established interfaces  required to share data among DHS systems in support of entry  processing at 115 airports and 14 seaports.   ●  Increment 1B (air and sea exit) involves the testing of exit devices  to collect biometric exit data for select foreign nationals. Three exit  alternatives were pilot tested at 11 air and sea ports of entry. These  alternatives are as follows.   ●  Kiosk—A self-service device (including a touch screen interface,  document scanner, finger scanner, digital camera, and receipt  printer) that captures a digital photograph and fingerprint and  prints out an encoded receipt.   ●  Mobile device—A hand-held device that is operated by a  workstation attendant and includes a document scanner, finger  scanner, digital camera, and receipt printer to capture a digital  photograph and fingerprint.  ●  Validator—A hand-held device that is used to capture a digital  photograph and fingerprint, which are then matched to the  photograph and fingerprint captured via the kiosk and encoded  in the receipt.
Increment 2 focuses primarily on extending US-VISIT to land ports  of entry. It is divided into three parts—2A, 2B, and 2C.  ●  Increment 2A (air, sea, and land entry) includes the capability to  biometrically compare and authenticate valid machine-readable  visas and other travel and entry documents at all ports of entry.  Increment 2A was deployed on October 23, 2005, according to  program officials. It also includes the deployment by October 26,  2006, of the capability to read biometrically enabled passports from  visa waiver countries.  ●  Increment 2B (land entry) redesigned the Increment 1 entry solution  and expanded it to the 50 busiest land ports of entry. The process  for issuing entry/exit forms was redesigned to enable the electronic  capture of biographic, biometric (unless the traveler is exempt), and  related travel documentation for arriving travelers. This increment  was deployed to the busiest 50 U.S. land border ports of entry on  December 29, 2004. Before Increment 2B, all information on the  entry/exit forms was hand written. The redesigned process provides  for electronically capturing the biographic data on the entry/exit  form. In some cases, Customs and Border Protection (CBP) officers  enter the data electronically and then print the completed form.   ●  Increment 2C (land entry and exit) is to provide the capability to  automatically, passively, and remotely record the entry and exit of  covered individuals using radio frequency (RF) technology tags at  primary inspection and exit lanes. This tag includes a unique ID  number that is to be embedded in each entry/exit form, thus  associating a unique number with a US-VISIT record for the person  holding that form. One of DHS’s goals in using this technology is to  improve the ability to collect entry and exit information. In August  2005, the program office deployed the technology to three land ports  of entry to verify the feasibility of using passive RF technology to  record traveler entries and exits from the number embedded in the  entry/exit form. The results of this demonstration are to be reported  in February 2006.
Increment 3 extended Increment 2B (land entry) capabilities to 104  land ports of entry; this increment was essentially completed as of  December 19, 2005.
Increment 4 is the strategic US-VISIT program capability, which  program officials stated will likely consist of a further series of  incremental releases or mission capability enhancements that will  support business outcomes. The program reports that it has worked  with its prime contractor and partners to develop this overall vision  for the immigration and border management enterprise.
All increments before Increment 4 depend on the interfacing and  integration of existing systems, including the following:  ●  The Arrival and Departure Information System (ADIS) stores   ●  noncitizen traveler arrival and departure data received from air  and sea carrier manifests,   ●  arrival data captured by CBP officers at air and sea ports of  ●  I-94 issuance data captured by CBP officers at Increment 2B land  ●  departure information captured at US-VISIT biometric departure  pilot (air and sea) locations,   ●  pedestrian arrival information and pedestrian and vehicle  departure information captured at Increment 2C port of entry  locations, and  ●  status update information provided by SEVIS and CLAIMS 3  (described below).
ADIS provides record matching, query, and reporting functions.  ●  The passenger processing component of the Treasury Enforcement  Communications System (TECS) includes two systems: Advance  Passenger Information System (APIS), a system that captures arrival  and departure manifest information provided by air and sea carriers,  and the Interagency Border Inspection System, a system that  maintains lookout data and interfaces with other agencies’  databases. CBP officers use these data as part of the admission  process. The results of the admission decision are recorded in TECS  and ADIS.  ●  The Automated Biometric Identification System (IDENT) collects  and stores biometric data about foreign visitors.  ●  The Student and Exchange Visitor Information System (SEVIS) and  the Computer Linked Application Information Management System  (CLAIMS 3) contain information on foreign students and foreign  nationals who request benefits, such as change of status or  extension of stay.
Some of these systems, such as IDENT, are managed by the program  office, while some systems are managed by other organizational  entities within DHS. For example, TECS is managed by CBP, SEVIS  is managed by Immigration and Customs Enforcement, CLAIMS 3 is  under United States Citizenship and Immigration Services, and ADIS  is jointly managed by CBP and US-VISIT.
US-VISIT also interfaces with other, non-DHS systems for relevant  purposes, including watch list updates and checks to determine  whether a visa applicant has previously applied for a visa or  currently has a valid U.S. visa. In particular, US-VISIT receives  biographic and biometric information from the Department of  State’s Consular Consolidated Database as part of the visa  application process, and returns fingerscan information and watch  list changes.
section_title: US-VISIT Capability Is Operating at Ports of Entry
paragraphs: Over the last 3 years, US-VISIT program officials and supporting  contractor staff have worked to meet challenging legislative time  frames, as well as a DHS-imposed requirement to use biometric  identifiers. Under law, for example, DHS was to create an electronic  entry and exit system to screen and monitor the stay of foreign  nationals who enter and leave the United States and implement the  system at (1) air and sea ports of entry by December 31, 2003,  (2) the 50 highest-volume land ports of entry by December 31, 2004,  and (3) the remaining ports of entry by December 31, 2005. It was  also to provide the means to collect arrival/departure data from  biometrically enabled and machine-readable travel documents at all  ports of entry.
To the program office’s credit, it has largely met its obligations  relative to an entry capability. For example, on January 5, 2004, it  deployed and began operating most aspects of its planned entry  capability at 115 airports and 14 seaports, and added the remaining  aspects in February 2005. During 2004, it also deployed and began  operating this entry capability in the secondary inspection areas of  the 50 highest volume land ports of entry. As of December 19, 2005,  it had deployed and begun operating its entry capability at all but 1  of the remaining 104 land ports of entry. The program has also been  working to define feasible and cost-effective exit solutions,  including technology feasibility testing at 3 land ports of entry and  operational performance evaluations at 11 air and sea ports of entry.
Moreover, the development and deployment of this entry capability  has occurred during a period of considerable organizational change,  starting with the creation of DHS from 23 separate agencies in early  2003, followed by the establishment of a US-VISIT program office  shortly thereafter—which was only about 5 months before it had to  meet its first legislative milestone. Compounding these program  challenges was the fact that the systems that were to be used in  building and deploying an entry capability were managed and  operated by a number of the separate agencies that had been  merged to form the new department, each of which was governed  by different policies, procedures, and standards.
As a result of the program’s efforts to deploy and operate an entry  capability, DHS reports that it has been able to apprehend and  prevent the entry of hundreds of criminal aliens: as of March 2005,  DHS reported that more than 450 people with records of criminal or  immigration violations have been prevented from entering. For  example, its biometric screening prevented the reentry of a  convicted felon, previously deported, who was attempting to enter  under an alias; standard biographic record checks using only names  and birth dates would have likely cleared the individual.
Another potential consequence, although difficult to demonstrate, is  the deterrent effect of having an operational entry capability.  Although deterrence is not an expressly stated goal of the program,  officials have cited it as a potential byproduct of having a publicized  capability at the border to screen entry on the basis of identity  verification and matching against watch lists of known and  suspected terrorists. Accordingly, the deterrent potential of the  knowledge that unwanted entry may be thwarted and the  perpetrators caught is arguably a layer of security that should not be  overlooked.
section_title: DHS Has Yet to Demonstrate that US-VISIT as Defined Is the Right Solution
paragraphs: A prerequisite for prudent investment in programs is having  reasonable assurance that a proposed course of action is the right  thing to do, meaning that it properly fits within the larger context of  an agency’s strategic plans and related operational and technology  environments, and that the program will produce benefits in excess  of costs over its useful life. We have made recommendations to DHS  aimed at ensuring that this is in fact the case for US-VISIT, and the  department has taken steps intended to address our  recommendations. These steps, however, have yet to produce  sufficient analytical information to demonstrate that US-VISIT as  defined is the right solution. Without this knowledge, investment in  the program cannot be fully justified.
subsections: section_title: Operational and Technological Context Are Still Being Defined
paragraphs: Agency programs need to properly fit within a common strategic  context or frame of reference governing key aspects of program  operations—e.g., what functions are to be performed by whom,  when and where they are to be performed, what information is to be  used to perform them, and what rules and standards will govern the  application of technology to support them. Without a clear  operational context for US-VISIT, the risk is increased that the  program will not interoperate with related programs and thus not  cost-effectively meet mission needs.
In September 2003 we reported that DHS had not defined key  aspects of the larger homeland security environment in which US- VISIT would need to operate. For example, certain policy and  standards decisions had not been made, such as whether official  travel documents would be required for all persons who enter and  exit the country—including U.S. and Canadian citizens—and how  many fingerprints would be collected. Nonetheless, program  officials were making assumptions and decisions at that time that, if  they turned out to be inconsistent with subsequent policy or  standards decisions, would require US-VISIT rework. To minimize  the impact of these changes, we recommended that DHS clarify the  context in which US-VISIT is to operate.
About 28 months later, defining this operational context remains a  work in progress. For example, the program’s relationships and  dependencies with other closely allied initiatives and programs are  still unclear. According to the US-VISIT Chief Strategist, an  immigration and border management strategic plan was drafted in  March 2005 that shows how US-VISIT is aligned with DHS’s  organizational mission and that defines an overall vision for  immigration and border management. According to this official, the  vision provides for an immigration and border management  enterprise that unifies multiple internal departmental and other  external stakeholders with common objectives, strategies,  processes, and infrastructures. As of December 2005, however, we  were told that this strategic plan has not been approved.
In addition, since the plan was drafted, DHS has reported that other  relevant initiatives have been undertaken. For example:  ●  The DHS Security and Prosperity Partnership of North America is  to, among other things, establish a common approach to securing  the countries of North America—the United States, Canada, and  Mexico—by, for example, implementing a border facilitation  strategy to build capacity and improve the legitimate flow of people  and cargo at our shared borders.   ●  The DHS Secure Border Initiative is to implement a comprehensive  approach to securing our borders and combating illegal  immigration.
According to the Chief Strategist, portions of the strategic plan are  being incorporated into these initiatives, but these initiatives and  their relationships with US-VISIT are still being defined.
Similarly, the mission and operational environment of US-VISIT are  related to those of another major DHS program—the Automated  Commercial Environment (ACE), which is a new trade processing  system that is planned to support the movement of legitimate  imports and exports and to strengthen border security. In addition,  both US-VISIT and ACE could potentially use common IT  infrastructures and services. As we reported in February 2005, the  program office recognized these similarities, but managing the  relationship between the two programs had not been a priority  matter. Accordingly, we recommended that DHS give priority to  understanding the relationships and dependencies between the US- VISIT and ACE programs.
Since our recommendation, the US-VISIT and ACE managers have  formed an integrated project team to, among other things, ensure  that the two programs are programmatically and technically aligned.  Program officials stated that the team has met three times since  April 2005 and plans to meet on a quarterly basis going forward. The  team has discussed potential areas of focus and agreed to three  areas: RF technology, program control, and data governance.  However, it does not have an approved charter, and it has not  developed explicit plans or milestone dates for identifying the  dependencies and relationships between the two programs.
It is important that DHS define the operational context for US-VISIT,  as well as its relationships and dependencies with closely allied  initiatives and such programs as ACE. The more time it takes to  settle these issues, the more likely that extensive and expensive  rework will be needed at a later date.
section_title: Return on Investment Has Yet to be Determined
paragraphs: Prudent investment also requires that an agency have reasonable  assurance that a proposed program will produce mission value  commensurate with expected costs and risks. Thus far, DHS has yet  to develop an adequate basis for knowing that this is the case for its  early US-VISIT increments. Without this knowledge, it cannot  adequately ensure that these increments are justified.
Assessments of costs and benefits are extremely important, because  the decision to invest in any capability should be based on reliable  analyses of return on investment. According to OMB guidance,  individual increments of major systems are to be individually  supported by analyses of benefits, cost, and risk. In addition, OMB  guidance on the analysis needed to justify investments states that  such analysis should meet certain criteria to be considered  reasonable. These criteria include, among other things, comparing  alternatives on the basis of net present value and conducting  uncertainty analyses of costs and benefits. (DHS has also issued  guidance on such economic analyses, which is consistent with that  of OMB.) Without reliable analyses, an organization cannot be  reasonably assured that a proposed investment is a prudent and  justified use of resources.
In September 2003, we reported that the program had not assessed  the costs and benefits of Increment 1. Accordingly, we  recommended that DHS perform such assessments for future  increments. In February 2005, we reported that although the  program office had developed a cost-benefit analysis for Increment  2B (which provides the capability for electronic collection of  traveler information at land ports of entry), it had again not  justified the investment, because its treatment of both benefits and  costs was unclear and insufficient. Further, we reported that the  cost estimates on which the cost-benefit analysis was based were of  questionable reliability, because effective cost-estimating practices  were not followed. Accordingly, we recommended that DHS follow  certain specified practices for estimating the costs of future  increments.
Since our February 2005 report, the program has developed a cost- benefit analysis for Increment 1B (which is to provide exit  capabilities at air and sea ports of entry). The latest version of this  analysis, dated June 23, 2005, identifies potential costs and benefits  for three exit solutions at air and sea ports of entry and provides a  general rationale for the viability of the three alternatives  described. This latest analysis meets some but not all the OMB  criteria for economic analyses. For example, it explains why the  investment was needed, and it shows that at least two alternatives  to the status quo were considered. However, it does not include, for  example, a complete uncertainty analysis for the three exit  alternatives evaluated. That is, it does not include a sensitivity  analysis for the three alternatives, which is a major part of an  uncertainly analysis. (A sensitivity analysis is a quantitative  assessment of the effect that a change in a given assumption—such  as unit labor cost—will have on net present value.) A complete  analysis of uncertainty is important because it provides decision  makers with a perspective on the potential variability of the cost and  benefit estimates should the facts, circumstances, and assumptions  change.
In addition, the quality of a cost-benefit analysis is dependent on the  quality of the cost assessments on which it is based. However, the  cost estimate associated with the June 2005 cost-benefit analysis for  the three exit solutions (Increment 1B) did not meet key criteria for  reliable cost estimating. For example, it did not include a detailed  work breakdown structure. A work breakdown structure serves to  organize and define the work to be performed, so that associated  costs can be identified and estimated. Thus, it provides a reliable  basis for ensuring that the estimates include all relevant costs.
Program officials stated that they recognize the importance of  developing reliable cost estimates and have initiated actions to more  reliably estimate the costs of future increments. For example, the  program has chartered a cost analysis process action team, which is  to develop, document, and implement a cost analysis policy,  process, and plan for the program. Program officials also stated that  they have hired additional contracting staff with cost-estimating  experience.
Strengthening the program’s cost-estimating capability is extremely  important. The absence of reliable cost estimates impedes, among  other things, both the development of reliable economic justification  for program decisions and the effective measurement of  performance.
section_title: Analysis of Program Impacts and Options Is Being Performed
paragraphs: Program decisions and planning depend on adequate analyses and  assessments of program impacts and options. The department has  begun to develop such analyses, but some of these, such as its  analyses of the operational impact of Increment 2B and of the  options for its exit capability, do not yet provide an adequate basis  for investment and deployment decisions.
We reported in May 2004 that the program had not assessed its  workforce and facility needs for Increment 2B (which provides the  capability for electronic collection of traveler information at land  ports of entry). Because of this, we questioned the validity of the  program’s assumptions and plans concerning workforce and  facilities, since the program lacked a basis for determining whether  its assumptions were correct and thus whether its plans were  adequate. Accordingly, we recommended that DHS assess the full  impact of Increment 2B on workforce levels and facilities at land  ports of entry, including performing appropriate modeling exercises.
Seven months later, the program office evaluated Increment 2B  operational performance, with the stated purpose of determining the  effectiveness of Increment 2B performance at the 50 busiest land  ports of entry. For this evaluation, the program office established a  baseline for comparing the average times to issue and process  entry/exit forms at 3 of these 50 ports of entry. The program office  then conducted two evaluations of the processing times at the three  ports, first after Increment 2B was deployed as a pilot, and next 3  months later, after it was deployed to all 50 ports of entry. The  evaluation results showed that the average processing times  decreased for all three sites. Program officials concluded that these  results supported their workforce and facilities planning  assumptions that no additional staff was required to support  deployment of Increment 2B and that minimal modifications were  required at the facilities.
However, the scope of the evaluations is not sufficient to satisfy the  evaluations’ stated purpose or our recommendation for assessing  the full impact of 2B. For example, the selection of the three sites,  according to program officials, was based on a number of factors,  including whether the sites already had sufficient staff to support  the pilot. Selecting sites based on this factor could affect the results,  and it presupposes that not all ports of entry have the staff needed  to support 2B. In addition, evaluation conditions were not always  held constant: specifically, fewer workstations were used to process  travelers in establishing the baseline processing times at two of the  ports of entry than were used during the pilot evaluations.
Moreover, CBP officials from a land port of entry that was not an  evaluation site (San Ysidro) told us that US-VISIT deployment has  not reduced but actually lengthened processing times. (San Ysidro  processes the highest volume of travelers of all land ports of entry.)  Although these officials did not provide specific data to support  their statement, their perception nevertheless raises questions about  the potential impact of Increment 2B on the 47 sites that were not  evaluated.
Similarly, in February 2005, we reported that US-VISIT had not  adequately planned for evaluating the alternatives for Increment 1B  (which provides exit capabilities at air and sea ports of entry)  because the scope and timeline of its exit pilot evaluation were  compressed. Accordingly, we recommended that DHS reassess  plans for deploying an exit capability to ensure that the scope of the  exit pilot provides for adequate evaluation of alternative solutions.
Over the last 11 months, the program office has taken actions to  expand the scope and time frames of the pilot. For example, it  increased the number of ports of entry in the pilot from 5 to 11, and  it also extended the time frame by about 7 months. Further,  according to program officials, they were able to achieve the target  sample sizes necessary to have a 95 percent confidence level in their  results.
Nevertheless, questions remain about whether the exit alternatives  have been adequately evaluated to permit selection of the best exit  solution for national deployment. For example, one of the criteria  against which the alternatives were evaluated was the rate of  traveler compliance with US-VISIT exit policies (that is, foreign  travelers providing information as they exit the United States).  However, across the three alternatives, the average compliance with  these policies was only 24 percent, which raises questions as to their  effectiveness. The evaluation report cites several reasons for the  low compliance rate, including that compliance during the pilot was  voluntary. The report further concludes that national deployment of  the exit solution will not meet the desired compliance rate unless  the exit process incorporates an enforcement mechanism, such as  not allowing persons to reenter the United States if they do not  comply with the exit process. Although an enforcement mechanism  might indeed improve compliance, program officials stated that no  formal evaluation has been conducted of enforcement mechanisms  or their possible effect on compliance. The program director agreed  that additional evaluation is needed to assess the impact of  implementing potential enforcement mechanisms and plans to do  such evaluation.
section_title: DHS Is Still Establishing Needed Program Management Capabilities
paragraphs: Establishing effective program management capabilities is  important to ensure that an organization is going about delivering a  program in the right way. Accordingly, we have made  recommendations to establish specific people and process  management capabilities. While DHS is making progress in  implementing many of our recommendations in this area, this  progress has often been slow.
One area in which DHS has made good progress is in implementing  our recommendations to establish the human capital capabilities  necessary to manage US-VISIT. In September 2003, we reported that  the US-VISIT program had not fully staffed or adequately funded its  program office or defined specific roles and responsibilities for  program office staff. Our prior experience with major acquisitions  like US-VISIT shows that to be successful, they need, among other  things, to have adequate resources, and program staff need to  understand what they are to do, how they relate to each other, and  how they fit in their organization. In addition, prior research and  evaluations of organizations show that effective human capital  management can help agencies establish and maintain the  workforce they need to accomplish their missions. Accordingly, we  recommended that DHS ensure that human capital and financial  resources are provided to establish a fully functional and effective  program office, and that the department define program office  positions, roles, and responsibilities. We also recommended that  DHS develop and implement a human capital strategy for the  program office that provides for staffing positions with individuals  who have the appropriate knowledge, skills, and abilities.
DHS has implemented our recommendation that it define program  office positions, roles, and responsibilities, and it has partially  completed our two other people-related recommendations. It has  filled most of its planned government positions and is on the way to  filling the rest, and it has filled all of its planned contractor  positions. However, the program completed a workforce analysis in  February 2005 and requested additional positions based on the  results. Securing these necessary resources will be a continuing  challenge.
In addition, as we reported in February 2005, the program office,  working with the Office of Personnel Management, developed a  draft human capital plan that employed widely accepted human  capital planning tools and principles (for example, it included an  action plan that identified activities, their proposed completion  dates, and the office responsible for the action). In addition, the  program office had completed some of the activities in the plan.  Since then, the program office has finalized the human capital plan,  completed more activities, and formulated plans to complete others  (for example, according to the program office, it has completed an  analysis of its workforce to determine diversity trends, retirement  and attrition rates, and mission-critical and leadership competency  gaps, and it has plans to complete an analysis of workforce data to  maintain strategic focus on preserving the skills, knowledge, and  leadership abilities required for the US-VISIT program’s success).
Program officials also said that the reason they have not completed  several activities in the plan is that these activities are related to the  department’s new human capital initiative, MAXHR. Because this  initiative is to include the development of departmentwide  competencies, program officials told us that it could potentially  affect ongoing program activities related to competencies. As a  result, these officials said that they are coordinating these activities  closely with the department as it develops and implements this new  initiative, which is currently being reviewed by the DHS Deputy  Secretary.
DHS’s progress in implementing our human capital  recommendations should help ensure that it has sufficient staff with  the right skills and abilities to successfully execute the program.  Having such staff has been and will be particularly important in light  of the program’s more limited progress to date in establishing  program management process capabilities.
DHS’s progress in establishing effective processes governing how  program managers and staff are to perform their respective roles  and responsibilities has generally been slow. In our experience,  weak process management controls typically result in programs  falling short of expectations. From September 2003, we have made  numerous recommendations aimed at enabling the program to  strengthen its process controls in such areas as acquisition  management, test management, risk management, configuration  management, capacity management, security, privacy, and  independent verification and validation (IV&V). DHS has not yet  completed the implementation of any of our recommendations in  these areas, with one exception. It has ensured that the program  office’s IV&V contractor was independent of the products and  processes that it was verifying and validating, as we recommended.  In July 2005, the program office issued a new contract for IV&V  services after following steps to ensure the contractor’s  independence (for example, IV&V contract bidders were to be  independent of the development and integration contractors and are  prohibited from soliciting, proposing, or being awarded work for the  program other than IV&V services). If effectively implemented, these  steps should adequately ensure that verification and validation  activities are performed in an objective manner, and thus should  provide valuable assistance to program managers and decision  makers.
In the other management areas, DHS has partially completed or has  only begun to address our recommendations, and more remains to  be done. For example, DHS has not completed the development and  implementation of key acquisition controls. We reported in  September 2003 that the program office had not defined key  acquisition management controls to support the acquisition of US- VISIT, increasing the risk that the program would not satisfy system  requirements or meet benefit expectations on time and within  budget. Accordingly, we recommended that DHS develop and  implement a plan for satisfying key acquisition management  controls in accordance with best practices.
The program office has recently taken steps to lay the foundation  for establishing key acquisition management controls. For example,  it has developed a process improvement plan to define and  implement these controls that includes a governance structure for  overseeing improvement activities. In addition, the program office  has recently completed a self-assessment of its acquisition process  maturity, and it plans to use the assessment results to establish a  baseline of its acquisition process maturity as a benchmark for  improvement. According to program officials, the assessment  included key process areas that are generally consistent with the  process areas cited in our recommendation. The program has  ranked these process areas and plans to focus on those with highest  priority. (Some of these high-priority process areas are also areas in  which we have made recommendations, such as configuration  management and risk management.)
The improvement plan is currently being updated to reflect the  results of the baseline assessment and to include a work breakdown  structure, process prioritization, and resource estimates. According  to a program official, the goal is to conduct a formal appraisal to  assess the capability level of some or all of the high-priority process  areas by October 2006.
These recent steps provide a foundation for progress, but fully and  effectively implementing key acquisition management controls takes  considerable time, and DHS is still in the early stages of the process.  Therefore, it is important that these improvement efforts stay on  track. Until these controls are effectively implemented, US-VISIT  will be at risk of not delivering promised capabilities on time and  within budget.
Another management area of high importance to a complex program  like US-VISIT is test management. The purpose of system testing is  to identify and correct system defects before the system is deployed.  To be effective, testing activities should be planned and  implemented in a structured and disciplined fashion. Among other  things, this includes developing effective test plans to guide the  testing activities and ensuring that test plans are developed and  approved before test execution.
In this area also, DHS’s progress responding to our recommendation  has been limited. We reported in May 2004, and again in February  2005, that system testing was not based on well-defined test plans,  and thus the quality of testing being performed was at risk. Because  DHS test plans were not sufficiently well-defined to be effective, we  recommended that before testing begins, DHS develop and approve  test plans that meet the criteria that relevant systems development  guidance prescribes for effective test plans: namely, that they  (1) specify the test environment; (2) describe each test to be  performed, including test controls, inputs, and expected outputs;  (3) define the test procedures to be followed in conducting the tests;  and (4) provide traceability between the test cases and the  requirements to be verified by the testing.
About 20 months later, the quality of the system test plans, and thus  system testing, is still a challenge. To the program’s credit, the test  plans for the Proof of Concept for Increment 2C, dated June 28, 2005  (which introduces RF technology to automatically record the entry  and exit of covered individuals), satisfied part of our  recommendation. Specifically, the test plan for this increment was  approved on June 30, 2005, before testing began (according to  program officials, it began on July 5, 2005). Further, the test plan  described, for example, the scope, complexity, and completeness of  the test environment; it described the tests to be performed,  including a high-level description of controls, inputs, and outputs;  and it identified the test procedures to be performed.
However, the test plan did not adequately trace between test cases  and the requirements to be verified by testing. For example, about  70 percent of the requirements that we analyzed did not have  specific references to test cases. Further, we identified traceability  inconsistencies, such as one requirement that was mapped to over  50 test cases, even though none of the 50 cases referenced the  requirement.
Time and resource constraints were identified as the reasons that  test plans have not been complete. Specifically, program officials  stated that milestones do not permit existing testing/quality  personnel the time required to adequately review testing  documents. According to these officials, even when the start of  testing activities is delayed because, for example, requirements  definition or product development takes longer than anticipated,  testing milestones are not extended.
Without complete test plans, the program does not have adequate  assurance that the system is being fully tested, and thus  unnecessarily assumes the risk of system defects not being detected  and addressed before the system is deployed. This means that the  system may not perform as intended when deployed, and defects  will not be addressed until late in the systems development cycle,  when they are more difficult and time-consuming to fix. This has in  fact happened already: postdeployment system interface problems  surfaced for Increment 1, and manual work-arounds had to be  implemented after the system was deployed.
Until process management weaknesses such as these are addressed,  the program will continue to be overly dependent on the exceptional  performance of individuals to produce results. Such dependence  increases the risk of the US-VISIT program falling short of  expectations.
section_title: DHS Has Yet to Fully Establish Program Accountability Mechanisms
paragraphs: To better ensure that US-VISIT and DHS meet expectations, we  made recommendations related to measuring and disclosing  progress against program commitments. Thus far, such performance  and accountability mechanisms have yet to be fully established.  Measurements of the operational performance of the system are  necessary to ensure that the system adequately supports mission  operations, and measurements of program progress and outcomes  are important for demonstrating that the program is on track and is  producing results. Without such measurements, program  performance and accountability can suffer.
As we reported in September 2003, the operational performance of  initial system increments was largely dependent on the performance  of existing systems that were to be interfaced to create these  increments. For example, we said that the performance of an  increment would be constrained by the availability and downtime of  the existing systems, some of which had known problems in these  areas. Accordingly, we recommended that DHS define performance  standards for each increment that are measurable and that reflect  the limitations imposed by this reliance on existing systems. In  February 2005, we reported that several technical performance  standards for increments 1 and 2B had been defined, but that it was  not clear that these standards reflected the limitations imposed by  the reliance on existing systems. Since then, the program office has  defined certain other technical performance standards for the next  increment (Increment 2C, Phase 1), including standards for  availability. Consistent with what we reported, the functional  requirements document states that these performance standards are  largely dependent upon those of the current systems, and for system  availability, it sets an aggregated availability standard for Increment  2C components. However, the document does not contain sufficient  information for a determination of whether these performance  standards actually reflect the limitations imposed by reliance on  existing systems. Unless the program defines performance  standards that do this, it will be unable to identify and effectively  address performance shortfalls.
Similarly, as we observed in June 2003, to permit meaningful  program oversight, it is important that expenditure plans describe  how well DHS is progressing against the commitments made in prior  expenditure plans. The expenditure plan for fiscal year 2005 (the  fourth US-VISIT expenditure plan) does not describe progress  against commitments made in the previous plans. For example,  according to the fiscal year 2004 plan, US-VISIT was to analyze, field  test, and begin deploying alternative approaches for capturing  biometrics during the exit process. However, according to the fiscal  year 2005 plan, US-VISIT was to expand its exit pilot sites during the  summer and fall of 2004, and it would not deploy the exit solution  until fiscal year 2005. The plan does not explain the reason for this  change from its previous commitment nor its potential impact. Nor  does it describe the status of the exit pilot testing or deployment,  such as whether the program has met its target schedule or whether  the schedule has slipped.
Additionally, the fiscal year 2004 plan stated that $45 million in fiscal  year 2004 was to be used for exit activities. However, in the fiscal  year 2005 plan, the figure for exit activities was $73 million in fiscal  year 2004 funds. The plan does not highlight this difference or  address the reason for the change in amounts. Also, although the  fiscal year 2005 expenditure plan includes benefits stated in the  fiscal year 2004 plan, it does not describe progress in addressing  those benefits, even though in the earlier plan, US-VISIT stated that  it was developing metrics for measuring the projected benefits,  including baselines by which progress could be assessed. The fiscal  year 2005 plan again states that performance measures are under  development.
Figure 1 provides our analysis of the commitments made in the  fiscal year 2003 and 2004 plans, compared with progress reported  and planned in February 2005.
The deployment of an exit capability, an important aspect of the  program that was to result from the exit pilots shown in the figure,  further illustrates missed commitments that need to be reflected in  the next expenditure plan. In the fiscal year 2005 expenditure plan,  the program committed to deploying an exit capability to air and sea  ports of entry by September 30, 2005. Although US-VISIT has  completed its evaluation of exit solutions at 11 pilot sites (9 airports  and 2 seaports), no decision has yet been made on when an exit  capability will be deployed. According to program officials,  deployment to further sites would take at least 6 months from the  time of the decision. This means that the program office will not  meet its commitment.
Another accountability mechanism that we recommended in May  2004 is for the program to develop a plan, including explicit tasks  and milestones, for implementing all our open recommendations,  and report on progress, including reasons for delays, both to  department leadership (the DHS Secretary and Under Secretary) in  periodic reports and to the Congress in all future expenditure plans.  The department has taken action to address this recommendation,  but the initial report does not disclose enough information for a  complete assessment of progress. The program office did assign  responsibility to specific individuals for preparing the  implementation plan, and it developed a report identifying the  person responsible for each recommendation and summarizing  progress. This report was provided for the first time to the DHS  Deputy Secretary on October 3, 2005, and the program office plans  to forward subsequent reports every 6 months. However, some of  the report’s progress descriptions are inconsistent with our  assessment. For example, the report states that the impact of  Increment 2B on workforce levels and facilities at land ports of  entry has been fully assessed. However, as mentioned earlier,  evaluation conditions were not always held constant—that is, fewer  workstations were used to process travelers in establishing the  baseline processing times at two of the ports of entry than were  used during the pilot evaluations.
In addition, the report does not specifically describe progress  against most of our recommendations. For example, we  recommended that the program reassess plans for deploying an exit  capability to ensure that the scope of the exit pilot provides for  adequate evaluation of alternative solutions. With regard to the exit  evaluation, the report states that the program office has completed  exit testing and has forwarded the exit evaluation report to the  Deputy Secretary for a decision. However, it does not state whether  the program office had expanded the scope or time frames of the  pilot.
In closing, I would emphasize that the program has met many of the  demanding requirements in law for deployment of an entry-exit  system, owing, in large part, to the hard work and dedication of the  program office and its contractors, as well as the close oversight  and direction of the House and Senate Appropriations Committees.  Nevertheless, core capabilities, such as exit, have yet to be  established and implemented, and fundamental questions about the  program’s fit within the larger homeland security context and its  return on investment remain unanswered. Moreover, the program is  overdue in establishing the means to effectively manage the delivery  of future capabilities. The longer the program proceeds without  these, the greater the risk that the program will not meet its  commitments. Measuring and disclosing the extent to which these  commitments are being met are also essential to holding the  department accountable, and thus are an integral aspect of effective  program management. Our recommendations provide a  comprehensive framework for addressing each of these important  areas and thus ensuring that the program as defined is the right  solution, that delivery of this solution is being managed in the right  way, and that accountability for both is in place. We look forward to  continuing to work constructively with the program to better ensure  the program’s success.
Mr. Chairman, this concludes my statement. I would be happy to  answer any questions that you or members of the committee may  have at this time.
section_title: Contact and Acknowledgement
paragraphs: If you should have any questions about this testimony, please  contact Randolph C. Hite at (202) 512-3439 or hiter@gao.gov. Other  major contributors to this testimony included Tonia Brown, Barbara  Collier, Deborah Davis, James Houtz, Scott Pettis, and Dan Wexler.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.