US-VISIT is a governmentwide program intended to enhance the security of U.S. citizens and visitors, facilitate legitimate travel and trade, ensure the integrity of the U.S. immigration system, and protect the privacy of our visitors. Its scope includes the pre-entry, entry, status, and exit of hundreds of millions of foreign national travelers who enter and leave the United States at over 300 air, sea, and land POEs, and the provision of new analytical capabilities across the overall process. To achieve its goals, US-VISIT uses biometric information (digital fingerscans and photographs) to verify identity. In many cases, the US-VISIT process begins overseas at U.S. consular offices, which collect biometric information from applicants for visas and check this information against a database of known criminals and suspected terrorists. When a visitor arrives at a POE, the biometric information is used to verify that the visitor is the person who was issued the visa. In addition, at certain sites, visitors are required to confirm their departure by undergoing US-VISIT exit procedures—that is, having their visas or passports scanned and undergoing fingerscanning. The exit confirmation is added to the visitor’s travel records to demonstrate compliance with the terms of admission to the United States. (App. III provides a detailed description of the pre-entry, entry, status, exit, and analysis processes.) collecting, maintaining, and sharing information on certain foreign nationals who enter and exit the United States; identifying foreign nationals who (1) have overstayed or violated the terms of their admission; (2) may be eligible to receive, extend, or adjust their immigration status; or (3) should be apprehended or detained by law enforcement officials; detecting fraudulent travel documents, verifying traveler identity, and determining traveler admissibility through the use of biometrics; and facilitating information sharing and coordination within the immigration and border management community. In July 2003, DHS established a program office with responsibility for managing the acquisition, deployment, operation, and sustainment of the US-VISIT system and its associated supporting people (e.g., Customs and Border Protection (CBP) officers), processes (e.g., entry/exit policies and procedures), and facilities (e.g., inspection booths and lanes), in coordination with its stakeholders (CBP and the Department of State). As of October 2005, about $1.4 billion has been appropriated for the program, and, according to program officials, about $962 million has been obligated. DHS plans to deliver US-VISIT capability in four increments, with Increments 1 through 3 being interim, or temporary, solutions that fulfill legislative mandates to deploy an entry/exit system, and Increment 4 being the implementation of a long-term vision that is to incorporate improved business processes, new technology, and information sharing to create an integrated border management system for the future. In Increments 1 through 3, the program is building interfaces among existing (“legacy”) systems; enhancing the capabilities of these systems; and deploying these capabilities to air, sea, and land POEs. These increments are to be largely acquired and implemented through existing system contracts and task orders. In May 2004, DHS awarded an indefinite-delivery/indefinite-quantity prime contract to Accenture and its partners. According to the contract, the prime contractor will help support the integration and consolidation of processes, functionality, and data, and it will develop a strategy to build on the technology and capabilities already available to produce the strategic solution, while also assisting the program office in leveraging existing systems and contractors in deploying the interim solutions. Increment 1 concentrates on establishing capabilities at air and sea POEs. It is divided into two parts—1 and 1B. Increment 1 (air and sea entry) includes the electronic capture and matching of biographic and biometric information (two digital index fingerscans and a digital photograph) for selected foreign nationals, including those from visa waiver countries. Increment 1 was deployed on January 5, 2004, for individuals requiring a nonimmigrant visa to enter the United States, through the modification of pre-existing systems. These modifications accommodated the collection and maintenance of additional data fields and established interfaces required to share data among DHS systems in support of entry processing at 115 airports and 14 seaports. Increment 1B (air and sea exit) involves the testing of exit devices to collect biometric exit data for select foreign nationals at 11 airports and seaports. Three exit alternatives were pilot tested: Kiosk—A self-service device (which includes a touch-screen interface, document scanner, finger scanner, digital camera, and receipt printer) that captures a digital photograph and fingerprint and prints out an encoded receipt. Mobile device—A hand-held device that is operated by a workstation attendant; it includes a document scanner, finger scanner, digital camera, and receipt printer and is used to capture a digital photograph and fingerprint. Validator—A hand-held device that is used to capture a digital photograph and fingerprint, which are then matched to the photograph and fingerprint captured via the kiosk and encoded in the receipt. Increment 2 focuses primarily on extending US-VISIT to land POEs. It is divided into three parts—2A, 2B, and 2C. Increment 2A (air, sea, and land) includes the capability to biometrically compare and authenticate valid machine-readable visas and other travel and entry documents issued by State and DHS to foreign nationals at all POEs. Increment 2A was deployed on October 23, 2005, according to program officials. It also includes the deployment by October 26, 2006, of technology to read biometrically enabled passports from visa waiver countries. Increment 2B (land entry) redesigns the Increment 1 entry solution and expands it to the 50 busiest land POEs. The process for issuing Form I-94 was redesigned to enable the electronic capture of biographic, biometric (unless the traveler is exempt), and related travel documentation for arriving travelers. This increment was deployed to the busiest 50 U.S. land border POEs as of December 29, 2004. Before Increment 2B, all information on the Form I-94s was handwritten. The redesigned systems electronically capture the biographic data included in the travel document. In some cases, the form is completed by CBP officers, who enter the data electronically and then print the form. Increment 2C is to provide the capability to automatically, passively, and remotely record the entry and exit of covered individuals using radio frequency (RF) technology tags at primary inspection and exit lanes. An RF tag that includes a unique ID number is to be embedded in each Form I-94, thus associating a unique number with a record in the US-VISIT system for the person holding that Form I-94. In August 2005, the program office deployed the technology to five border crossings (three POEs) to verify the feasibility of using passive RF technology to record traveler entries and exits via a unique ID number embedded in the CBP Form I-94. The results of this demonstration are to be reported in February 2006. Increment 3 extended Increment 2B (land entry) capabilities to 104 land POEs; this increment was essentially completed as of December 19, 2005. Increment 4 is the strategic US-VISIT program capability, which program officials stated will likely consist of a further series of incremental releases or mission capability enhancements that will support business outcomes. The program reports that it has worked with its prime contractor and partners to develop this overall vision for the immigration and border management enterprise. Increments 1 through 3 include the interfacing and integration of existing systems and, with Increment 2C, the creation of a new system, the Automated Identification Management System (AIDMS). The three main existing systems are as follows: The Arrival Departure Information System (ADIS) stores noncitizen traveler arrival and departure data received from air and arrival data captured by CBP officers at air and sea POEs, Form I-94 issuance data captured by CBP officers at Increment 2B departure information captured at US-VISIT biometric departure pilot (air and sea) locations, pedestrian arrival information and pedestrian and vehicle departure information captured at Increment 2C POE locations, and status update information provided by the Student and Exchange Visitor Information System (SEVIS) and the Computer Linked Application Information Management System (CLAIMS 3) (described below). ADIS provides record matching, query, and reporting functions. The passenger processing component of the Treasury Enforcement Communications System (TECS) includes two systems: Advance Passenger Information System (APIS), a system that captures arrival and departure manifest information provided by air and sea carriers, and the Interagency Border Inspection System, a system that maintains lookout data and interfaces with other agencies’ databases. CBP officers use these data as part of the admission process. The results of the admission decision are recorded in TECS and ADIS. The Automated Biometric Identification System (IDENT) collects and stores biometric data on foreign visitors. US-VISIT also exchanges biographic information with other DHS systems, including SEVIS and CLAIMS 3. These two systems contain information on foreign students and foreign nationals who request benefits, such as a change of status or extension of stay. Some of the systems previously described, such as IDENT and the new AIDMS, are managed by the program office, while some systems are managed by other organizational entities within DHS. For example, TECS is managed by CBP, SEVIS is managed by Immigration and Customs Enforcement, CLAIMS 3 is under United States Citizenship and Immigration Services, and ADIS is jointly managed by CBP and US-VISIT. US-VISIT also interfaces with other, non-DHS systems for relevant purposes, including watch list updates and checks to determine whether a visa applicant has previously applied for a visa or currently has a valid U.S. visa. In particular, US-VISIT receives biographic and biometric information from State’s Consular Consolidated Database as part of the visa application process, and returns fingerscan information and watch list changes. The US-VISIT program office structure includes nine component offices. Each of the program offices includes a director and subordinate organizational units, as established by the director. The responsibilities for each office are stated below. Figure 1 shows the program office structure, including its nine offices. The roles and responsibilities for each of the nine offices include the following: Chief Strategist is responsible for developing and maintaining the strategic vision, strategic documentation, transition plan, and business case. Budget and Financial Management is responsible for establishing the program’s costs estimates; analysis; and expenditure management policies, processes, and procedures that are required to implement and support the program by ensuring proper fiscal planning and execution of the budget and expenditures. Mission Operations Management is responsible for developing business and operational requirements based on strategic direction provided by the Office of the Chief Strategist. Outreach Management is responsible for enhancing awareness of US-VISIT requirements among foreign nationals, key domestic audiences, and internal stakeholders by coordinating outreach to media, third parties, key influencers, Members of Congress, and the traveling public. Information Technology Management is responsible for developing technical requirements based on strategic direction provided by the Office of the Chief Strategist and business requirements developed by the Office of Mission Operations Management. Implementation Management is responsible for developing accurate, measurable schedules and cost estimates for the delivery of mission systems and capabilities. Acquisition and Program Management is responsible for establishing and managing the execution of program acquisition and management policies, plans, processes, and procedures. Administration and Training is responsible for developing and administering a human capital plan that includes recruiting, hiring, training, and retaining a diverse workforce with the competencies necessary to accomplish the mission. Facilities and Engineering Management is responsible for establishing facilities and environmental policies, procedures, processes, and guidance required to implement and support the program office. In response to legislative mandate, we have issued four reports on DHS’s annual expenditure plans for US-VISIT. Our reports have, among other things, assessed whether the plans satisfied the legislative conditions and provided observations on the plans and DHS’s program management. As a result of our assessments, we made 24 recommendations aimed at improving both plans and program management, all of which DHS has agreed to implement. Of these 24 recommendations, 18 address risks stemming from program management. The current status of DHS’s implementation of our 18 recommendations on program risks is mixed, but progress in critical areas has been slow. For example, over 2 years have passed, and the program office has yet to develop a security plan consistent with federal guidance or to economically justify its investment in system increments. According to the Program Director, the pace of progress is attributable to competing demands on time and resources. DHS agreed to implement all 18 recommendations. Of these 18, DHS has completely implemented 2, has partially implemented 11, and is in the process of implementing another 5. Of the 11 that are partially implemented, 7 are about 2 years old, and 4 are about 10 to 19 months old. Of the 5 that are in progress, 3 are about 10 months old. These 18 recommendations are aimed at strengthening the program’s management effectiveness. The longer that the program takes to implement the recommendations, the greater the risk that the program will not meet its goals on time and within budget. Figure 2 provides an overview of the extent to which each recommendation has been implemented. The figure is followed by sections providing details on each recommendation and our assessment of its implementation status. In June 2003, we reported that the Immigration and Naturalization Service had not developed a security plan and performed a privacy impact assessment for the entry exit program (as US-VISIT was then known). A security plan and privacy impact assessment are important to understanding system requirements and ensuring that the proper safeguards are in place to protect system data and resources. System acquisition best practices and federal guidance advocate understanding and defining security and privacy requirements both early and continuously in a system’s life cycle, and effectively planning for their satisfaction. Accordingly, we recommended that DHS do the following: Develop and begin implementing a system security plan, and perform a privacy impact assessment and use the results of the analysis in near-term and subsequent system acquisition decision making. Since we made the system security plan recommendation about 2½ years ago, its implementation has been slow. For example, we reported in September 2003 and again in May 2004 that the program office had not developed a security plan. In February 2005, we reported that the program office had developed a security plan, dated September 2004, and that this plan was generally consistent with federal guidance. That is, the plan provided an overview of system security requirements, described the controls in place or planned for meeting those requirements, referred to the applicable documents that prescribe the roles and responsibilities for managing the US-VISIT component systems, and addressed security awareness and training. However, the program office had not conducted a risk assessment or included in the plan when an assessment would be completed. According to guidance from the Office of Management and Budget (OMB), the security plan should describe the methodology that is used to identify system threats and vulnerabilities and to assess risks, and it should include the date the risk assessment was completed. According to program officials, they completed a programwide risk assessment in December 2005, but have yet to provide a copy of the assessment to us. Therefore, we cannot confirm that the assessment has been done, and done properly. The absence of a risk assessment and a security plan that reflects this assessment is a significant program weakness. Risk assessments are critical to establishing effective security controls because they provide the basis for establishing appropriate policies and selecting cost-effective controls to implement these policies. Without such an assessment, US-VISIT does not have adequate assurance that it knows the risks associated with the program and thus whether it has implemented effective controls to address them. Notwithstanding these limitations in the security plan, the program office has begun to implement aspects of its September 2004 security plan. For example, the Information Systems Security Manager told us that a security awareness program is established and key personnel have attended security training. Since June 2003, US-VISIT has also developed and periodically updated a privacy impact assessment. An initial impact assessment was issued in January 2004, and a revised assessment was issued in September 2004. A more recent assessment, dated July 2005, reflects changes related to Increments 1B and 2C. Each of these assessments is generally consistent with OMB guidance. That is, each of the assessments addressed most OMB requirements, including the impact that the system will have on individual privacy, the privacy consequences of collecting the information, and alternatives considered to collect and handle information. The most recent impact assessment, for example, states that three alternatives were considered for Increment 1B—the kiosk, the mobile device, and the validator (a combination of the two)—and discusses proposals to mitigate the privacy risks of all three, such as by limiting the duration of data retention on the exit devices and using encryption. However, OMB guidance also requires that privacy impact assessments developed for systems under development address privacy in relevant system documentation, including statements of need, functional requirements documents, and cost-benefit analyses. As we reported about previous privacy impact assessments, privacy is only partially addressed in system documentation. For example, the Increment 1B cost-benefit analysis assesses the privacy risk associated with each exit alternative, and the Increment 2C business requirements state that all solutions are to be compliant with privacy laws and regulations and adhere to US-VISIT privacy policy. However, we did not find privacy in the Increment 1B business requirements or the Increment 2C functional requirements. Program officials, including the US-VISIT Privacy Officer, acknowledged that privacy is not included in the system documentation, but stated that privacy is considered in the development of the documentation and that the privacy office reviews key system documentation at relevant times during the system development life cycle. Nevertheless, we did not find evidence of privacy being addressed in the system documentation, and program officials acknowledged that it was not included. Until the program performs a risk assessment and fully implements a security plan that reflects this assessment, it cannot adequately ensure that US-VISIT is cost-effectively safeguarding assets and data. Moreover, without reflecting privacy in system documentation, it cannot adequately ensure that privacy needs are being fully addressed. We reported in September 2003 that the program office had not defined key acquisition management controls to support the acquisition of US-VISIT, and therefore its efforts to acquire, deploy, operate, and maintain system capabilities were at risk of not satisfying system requirements and of not meeting benefit expectations on time and within budget. The Capability Maturity Model–Integration® (CMMI) developed by Carnegie Mellon University’s Software Engineering Institute (SEI) explicitly defines process management controls that are recognized hallmarks of successful organizations and that, if implemented effectively, can greatly increase the chances of successfully acquiring software-intensive systems. SEI’s CMMI model uses capability levels to assess process maturity. Because establishing the basic acquisition process capabilities, according to SEI, can take on average about 19 months, we recognized the importance of starting early to build effective acquisition management capabilities by recommending that DHS do the following: Develop and implement a plan for satisfying key acquisition management controls, including acquisition planning, solicitation, requirements management, program management, contract tracking and oversight, evaluation, and transition to support, and implement the controls in accordance with SEI guidance. The program office has recently taken foundational steps to establish key acquisition management controls. For example, it has developed a process improvement plan, dated May 16, 2005 (about 20 months after our recommendation), to define and implement these controls. As part of its improvement program, the program office is implementing a governance structure for overseeing improvement activities, consisting of three groups: a Management Steering Group, an Enterprise Process Group, and Process Action Teams. Specific roles for each of these groups are described below. The Management Steering Group is to provide policy and procedural guidance and to oversee the entire improvement program. The steering group is chaired by the US-VISIT Director, with the Deputy Director and the functional office directors serving as core members. The Enterprise Process Group is to provide planning, management, and operational guidance in day-to-day process improvement activities. The group is chaired by the process improvement leader and is composed of individuals from each functional office. Process Action Teams are to provide specific process documentation and to provide implementation support and training services. These teams are to be active as long as a particular process improvement initiative is under way. To date, the program office has chartered five process teams—configuration management, cost analysis, process development, communications, and policy. In addition, the program office has recently completed a self-assessment of its acquisition process maturity, and it plans to use the assessment results to establish a baseline of its acquisition process maturity for improvement. According to program officials, the assessment included 13 key process areas that are generally consistent with the process areas cited in our recommendation. The program has ranked these 13 process areas according to their priority, and, for initial implementation, it plans to focus on the following 6: Configuration management. Establishing and maintaining the integrity of the products throughout their life cycle. Process and product quality assurance. Taking actions to provide management with objective insight into the quality of products and processes. Project monitoring and control. Tracking the project’s progress so that appropriate corrective actions can be taken when performance deviates significantly from plans. Project planning. Establishing and maintaining plans for work activities. Requirements management. Managing the requirements and ensuring a common understanding of the requirements between the customer and the product developers. Risk management. Identifying potential problems before they occur so that they can be mitigated to minimize any adverse impact. The improvement plan is currently being updated to reflect the results of the baseline assessment and to include a detailed work breakdown structure, process prioritization, and resource estimates. According to the Director, Acquisition and Program Management Office (APMO), the goal is to conduct a formal SEI appraisal to assess the capability level of some or all of the six processes by October 2006. Notwithstanding the recent steps to begin addressing our recommendation, much work remains to fully implement key acquisition management controls. Moreover, effectively implementing these controls takes considerable time. Therefore, it is important that these improvement efforts stay on track. Until these processes are effectively implemented, US-VISIT will be at risk of not delivering promised capabilities on time and within budget. In September 2003, we reported that the program had not assessed the costs and benefits of Increment 1, which is extremely important because the decision to invest in any capability should be based on reliable analyses of return on investment. Further, according to OMB guidance, individual increments of major systems are to be individually supported by analyses of benefits, cost, and risk. Without reliable analyses, an organization cannot adequately know that a proposed investment is a prudent and justified use of limited resources. Accordingly, we recommended that DHS do the following: Determine whether proposed US-VISIT increments will produce mission value commensurate with cost and risks and disclose to the Congress planned actions. As we reported in September 2003 and again in February 2005, the program office did not justify its planned investment in Increments 1 and 2B, respectively, based on expected return on investment. Since then, the program has developed a cost-benefit analysis for Increment 1B. OMB has issued guidance concerning the analysis needed to justify investments. According to this guidance, such analyses should meet certain criteria to be considered reasonable. These criteria include, among other things, comparing alternatives on the basis of net present value and conducting uncertainty analyses of costs and benefits. DHS has also issued guidance on such economic analyses that is consistent with that of OMB. The latest cost-benefit analysis for Increment 1B (dated June 23, 2005) identifies potential costs and benefits for three exit solutions at air and sea POEs and provides a general rationale for the viability of the three alternatives described. This latest analysis meets four of eight OMB economic analysis criteria. However, it does not, for example, include a complete uncertainty analysis (i.e., both a sensitivity analysis and a Monte Carlo simulation) for the three exit alternatives evaluated. That is, the cost-benefit analysis does include a Monte Carlo simulation, but it does not include a sensitivity analysis for the three alternatives. An analysis of uncertainty is important because it provides decision makers with a perspective on the potential variability of the cost and benefit estimates should the facts, circumstances, and assumptions change. Table 1 summarizes our analysis of the extent to which US-VISIT’s June 23, 2005, cost-benefit analysis for Increment 1B satisfies eight OMB criteria. It is important that the program adhere to relevant guidance in developing its incremental cost-benefit analyses. If this is not done, the reliability of the analyses is diminished, and an adequate basis for prudent investment decision making does not exist. Moreover, if the mission value of a proposed investment is not commensurate with costs, it is vital that this information be fully disclosed to DHS and congressional decision makers. The underlying intent of our recommendation is that this information be available to inform such decisions. In September 2003, we reported that key aspects of the larger homeland security environment in which US-VISIT would need to operate had not been defined. For example, we stated that certain policy and standards decisions had not been made (e.g., whether official travel documents will be required for all persons who enter and exit the country, including U.S. and Canadian citizens, and how many fingerprints are to be collected). In the absence of this operational context, program officials were making assumptions and decisions that, if they proved inconsistent with subsequent policy or standards decisions, would require US-VISIT rework. To minimize the impact of these changes, we recommended that DHS do the following: Clarify the operational context in which US-VISIT is to operate. After about 27 months, defining this operational context remains a work in progress. According to the Chief Strategist, an immigration and border management strategic plan was drafted in March 2005 that shows how US-VISIT is aligned with DHS’s organizational mission and defines an overall vision for immigration and border management. This official stated that this vision provides for an immigration and border management enterprise that unifies multiple internal departmental and other external stakeholders with common objectives, strategies, processes, and infrastructures. Since the plan was drafted, DHS has reported that other relevant initiatives have been undertaken, such as the Security and Prosperity Partnership of North America and the Secure Border Initiative. The Security and Prosperity Partnership is to, among other things, establish a common approach to securing the countries of North America— the United States, Canada, and Mexico—by, for example, implementing a border facilitation strategy to build capacity and improve the legitimate flow of people and cargo at our shared borders. The Secure Border Initiative is to implement a comprehensive approach to securing our borders and reducing illegal immigration. According to the Chief Strategist, while portions of the strategic plan are being incorporated into these initiatives, these initiatives and their relationship with US-VISIT are still being defined. We have yet to receive the US-VISIT strategic plan because, according to program officials, it had not yet been approved by DHS management. Until US-VISIT’s operational context is fully defined, DHS is increasing its risk of defining, establishing, and implementing a program that is duplicative of other programs and not interoperable with them. This in turn will require rework to address these areas. While this issue was significant 27 months ago, when we made the recommendation, it is still more significant now. We reported in September 2003 that the program had not fully staffed its program office. Our prior experience with major acquisitions like US-VISIT shows that to be successful, they need, among other things, to have adequate resources. Accordingly, we recommended that DHS do the following: Ensure that human capital and financial resources are provided to establish a fully functional and effective program office. About 2 years later, US-VISIT had filled 102 of its 115 planned government positions and all of its planned 117 contractor positions. For the remaining 13 government positions, 5 positions had been selected (pending completion of security clearances), and recruitment action was in process for filling the remaining 8 vacancies. According to the Office of Administration and Training Manager, funding is available to complete the hiring of all 115 government employees. Notwithstanding this progress, in February 2005, US-VISIT completed a workforce analysis and requested additional positions based on the results. According to program officials, a revised analysis was submitted in the summer of 2005, but the request has not yet been approved. Figure 3 shows the program office organization structure and functions and how many of the 115 positions needed have been filled. Securing necessary resources will be a continuing challenge and an essential ingredient to the program’s ability to acquire, deploy, operate, and maintain system capabilities on time and within budget. We reported in September 2003 that the program had not defined specific roles and responsibilities for its staff. Our prior experience and leading practices show that for major acquisitions like US-VISIT to be successful, program staff need, among other things, to understand what they are to do, how they relate to each other, and how they fit in their organization. Accordingly, we recommended that DHS do the following: Define program office positions, roles, and responsibilities. The program office has developed charters for its nine component offices that include roles and responsibilities for each. For example, the Acquisition and Program Management Office is responsible, among other things, for establishing acquisition and program management policies; coordinating development of configuration management plans and project schedules, including the integrated milestone schedule; and developing policies and procedures for guidance and oversight of systems development and implementation activities. The program has also defined a set of core competencies (knowledge, skills, and abilities) for each position. For example, it has defined critical competencies for program and management analysts that include, among others, flexibility, interpersonal skills, organizational awareness, oral communication, problem solving, and teamwork. These efforts to define position, roles, and responsibilities should help in managing the program effectively. As previously stated, we reported in September 2003 that US-VISIT had not fully staffed its program office or defined roles and responsibilities for its program staff. We observed that prior research and evaluations of organizations showed that effective human capital management can help agencies establish and maintain the workforce they need to accomplish their missions. Accordingly, we recommended that DHS do the following: Develop and implement a human capital strategy for the program office that provides for staffing positions with individuals who have the appropriate knowledge, skills, and abilities. In February 2005, we reported that the program office, in conjunction with the Office of Personnel Management (OPM), developed a draft human capital plan that employed widely accepted human capital planning tools and principles. The draft plan included, for example, an action plan that identified activities, proposed completion dates, and the office (OPM or the program office) responsible for the action. We also reported that the program office had completed some of the activities, such as designating a liaison responsible for ensuring alignment between departmental and program human capital policies. Since then, the program office has finalized the human capital plan and completed more activities. For example, program officials told us that they have analyzed the program office’s workforce to determine diversity trends, retirement and attrition rates, and mission-critical and leadership competency gaps; updated the program’s core competency requirements to ensure alignment between the program’s human capital and business needs; developed an orientation program for new employees; and administered competency assessments to incoming employees. Program officials also told us that they have plans to complete other activities, such as developing a staffing forecast to inform succession planning; analyzing workforce data to maintain strategic focus on preserving the skills, knowledge, and leadership abilities required for the US-VISIT program’s success; and developing organizational leadership competency models for the program’s senior executive, managerial, and supervisory levels. In addition, the officials said that several activities in the plan have not been completed, such as assessing the extent of any current employees’ competency gaps and developing a competency-based listing of training courses. These officials said that the reason these activities have not been completed is that they are related to the department’s new human capital initiative, MAXHR, which is to provide greater flexibility and accountability in the way employees are paid, developed, evaluated, afforded due process, and represented by labor organizations. MAXHR is to include the development of departmentwide competencies. Because of this, the officials told us that it could potentially impact the program’s ongoing competency-related activities. As a result, these officials said that they are coordinating these activities closely with the department as it develops and implements this new initiative, which is currently being reviewed by the DHS Deputy Secretary for approval. Until US-VISIT fully implements a comprehensive human capital strategy, it will continue to risk not having staff with the right skills and abilities to successfully execute the program. We reported in September 2003 that the operational performance of initial system increments was largely dependent on the performance of existing systems that were to be interfaced to create these increments. For example, we said that the performance of an increment will be constrained by the availability and downtime of the existing systems that it includes. Accordingly, we recommended that DHS do the following: Define performance standards for each increment that are measurable and reflect the limitations imposed by relying on existing systems. In February 2005 (17 months later), we reported that several technical performance standards for Increments 1 and 2B had been defined, but that it was not clear that these standards reflected the limitations imposed by the reliance on existing systems. Since then, for the Increment 2C Proof of Concept (Phase 1), the program office has defined certain other performance standards. For example, the functional requirements document for Increment 2C (Phase 1) defines several technical performance standards, including reliability, recoverability, and availability. For each, the document states that the performance standard is largely dependent on those of Increment 2B. More specifically, the document states that Phase 1 system availability is largely dependent upon the individual and collective availability of the current systems. The document also states that the Increment 2C components shall have an aggregated availability greater than or equal to 97.5 percent. However, the document does not contain sufficient information to determine whether these performance standards actually reflect the limitations imposed by reliance on existing systems. To further develop performance standards, the program office has prepared a Performance Engineering Plan, dated March 31, 2005, that links US-VISIT performance engineering activities to its System Development Life Cycle. Further, the plan (1) provides a framework to be used to align its business, application, and infrastructure performance goals and measures; (2) describes an approach to translate business goals into operational measures, and then to quantitative metrics; and (3) identifies system performance measurement areas (effectiveness, efficiency, reliability, and availability). According to program officials, they intend to establish a group to develop action plans for implementing the engineering plan, but did not have a time frame for doing so. Without defining performance standards that reflect the limitations of the existing systems upon which US-VISIT relies, the program lacks the ability to identify and effectively address performance shortfalls. In September 2003, we reported that US-VISIT was a risky undertaking because of several factors inherent to the program, such as its large scope and complexity, as well as because of various program management weaknesses. We concluded that these risks, if not effectively managed, would likely cause program cost, schedule, and performance problems. Risk management is a continuous, forward-looking process that is intended either to prevent such problems from occurring or to minimize their impact if they occur by proactively identifying risks, implementing risk mitigation strategies, and measuring and disclosing progress in doing so. Because of the importance of effectively managing program risks, we recommended that DHS do the following: Develop and implement a risk management plan and ensure that all high risks and their status are reported regularly to the executive body. About 2 years later, the program office has developed and has begun implementing a risk management plan. The plan, which was approved in September 2005, includes, among other things, a process for identifying, analyzing, handling, and monitoring risk. It also defines the governance structure to be used in overseeing and managing the process. The program also maintains a risk database, which includes, among other things, a description of the risk, its priority (e.g., high, medium, or low), and its mitigation strategy. According to program officials, the database is currently available to program management and staff. The program has also begun implementing its risk management plan. For example, it has established a Risk Review Board, Risk Review Council, and Risk Owners to govern its risk activities. The roles and responsibilities are described below. The Risk Review Board directs all risk governance within the program and provides the mechanism to escalate/transfer the consideration of risks to program governing boards and to organizations external to the program. The Risk Review Council oversees and manages program-related risks that are significant, controversial, or cross-project or that may require escalation to the Risk Review Board. Risk Owners analyze, handle, and monitor risks. However, full implementation of the risk management plan has yet to occur. As part of its CMMI process maturity baseline self-assessment (previously discussed), the program office found that the risk management process detailed in its plan was not being consistently applied across the program. In response, according to program officials, they have developed risk management training and began conducting training sessions in November 2005. These officials also stated that the Risk Review Board, where risks are reviewed with program executives, has been meeting monthly since September 2005. With respect to regular risk reports to program executives, the plan includes thresholds for escalating risks within the risk governance structure and to DHS governance entities. For example, risks are to be elevated to the Risk Review Board when the cost of the project exceeds more than 5 percent of the project baseline cost, the schedule slippage exceeds more than 5 percent of the baseline schedule, major areas of scope are affected, or quality reduction requires approval. However, program officials stated that these thresholds are not currently being applied. They further stated that although the plan allows for escalation of risks to officials outside the program office, doing so is at the discretion of the Program Director; in addition, according to these officials, although high risks are not routinely escalated outside the program, selected high risks have been disclosed to the Assistant Secretary for Policy in weekly program status reports. As of December 5, 2005, the Program Director proposed submitting monthly reports of high-priority risks and issues through the Assistant Secretary for Policy to the Deputy Secretary. Until US-VISIT fully implements its risk management plan and process, it cannot be assured that all program risks are being identified and managed in order to effectively mitigate any negative impact on the program’s ability to deliver promised capabilities on time and within budget. We reported in May 2004, and again in February 2005, that system testing was not based on well-defined test plans, and thus the quality of testing being performed was at risk. The purpose of system testing is to identify and correct system defects (i.e., unmet system functional, performance, and interface requirements) and thereby obtain reasonable assurance that the system performs as specified before it is deployed and operationally used. To be effective, testing activities should be planned and implemented in a structured and disciplined fashion. Among other things, this includes developing effective test plans to guide the testing activities and ensuring that test plans are developed and approved before test execution. According to relevant systems development guidance, an effective test plan (1) specifies the test environment; (2) describes each test to be performed, including test controls, inputs, and expected outputs; (3) defines the test procedures to be followed in conducting the tests; and (4) provides traceability between the test cases and the requirements to be verified by the testing. Because these criteria were not being met, we recommended that DHS do the following: Develop and approve test plans before testing begins that (1) specify the test environment; (2) describe each test to be performed, including test controls, inputs, and expected outputs; (3) define the test procedures to be followed in conducting the tests; and (4) provide traceability between test cases and the requirements to be verified by the testing. About 19 months later, the quality of the system test plans, and thus system testing, is still problematic. To the program’s credit, the test plans for the Increment 2C Proof of Concept (Phase 1), dated June 28, 2005, satisfied part of our recommendation. Specifically, the test plan for this increment was approved on June 30, 2005, and, according to program officials, testing began on July 5, 2005. Further, the test plan described, for example, the scope, complexity, and completeness of the test environment, and it described the tests to be performed, including a high-level description of controls, inputs, and outputs, and it identified test procedures to be performed. However, the test plan did not adequately trace between test cases and the requirements to be verified by testing. For example, 300 of the 438 functional requirements, or about 70 percent of the requirements that we analyzed, did not have specific references to test cases. In addition, we identified traceability inconsistencies, including the following: One requirement was mapped to over 50 test cases, but none of the 50 cases referenced the requirement. One requirement was mapped to a group of test cases in the traceability matrix, but several of the test cases to which the requirement was mapped did not reference the requirement, and several test cases referenced the requirement and were not included in the traceability matrix. One requirement was mapped to all but one of the test cases within a particular group of test cases, but that test case did refer to the requirement. Time and resources were identified as the reasons that test plans have not been complete. Specifically, program officials stated that milestones do not permit existing testing/quality personnel the time required to adequately review testing documents. According to these officials, even when the start of testing activities is delayed because, for example, requirements definition or product development takes longer than anticipated, testing milestones are not extended. Without complete test plans, the program does not have adequate assurance that the system is being fully tested, and thus unnecessarily assumes the risk that system defects will not be detected and addressed before the system is deployed. This means that the system may not perform as intended when deployed, and defects will not be addressed until late in the systems development cycle, when they are more difficult and time-consuming to fix. As we previously reported, this has happened: postdeployment system interface problems surfaced for Increment 1, and manual work-arounds had to be implemented after the system was deployed. We reported in May 2004 that the program had not assessed its workforce and facility needs for Increment 2B. Because of this, we questioned the validity of the program’s workforce and facility assumptions used to develop its workforce and facility plans, noting that the program lacked a basis for determining whether its assumptions and thus its plans were adequate. Accordingly, we recommended that DHS do the following: Assess the full impact of Increment 2B on land POE workforce levels and facilities, including performing appropriate modeling exercises. Seven months later, the program office evaluated Increment 2B operational performance. The purpose of the evaluation was to determine the effectiveness of Increment 2B performance at the 50 busiest land POEs. To assist in the evaluation, the program office established a baseline for comparing the average Form I-94 or Form I-94W issuance processing times at 3 of the 50 POEs where processing times were to be evaluated. The program office then conducted two evaluations of the processing times at the 3 POEs following Increment 2B deployment. The first was in December 2004, after Increment 2B was deployed to these sites as a pilot, and the second was in February 2005, after Increment 2B was deployed to all 50 POEs. The evaluation results showed that the average processing times decreased for all 3 sites. Table 2 compares the results of the two evaluations and the baseline. According to program officials, these evaluations supported the workforce and facilities planning assumption that no additional staff were required to support deployment of Increment 2B, and that minimal modifications to interior workspace were required to accommodate biometric capture devices and printers and to install electrical circuits. These officials stated that modifications to existing officer training and interior space were the only changes needed. However, the scope of the evaluation was too limited to satisfy the evaluation’s stated purpose or our recommendation for assessing the full impact of Increment 2B. Specifically, program officials stated that the evaluation focused on the time to process Form I-94s and not on operational effectiveness, including workforce impacts and traveler waiting time. Second, the 3 sites were selected, according to program officials, on the basis of a number of factors, including whether the sites already had sufficient staff to support the pilot. Selecting sites on the basis of this factor could affect the results and presupposes that not all POEs have the staff needed to support Increment 2B. Third, evaluation conditions were not always held constant. For example, fewer workstations were used to process travelers in establishing the baseline processing times at 2 of the POEs—Port Huron (9 versus 14) and Douglas (4 versus 6)—than were used during the pilot evaluations. Moreover, CBP officials from 1 POE, which was not an evaluation site, told us that US-VISIT has actually lengthened processing times. (San Ysidro processes the highest volume of travelers of all land POEs.) While these officials did not provide specific data to support this statement, it nevertheless raises questions about the potential impact of Increment 2B on the 47 sites that were not evaluated. It is important that the impact of Increment 2B on workforce and facilities be fully assessed. Since we made our recommendation, Increment 2B deployment and operational facts and circumstances have materially changed, making the implementation of our recommendation using predeployment baseline data for the other 47 sites impractical. Nevertheless, other alternatives, such as surveying officials at these sites to better understand the increment’s impact on workforce levels and facilities, have yet to be explored. Until they are, the program may not be able to accurately project resource needs or make required modifications to achieve its goals of minimizing US-VISIT’s impact on POE processing times. We reported in May 2004 that US-VISIT had not established effective configuration management practices. Configuration management establishes and maintains the integrity of system components and items (e.g., hardware, software, and documentation). A key ingredient is a change control board to evaluate and approve proposed configuration changes. Accordingly, we concluded that the program did not have adequate assurance that approved system changes were actually made, and that changes made to the component systems (for non–US-VISIT purposes) did not interfere with US-VISIT functionality. Accordingly, we recommended that DHS do the following: Implement effective configuration management practices, including establishing a US-VISIT change control board to manage and oversee system changes. After 19 months, US-VISIT has begun implementing configuration management practices. To its credit, the program recently issued a configuration management policy (September 2005) and prepared a draft configuration management plan (August 2005). The policy contains guiding principles, direction, and expectations for planning and performing configuration management, and includes activities, authorities, and responsibilities. The draft plan describes the configuration management governance structure, including organizational entities and their responsibilities, the processes and procedures to be applied, and how controls are to be applied to products. The governance structure includes the Executive Configuration Control Board and the Configuration Management Impact Review Team. According to its charter, the configuration control board is responsible for determining the status of requested configuration changes and resolving any conflicts related to those changes for US-VISIT–managed systems (i.e., not for US-VISIT component systems managed by other DHS organizations). The Impact Review Team, which reports to the board, is responsible for reviewing requests for system changes and submitting a recommendation to the appropriate change review authority (i.e., either the US-VISIT control board or the control board in the DHS organization that manages the component system). According to program officials, for US-VISIT–managed systems, the review authority is the Executive Configuration Control Board. For other systems, such as TECS (which CBP manages), the US-VISIT review team may submit a recommendation to the appropriate control board (in this case, the CBP Control Board). The APMO director stated that the planned configuration management program is intended to complement rather than replace the configuration management programs for the legacy systems. That is, change requests approved by the US-VISIT Executive Configuration Control Board that require changes to a legacy system will be coordinated with the board having responsibility for that system. This means, however, that changes to component systems (e.g., IDENT, ADIS, and TECS) that are initiated and approved by another DHS organization, and that could affect US-VISIT performance, are not subject to US-VISIT configuration management processes and are not also being examined and approved by the US-VISIT control board. This lack of US-VISIT control was the impetus for our recommendation. Although US-VISIT has recently taken steps to begin addressing our recommendation, the program still does not adequately control changes to the component systems upon which US-VISIT performance depends. Until programwide configuration management practices are implemented, the program does not have an effective means for ensuring that approved system changes are actually made and that changes made to the component systems for non–US-VISIT purposes do not compromise US-VISIT functionality and performance. We reported in May 2004 that the program office’s independent verification and validation (IV&V) contractor was not independent of the products and processes that it was verifying and validating. The purpose of IV&V is to provide management with objective insight into the program’s processes and associated work products. Its use is a recognized best practice for large and complex system development and acquisition projects like US-VISIT. To be effective, the verification and validation function is to be performed by an entity that is independent of the processes and products that are being reviewed. Accordingly, we recommended that DHS do the following: Ensure the independence of the IV&V contractor. In July 2005, the program office issued a new contract for IV&V services. To ensure the contactor’s independence, the program office (1) required that IV&V contract bidders be independent of the development and integration contractors; (2) reviewed each of the bidder’s affiliations with the prime contract; (3) included provisions in the contract that prohibit the contractor from soliciting, proposing, or being awarded work (other than IV&V services) for the program; (4) required all contractor personnel to certify that they do not have any conflicts of interest; and (5) ensured that the contractor’s management plan (Oct. 17, 2005) describes how the contractor will ensure technical, managerial, and financial independence. Such steps, if effectively enforced, should adequately ensure that verification and validation activities are performed in an objective manner and, thus, should provide valuable assistance to program managers and decision makers. We reported in May 2004 that US-VISIT’s overall progress on implementing our recommendations had been slow, and considerable work remained to fully address them. As we also noted, given that most of our recommendations focused on fundamental limitations in US-VISIT’s ability to manage the program, it was important to implement the recommendations quickly and completely. Accordingly, we recommended that DHS do the following: Develop a plan, including explicit tasks and milestones, for implementing all of our open recommendations and periodically report to the DHS Secretary and Under Secretary on progress in implementing this plan; and report this progress, including reasons for delays, in all future expenditure plans. About 19 months after our recommendation, the program assigned responsibility to specific individuals for preparing a plan, including specific actions and milestones, to address each recommendation. In addition, it developed a report that identifies the responsible person for each recommendation and summarizes progress made in implementing each. The program office provided this report for the first time to the DHS Deputy Secretary on October 3, 2005, and plans to forward subsequent reports every 6 months. However, the report’s description of progress on 4 recommendations is inconsistent with our assessment, as discussed below: First, the report states that the program completed a privacy impact assessment that is in full compliance with OMB guidance. As previously discussed, an assessment has been developed, but OMB guidance requires that these assessments for systems under development (such as Increment 2C) address privacy in the system’s documentation. Increment 2C systems documentation does not address privacy and therefore is not fully compliant with OMB guidance. Second, the report states that a human capital strategy has been completed. However, as previously discussed, several of the activities in the human capital plan have yet to be implemented. For example, the program has not developed a staffing forecast to inform succession planning. Third, the report states that the impact of Increment 2B on land POE workforce levels and facilities has been fully assessed. However, as we previously stated, the scope of the evaluations was not sufficient to satisfy our recommendation. For example, program officials stated that the evaluation focused on the time to process Form I-94s and not on operational effectiveness, including workforce impacts and traveler waiting time. Moreover, officials at the largest land POE told us that the effect of Increment 2B was the opposite of that reported in the pilot results. Fourth, the report states that the program has partially completed implementing configuration management practices. However, as previously discussed, the program office has yet to implement practices or establish a configuration control board with authority over all changes affecting US-VISIT functionality and performance, including those made to component systems for non–US-VISIT purposes, which was the intent of our recommendation. In addition, the report does not specifically describe progress against 11 of our other recommendations, so that we could not determine whether the program’s assessment is consistent with ours (described in this report). For example, we recommended that the program reassess plans for deploying an exit capability to ensure that the scope of the exit pilot provides for adequate evaluation of alternative solutions. The report states that the program office has completed exit testing and has forwarded the exit evaluation report to the Deputy Secretary for a decision. However, it does not state whether the program office had expanded the scope or time frames of the pilot. Fully understanding and disclosing progress against our recommendations are essential to building the capability needed to effectively manage the program, and to ensuring that key decision makers have the information needed to make well-informed choices among competing investment options. We reported in February 2005 that US-VISIT had not followed effective practices to develop cost estimates for its system increments, and thus the reliability of its cost estimates was questionable. Such cost-estimating practices are embedded in the 13 criteria in SEI’s checklist for determining the reliability of cost estimates. Of these 13 criteria, we reported in February 2005 that the program’s cost estimate met 2, partially met 6, and did not meet 5. Accordingly, we recommended that DHS do the following: Follow effective practices for estimating the costs of future increments. The latest US-VISIT–related cost estimate is for Increment 1B. This estimate is in the June 2005 cost-benefit analysis for Increment 1B and establishes the costs associated with three exit solutions for air and sea POEs. As was the case for the estimate described in our February 2005 report, this latest estimate also did not meet all 13 criteria, meeting 3 and partially meeting another 5. For example, these estimates did not include a detailed work breakdown structure and omitted important cost elements, such as system testing. A work breakdown structure serves to organize and define the work to be performed, so that associated costs can be identified and estimated. Thus, it provides a reliable basis for ensuring that the estimates include all relevant costs. In addition, the uncertainties associated with the Increment 1B cost estimate were not identified. An uncertainty analysis provides the basis for adjusting these estimates to reflect unknown facts and circumstances that could affect costs and identifies the risk associated with the cost estimate. Table 3 summarizes our analysis of the extent to which US-VISIT’s Increment 1B cost estimates satisfy SEI’s 13 criteria. Program officials stated that they recognize the importance of developing reliable cost estimates and have initiated actions to more reliably estimate the costs of future increments. For example, as part of its process improvement program, the program has chartered a cost-analysis process action team, which is to develop, document, and implement a cost-analysis policy, process, and plan for the program. Program officials also stated that they have hired additional contracting staff with cost-estimating experience. Strengthening the program’s cost-estimating capability is extremely important. The absence of reliable cost estimates, among other things, prevents the development of reliable economic justification for program decisions and impedes effective performance measurement. In February 2005, we reported that US-VISIT had not adequately planned for evaluating the Increment 1B exit alternative because its exit pilot evaluation’s scope and timeline were compressed. Accordingly, we recommended that DHS do the following: Reassess plans for deploying an exit capability to ensure that the scope of the exit pilot provides for adequate evaluation of alternative solutions and better ensures that the exit solution selected is in the best interest of the program. Over the last 10 months, the program office has taken actions to expand the scope and time frames of the pilot. For example, it extended the pilot from 5 to 11 POEs—9 airports and 2 seaports. It also extended the time frame for data collection and evaluation to April 2005, which is about 7 months beyond the date for which all exit pilot evaluation tasks were to be completed. Further, according to program officials, they achieved the target sample sizes necessary to have a 95 percent confidence level. Notwithstanding the expanded scope of the pilot, questions remain about whether the exit alternatives have been evaluated sufficiently to permit selection of the best exit solution for national deployment. For example, each of the three exit alternatives was evaluated against three criteria, including compliance with the US-VISIT exit process (i.e., foreign travelers providing information as they exit the United States). However, across the three alternatives, the average compliance with this process was only 24 percent, which raises questions as to the effectiveness of the three alternatives. The evaluation report cites several reasons for the low compliance rate, including that compliance during the pilot was voluntary. The report further concludes that national deployment of the exit solution will not have the desired compliance rate unless the exit process incorporates an enforcement mechanism, such as not allowing persons to reenter the United States if they do not comply with the exit process. Although an enforcement mechanism might indeed improve compliance, program officials stated that no formal evaluation has been conducted of enforcement mechanisms or their effect on compliance. The program director stated that he agrees that additional evaluation is needed to assess the impact of implementing potential enforcement mechanisms and plans to do so. Until the program office adequately evaluates the exit alternatives and knows whether the alternative to be selected will be effective, the program office will not be in a position to select the exit solution that is in the best interest of the program. This is very important because without an effective exit capability, the benefits and the mission value of US-VISIT are greatly diminished. We reported in February 2005 that the overall capacity of the system was not being effectively managed. At that time, US-VISIT, which comprises several legacy systems, was relying on the capacity management activities of these systems. It was not focused on the capacity requirements and performance of the collective systems that make up US-VISIT. This approach increases the risk that the system may not be properly designed and configured for efficient performance, and that it has insufficient processing and storage capacity for current, future, and unpredictable workload requirements. Accordingly, we recommended that DHS do the following: Develop and implement processes for managing the capacity of the US-VISIT system. According to program officials, they have initiated efforts to develop a capacity management process, including a high-level description of the necessary steps, such as identifying tools needed to implement the process. However, a plan, including specific tasks and milestones for developing and implementing capacity management processes, has not yet been developed. Until the program office develops a programwide capacity management program, it increases the risk that US-VISIT may not be able to adequately support program mission needs. We reported in February 2005 that the program office recognized that US-VISIT and the Automated Commercial Environment (ACE) have related missions and operational environments. In addition, US-VISIT and ACE could potentially develop, deploy, and use common information technology infrastructures and services. We also reported that managing this relationship has not been a priority. Accordingly, we recommended that DHS do the following: Make understanding the relationships and dependencies between the US-VISIT and ACE programs a priority matter, and report periodically to the Under Secretary on progress in doing so. US-VISIT and ACE managers met in February 2004, to identify potential areas for collaboration between the two programs and to clarify how the programs could best support the DHS mission and provide officers with the information and tools they need. According to program officials, they have established a US-VISIT/ACE integrated project team to, among other things, ensure that the two programs are programmatically and technically aligned. The team has discussed potential areas of focus and agreed to three areas: RF technology, program control, and data governance. However, it does not have an approved charter, and it has not developed explicit plans or milestone dates for identifying the dependencies and relationships between the two programs. Program officials stated that the team has met three times and plans to meet on a quarterly basis going forward. It is important that the relationships and dependencies between these two programs be managed effectively. The longer it takes for the programs to understand and exploit their relationships, the more rework will be needed at a later date to do so. Over the last 3 years, we have made recommendations aimed at correcting fundamental limitations in US-VISIT’s program management ability and thereby better ensuring the delivery of mission capability and value on time and commensurate with costs. While progress on the implementation of the recommendations is mixed, progress in critical areas has been slow. As with any program, introducing and institutionalizing the program management and accountability discipline at which our recommendations are aimed require investing time and resources while continuing to meet other program demands. In making such investment choices, it is important to remember that institutionalizing such program discipline in the near term will produce long-term payback in a program’s ability to meet these other demands. Accordingly, the longer that US-VISIT takes to implement our recommendations, the greater the risk that the program will not meet its stated goals and commitments. Our open recommendations are all aimed at strengthening US-VISIT program management and improving DHS’s ability to make informed US-VISIT investment decisions. With the exception of one, these recommendations are still relevant and applicable. Since we made our recommendation, facts and circumstances surrounding Increment 2B deployment and operational status have materially changed, making the collection of Increment 2B predeployment impractical. Nevertheless, the need remains to better understand the impact of US-VISIT entry capabilities on all land POEs. Until this understanding exists, the department will be challenged in its ability to accurately estimate and provide facilities and staff resource needs. To recognize both the need to fully assess the impact of US-VISIT entry capabilities on staffing levels and facilities at land POEs, as well as the current operational status of Increment 2B, we are closing our existing recommendation related to assessing the impact of Increment 2B. We recommend that the DHS Secretary direct the US-VISIT Program Director to explore alternative means of obtaining an understanding of the full impact of US-VISIT at all land POEs, including its impact on workforce levels and facilities; these alternatives should include surveying the sites that were not part of the previous assessment. In its written comments on a draft of this report, signed by the Director, Departmental GAO/OIG Liaison Office, and reprinted in appendix II, DHS stated that it agreed with many areas of the report and that our recommendations had made US-VISIT a stronger program. Further, the department stated that while it disagreed with certain areas of the report, it nevertheless concurred with the need to implement our open recommendations with all due speed and diligence. DHS commented specifically on 11 of the 18 recommendations discussed in the report. The recommendations, the department’s comments, and our responses follow: 1. Recommendation: Develop and begin implementing a system security plan, and perform a privacy impact assessment and use the results of the analysis in near-term and subsequent system acquisition decision making. DHS stated that this recommendation has been fully implemented. In support, it said that it has completed a US-VISIT security plan that is consistent with National Institute of Standards and Technology (NIST) guidance, and that it provided the plan to us in September 2004. It also stated that the security risk assessment aspect of this recommendation was established in February 2005, 20 months after we made the recommendation, and thus the age of the recommendation should be shown as 10 months rather that the 30 months cited in the report. The department also commented that there is no US-VISIT system, but rather a US-VISIT program with capabilities delivered by existing interconnected systems. According to the department, these component systems have been certified and accredited, consistent with NIST guidance, and as part of their certification and accreditation, security plans and risk assessments, as well as risk mitigation strategies, have been developed for each system. The department stated that it provided us with these system-level risk assessments, as well as system-specific action plans and milestones for implementing the mitigation strategies. In addition, the department noted that it completed a programwide risk assessment in December 2005 that specifically addresses information security issues that might not be captured in the system-specific documentation used to certify and accredit each system. In light of its system-specific certification and accreditation efforts, existing system-level risk assessments, and the program-level risk management process (see response 4 for discussion of the risk management process), DHS commented that it is inaccurate to state that US-VISIT officials are not in a position to know program risks, and the recommendation should be closed. While we agree that we received a copy of the US-VISIT security plan, dated September 2004, we do not agree that the plan satisfied all relevant federal guidance and that DHS has fully implemented our recommendation. In particular, it has not provided us with evidence that a programwide risk assessment has been done and that a security plan reflective of such an assessment exists. According to relevant guidance, a security plan should describe, among other things, the methodology that is to be used to identify system threats and vulnerabilities and to assess risks, and it should include the date the risk assessment was completed because the assessment is a necessary driver of the security controls described in the plan. As we reported in February 2005 and state in this report, the US-VISIT security plan did not include this information; further, although DHS stated in its comments that it completed this risk assessment in December 2005, this statement is contradicted by a statement elsewhere in its comments that it is still in the process of doing the assessment. In addition to this contradiction, DHS’s comments did not include any evidence to demonstrate that it has developed a complete risk assessment, such as a copy of the assessment. With regard to the age of the recommendation, we do not agree with DHS’s position that we established a new finding regarding the lack of a programwide risk assessment in our February 2005 report. Rather, as part of our analysis of actions to implement our prior recommendation to develop a security plan, which is to include information about the related security risk assessment, we observed that the plan did not indicate a date for completing a risk assessment in accordance with federal guidelines. Therefore, our position that about 30 months had passed from the time of our initial recommendation (June 2003) is accurate. With regard to the individual system-level risk assessments, we agree that we have received them. However, we do not agree that we have received the action plans and milestones cited in the comments. Regardless, we do not believe that system-level assessments are a sufficient substitute for a programwide assessment. Accordingly, our recommendation focused on the need for an integrated US-VISIT system risk assessment as part of security planning. While the system-level plans and risk assessments are relevant and useful, they neither individually nor collectively address the threats and vulnerabilities imposed as a result of these systems’ integration. By stating in its comments its commitment to having a programwide risk assessment that identifies and proposes mitigations for security risks that arise as a result of the interface and integration of the legacy systems, DHS is agreeing with our position. Moreover, without evidence that the program has completely assessed its risks, we continue to find no basis for how program officials would know the full range and degree of US-VISIT security risks. Our position in this regard has been reinforced by a recent DHS Inspector General report that identified a number of US-VISIT security risks. To further support its position that this recommendation has been fully implemented, DHS also commented that it has completed numerous privacy impact assessments and continues to update them to reflect system changes. In particular, it said that it updated the privacy impact assessment in December 2005 to reflect all increments and that it considers the assessment to be part of US-VISIT system documentation. It further commented that we appear to be unaware of privacy staff activities to review system documents and perform privacy risk assessments throughout the system life cycle. Nevertheless, the department acknowledged that its privacy work was not always noted within US-VISIT system documentation. Accordingly, DHS stated that it plans to appropriately reference all privacy requirements and privacy risk assessments in the program’s system documentation in the future. We agree that US-VISIT has developed and updated its privacy impact assessment and would note that our report states this fact. We do not agree, however, with the comment that we are not aware that the privacy staff review system documents and perform privacy risk assessments. In fact, it is because we were aware of these facts that we were careful to ensure that they were reflected in our report. The point that we are making is that privacy is not addressed in all relevant systems documentation, which DHS acknowledged in its comments. With regard to this point of agreement, we support the department’s stated plans to reference all privacy requirements and any privacy risk assessments in all relevant system documentation in the future. 2. Recommendation: Develop and implement a plan for satisfying key acquisition management controls, including acquisition planning, solicitation, requirements management, program management, contract tracking and oversight, evaluation, and transition to support, and implement the controls in accordance with SEI guidance. DHS commented that the report should reflect that US-VISIT had initially adopted Carnegie Mellon University’s Software Engineering Institute (SEI) Software Acquisition Capability Maturity Model® to guide its software-related process improvement efforts and that, in December 2004, it transitioned to SEI’s Capability Maturity Model–Integration (CMMI®). As a result, it said that the program’s process improvement strategy and plans, process development, and process appraisals are now aligned to the most applicable CMMI process areas. We agree that US-VISIT has transitioned to CMMI. We state in our report that US-VISIT has done so and that the key process areas it is addressing in its process improvement strategy and plan are consistent with those cited in our recommendation. We do not believe that this transition materially affects our recommendation, however, because even though the names of the key processes in these two models may in some cases differ, the processes and respective practices are fundamentally consistent. 3. Recommendation: Clarify the operational context in which US-VISIT is to operate. Consistent with our report, DHS commented that the operational context in which US-VISIT operates is in progress, meaning that it has yet to be fully established. For example, it said that the mission of DHS, and therefore the scope of US-VISIT activities to meet the mission, is continually expanding. Further, it acknowledged that more certainty in the operational context is desirable. In mitigation of the risks associated with not having a more stable operational context, DHS made several statements. For example, it said that the principal role of US-VISIT is to integrate information and immigration and border management systems across DHS and the State Department, and to facilitate agencies working toward a common environment that will eliminate redundancies. It also said that elements of its draft immigration and border management strategic plan are being used in current US-VISIT operations. In addition, the department said that mechanisms to mitigate the risks that we cited have been developed and are being implemented. We support DHS’s acknowledgment of the importance of having a well-defined operational context within which to define and implement US-VISIT and related border security programs. However, we do not believe that DHS’s comments provided any evidence showing that sufficient steps and activities to mitigate the associated risks have been taken or are planned. 4. Recommendation: Determine whether proposed US-VISIT increments will produce mission value commensurate with cost and risks and disclose to the Congress planned actions. DHS commented that its cost-benefit analysis (CBA) for Increment 1B conforms to relevant federal guidance, and noted that our expectations as to the scope and level of detail of analysis that should be included in the CBA document are inconsistent with its understanding of OMB Circular A-94 and DHS’s CBA workbook, which were used to guide the development of the CBA analysis. As an example, the department took exception with our statement that year-by-year benefit estimates were not reported by noting that the net present value was based on an estimate of annual benefits and costs, and that net present value could not be estimated without a year-by-year benefit analysis. The department further commented that a comprehensive uncertainty analysis was conducted because it completed a risk analysis, which is more comprehensive, rigorous, and appropriate than conducting a sensitivity analysis. In this regard, it added that the results of the risk analysis provided an indication of Increment 1B’s worthiness in light of existing uncertainty, rather than information on a specific CBA variable or another. The department further noted that it had provided some of these supporting analyses to us. DHS also stated that any investment that has a 5-year life cycle and is considered interim in nature will face considerable challenge in providing economic benefits commensurate with cost. We do not agree that the CBA fully conforms to relevant federal guidance. As our report states, for example, the analysis does not explicitly state the numerical value of the discount rate used for calculating each alternative’s net present value, and hence does not conform to OMB guidance. In addition, the cost estimates used in the analysis were not complete and reliably derived. In deriving the estimate, for example, the department did not clearly define the project’s life cycle to ensure that key factors were not overlooked and that the full cost of the program was included. (See response 10 below for more information on this point.) Last, while we agree that a year-by-year benefit analysis is a necessary component of a net present value determination, OMB nevertheless requires that the year-by-year benefit estimates be reported in the analysis to promote independent review of the estimates. Also, we do not agree that DHS performed a complete uncertainty analysis. According to OMB and DHS guidance, a complete uncertainty analysis should include both a risk analysis and a sensitivity analysis. However, the latter was not done. Thus, our point is not, as DHS comments suggest, that US-VISIT should have performed a sensitivity analysis instead of a risk analysis, but rather, that both types of analyses are necessary to completely examine investment uncertainty. 5. Recommendation: Develop and implement a risk management plan and ensure that all high risks and their status are reported regularly to the executive body. DHS commented that US-VISIT began the development and implementation of its risk management plan in 2004 immediately after we made our recommendation. It further commented that, as part of a CMMI maturity internal appraisal that it completed in July 2005, it found that the risk management process had not been consistently applied across the program. To address this, the department cited actions that it has taken to fully implement risk management, such as approving the risk management plan in September 2005; defining a risk governance structure; establishing and maintaining a risk database; and developing risk management training and providing this training to program personnel and contractors beginning in November 2005. We support the recent actions that the program cited as having been taken to strengthen risk management. However, the actions cited do not demonstrate that the risk management process is being consistently applied. Until US-VISIT fully implements its risk management plan and process, it cannot be assured that all program risks are being identified and managed in order to effectively mitigate any negative impact on the program’s ability to deliver promised capabilities on time and within budget. 6. Recommendation: Develop and approve test plans before testing begins that (1) specify the test environment; (2) describe each test to be performed, including test controls, inputs, and expected outputs; (3) define the test procedures to be followed in conducting the tests; and (4) provide traceability between test cases and the requirements to be verified by the testing. DHS stated that our report does not accurately reflect the status of the Increment 2C Phase 1 testing. In particular, it said that the issues associated with the traceability of requirements to test cases were minor and that the extent of the discrepancies is far less than what our report presents. It further stated that the discrepancies in our report are based on old traceability documentation and do not reflect revised documentation provided to us on November 9, 2005. We agree that DHS provided us with revised traceability matrixes after we had shared with them our analysis of the test plans and traceability matrixes, dated June 28, 2005, and June 27, 2005, respectively. However, the revised documentation referenced in DHS’s comments was provided in November 2005, about 4 months after testing began. This means that the test plans and traceability matrixes available at the time of testing—which are what we reviewed because they governed the scope and nature of actual testing performed—did not adequately trace between test cases and the requirements to be verified. Specifically, 300 of the 438 Increment 2C requirements, or about 70 percent, did not have specific references to test cases. 7. Recommendation: Implement effective configuration management practices, including establishing a US-VISIT change control board to manage and oversee system changes. DHS commented that a US-VISIT representative attends all configuration control board meetings for all applicable legacy component systems, and that any proposed change request from a legacy component control board that could affect US-VISIT functionality is brought to the attention of the US-VISIT Executive Configuration Control Board for consideration. We do not question these statements. However, we do not believe that they demonstrate that US-VISIT has adequate control over system changes that could affect the program. That is, they do not ensure that changes to the component systems that are initiated and approved by another DHS organization and that could affect US-VISIT performance are subject to US-VISIT configuration management and approval processes. US-VISIT could establish explicit and enforceable control over changes to the legacy systems through such mechanisms as defined and enforced memorandums of understanding among the affected DHS organizations. It was the lack of such control that prompted our recommendation. 8. Recommendation: Assess the full impact of Increment 2B on land POE workforce levels and facilities, including performing appropriate modeling exercises. The department stated that, given the imperative to meet the legislatively mandated time frames, the scope of Increment 2B was limited to only one part of POE operations—incorporating the collection of a biometric into the previously manual Form I-94 issuance process. It also stated that wait times are affected by various factors, including traffic volume, staffing levels, and availability of officers. Therefore, DHS focused the Increment 2B evaluation on just the change to this process. The department further commented that given the events since the evaluation—namely, Increment 2B full operations—it is not practical to collect and model baseline data for the 47 sites that were not part of the initial evaluation. Regarding the 3 pilot sites included in the assessment, the department stated that the sites were selected based on criteria developed from input from US-VISIT, as well as CBP operational constraints. The department further commented that the 3 sites provided a reasonable mix of travelers and they did not have other constraints that directly impacted the collection of performance data specific to Form I-94 issuance. DHS also stated that the I-94 processing times vary by POE, and therefore they are not easily generalized from one port to another. Further, the department commented that the number of workstations and officers available to operate those workstations to process applicants for a Form I-94 do not impact the time it takes to issue a Form I-94. We agree that the scope of the Increment 2B evaluation was limited to the I-94 issuance process, and that it did not address the increment’s impact on the POEs’ ability to meet other performance parameters. Our point is that the limited nature of the evaluation does not satisfy either the intent of our recommendation or DHS’s own stated purpose for the evaluation, which was to determine the effectiveness of Increment 2B performance at the 50 busiest land POEs. We also agree that the I-94 processing times vary by POE and cannot be easily generalized. It is for this reason, among others, that we questioned whether the 3 sites selected for the assessment were sufficiently representative to satisfy both our recommendation and the evaluation’s stated purpose. In addition, while we also agree that collecting pre-Increment 2B baseline data is not practical at this time, the fact remains that the operational impact of Increment 2B on workforce levels and facilities has not been adequately assessed, as evidenced by officials at 1 large POE telling us that processing times have increased and DHS’s recognition that each POE is somewhat different. In light of these new facts and circumstances, we are closing our existing recommendation and making a new recommendation to recognize the need for DHS to explore alternative means to assess the impact of US-VISIT entry capabilities at land POEs. This new recommendation will be shown as an open recommendation, and the original recommendation will be closed. 9. Recommendation: Develop a plan, including explicit tasks and milestones, for implementing all of our open recommendations and periodically report to the DHS Secretary and Under Secretary on progress in implementing this plan; and report this progress, including reasons for delays, in all future expenditure plans. DHS stated that it is untrue that 19 months had elapsed from the time we made this recommendation to the time that it assigned responsibilities to program officials for addressing each of our recommendations. In support, it commented that it issued its first plan to address our recommendations on August 18, 2003, and subsequent reports have been issued periodically that update progress in doing so. We agree that DHS has assigned responsibilities to specific individuals for addressing each recommendation. However, we have yet to be provided any evidence to support its statement that it issued the first report addressing our recommendations on August 18, 2003. Similarly, we have not received evidence showing that it has prepared a plan, including specific actions and milestones, for implementing all of our open recommendations, which is a focus of this recommendation. We would also observe that we made this recommendation in May 2004, and at that time the department stated that it agreed with the recommendation but did not indicate that it had taken any steps to address it, such as commenting that a report was issued on August 18, 2003. 10. Recommendation: Follow effective practices for estimating the costs of future increments. DHS either tacitly or explicitly agreed with our findings relative to its satisfaction of 8 of the 13 cost-estimating criteria presented in table 4 (now table 3) of our draft report. For example, it agreed that it did not clearly define the life cycle to which the cost estimate applies. It also agreed that it did not include a work breakdown structure, noting that it used the available project implementation schedule as a proxy for the activities related to the deployment of the exit alternatives. Regarding our five findings concerning its satisfaction of cost-estimating with which DHS disagreed, the department’s primary area of disagreement was with the intended purpose of the Increment 1B CBA that used the cost estimate, which it said in its comments was to inform decision makers about the relative worthiness of each of the three exit alternatives considered for deployment. Hence, DHS stated that the purpose of the CBA was to analyze only the costs associated with deploying an operational solution, not to analyze the costs and benefits of both developing and deploying alternative solutions. DHS further stated that the CBA thus includes only those costs to be incurred in deploying a selected alternative, and it does not include costs already incurred in developing system alternatives (i.e., sunk costs). It further commented that DHS guidance states that sunk costs are not relevant to the current investment analysis because “only current decisions can affect the future consequences of investment alternatives.” DHS also disagreed that the cost estimate in the CBA should have included nonrecurring development costs, and commented that it did appropriately size the task described in the cost estimates for each alternative exit solution, noting that sizing metrics related to software development were not relevant to deployment of the alternatives because development activities had already occurred and therefore are sunk costs. The department added that those sizing metrics that are relevant to the cost estimate are discussed in the CBA, as are the cost estimating parameters (i.e., those associated with deployment and not those associated with development and testing). In addition, DHS disagreed that DHS’s cost estimate excluded important cost categories, such as system testing, and stated that the estimate addresses labor, facilities, operations and maintenance, information technology, travel, and training costs. Once again, DHS emphasized that since the focus of the CBA was on operational deployment and not system design and development, system testing costs were not included because they were not considered relevant. DHS also reiterated its early point that the uncertainty analysis that it conducted was comprehensive. We agree that actual sunk costs should not be included in a CBA cost estimate. However, we disagree that the cost categories that DHS cited as not relevant are only costs that are associated with predeployment activities. Testing, for example, is an activity that is normally performed before, during, and following deployment, and thus the associated costs would be relevant to the stated purpose of the Increment 1B CBA. However, a testing cost category was missing from the CBA cost estimate, as was a cost category for software maintenance. Regarding DHS’s statement that it conducted a complete uncertainty analysis, we reiterate our previous point that a complete uncertainty analysis should include both a risk analysis and a sensitivity analysis, and the CBA did not include the latter. 11. Recommendation: Reassess plans for deploying an exit capability to ensure that the scope of the exit pilot provides for adequate evaluation of alternative solutions and better ensures that the exit solution selected is in the best interest of the program. Concerning the questions we raised about the adequacy of the exit pilots in light of the 24 percent compliance rate, DHS commented that we failed to consider the compliance rate of the previous exit pilot program, the National Security Entry Exit Registration System (NSEERS), which, according to DHS, had a 75 percent compliance rate. DHS added that NSEERS achieved this compliance rate with a very limited number of exit locations, and therefore, any of the three US-VISIT exit alternatives would have at least a 75 percent compliance rate once national deployment was completed. Further, the department commented that Immigration and Customs Enforcement (ICE) had recently conducted enforcement operations at the Denver International Airport, and that the compliance rate during these operations increased from 30 percent to over 90 percent. It then concluded that the combined results of the exit pilot evaluation, the NSEERS pilot, and the ICE enforcement activities at the Denver International Airport lead it to believe that the US-VISIT exit alternatives have been adequately evaluated. We do not agree with this conclusion because it is based on unsupported assumptions. Specifically, DHS did not provide any evidence to support its claim that that US-VISIT would achieve a comparable compliance rate to the NSEERS program. Moreover, even if DHS could achieve a 75 percent compliance rate for US-VISIT exit,that still means that 25 percent of eligible persons would not be complying with the US-VISIT exit process. Further, DHS did not provide any information about the recent enforcement actions conducted by ICE, nor did it provide any evidence that this is a practical and viable option for the US-VISIT exit solution. While we agree that enforcement actions may indeed increase the exit compliance rate, DHS has not yet assessed the impact of such a solution on the US-VISIT exit process. Further, the US-VISIT program director acknowledged the need to evaluate the impact of implementing potential enforcement actions on US-VISIT exit and planned to do so. We are sending copies of this report to the Chairmen and Ranking Minority Members of the Senate and House Appropriations Committees, as well as to the Chairmen and Ranking Minority Members of other Senate and House committees that have authorization and oversight responsibilities for homeland security. We are also sending copies to the Secretary of Homeland Security, Secretary of State, and the Director of OMB. Copies of this report will also be available at no charge on our Web site at www.gao.gov. Should you or your offices have any questions on matters discussed in this report, please contact me at (202) 512-3439 or at hiter@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV. Our objective was to determine the progress of the Department of Homeland Security (DHS) in implementing 18 of our recommendations pertaining to the U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT) program. To accomplish this objective, we reviewed and analyzed US-VISIT’s most recent status reports on the implementation of our open recommendations and related key documents, augmented as appropriate by interviews with program officials. More specifically, we analyzed relevant systems acquisition documentation, including the program’s process improvement plan, risk management plan, and configuration management plan. We also analyzed the US-VISIT security plan, privacy impact assessment, cost-benefit analysis, cost estimates, test plans, human capital plans, and related evaluations and assessments. In performing our analyses, we compared available documentation and program officials’ statements with relevant federal guidance and associated best practices. A more detailed description of our scope and methodology relative to the cost-benefit analysis, cost estimates, and test plans follows: Our analysis of the cost-benefit analysis focused on Increment 1B because this was the latest cost-benefit analysis and cost estimate prepared. In doing this analysis, we compared the US-VISIT cost-benefit analysis to eight criteria in Office of Management and Budget (OMB) guidance. Our analysis of the cost estimate also focused on Increment 1B for the same reason previously cited. In doing this analysis, we compared the estimate to 13 criteria from the Software Engineering Institute that we have previously reported to be the minimum set of actions needed to develop a reliable cost estimate. We then determined whether the criteria were satisfied, partially satisfied, or not satisfied using the definitions given below. Our analysis of the test plans focused on Increment 2C because it is the most recently tested increment. This analysis included determining the extent to which the test plans for this increment met 4 key criteria that we have previously reported as essential to effective test plans. In doing this analysis, we examined Increment 2C systems documentation, including business and functional requirements and traceability matrixes. We also independently traced 58 business requirements and 438 functional requirements to the test cases in the test plan. Further, we independently traced all test cases to the requirements to determine consistency. In performing our work, we used the following categories and definitions in deciding the extent to which each recommendation had been implemented. Specifically, we considered a recommendation completely implemented when documentation demonstrated that it had partially implemented when documentation indicated that actions were under way to implement it, and in progress when documentation indicated that action had been initiated to implement it. These categories and definitions are consistent with those used in our prior US-VISIT reports. In determining the amount of time it has taken to implement actions on our recommendations, we calculated the time from the date the report was issued through December 2005. We conducted our audit work at the US-VISIT program office in Rosslyn, Virginia, from August 2005 through December 2005, in accordance with generally accepted government auditing standards. US-VISIT involves complex processes governing the stages of a traveler’s visit to the United States (pre-entry, entry, status, and exit) and analysis of hundreds of millions of foreign national travelers at over 300 air, sea, and land ports of entry (POE). A simplified depiction of these processes is shown in figure 4. Pre-entry processing begins with initial petitions for visas, grants of visa status, or the issuance of travel documentation. When a foreign national applies for a visa at a U.S. consulate, biographic and biometric data are collected and shared with border management agencies. The biometric data are transmitted from the Department of State to DHS, where the prints are run against the Automated Biometric Identification System (IDENT) database to verify identity and to run a check against the biometric watch list. The results of the biometric check are transmitted back to State. A “hit” response prevents State’s system from printing a visa for the applicant until the information is reviewed and cleared by a consular officer. Pre-entry also includes transmission by commercial air and sea carriers of crew and passenger manifests to appropriate immigration officers before these carriers arrive in the United States. These manifests are transmitted through the Advanced Passenger Information System (APIS). The APIS lists are run against the biographic lookout system to identify those arrivals for whom biometric data are available. In addition, POEs review the APIS list in order to identify foreign nationals who need to be scrutinized more closely. When a foreign national arrives at a POE’s primary (air and sea) or secondary (land) inspection booth, the inspector, using a document reader, scans the machine-readable travel documents. APIS returns any existing records on the foreign national to the US-VISIT workstation screen, including manifest data matches and biographic lookout hits. When a match is found in the manifest data, the foreign national’s name is highlighted and outlined on the manifest data portion of the screen. Biographic information, such as name and date of birth, is displayed on the bottom half of the computer screen, along with a photograph obtained from State’s Consular Consolidated Database. The inspector at the booth scans the foreign national’s fingerprints (left and right index fingers) and takes a digital photograph. This information is forwarded to the IDENT database, where it is checked against stored fingerprints in the IDENT lookout database. If the foreign national’s fingerprints are already in IDENT, the system performs a match (a comparison of the fingerprint taken during the primary inspection to the one on file) to confirm that the person submitting the fingerprints is the person on file. If no prints are currently in IDENT, the foreign national is enrolled in US-VISIT (i.e., biographic and biometric data are entered into IDENT). During this process, the inspector also questions the foreign national about the purpose of his or her travel and length of stay. The inspector adds the class of admission and duration of stay information into the Treasury Enforcement Communications Systems, and stamps the “admit until” date on the Form I-94. If the foreign national is ultimately determined to be inadmissible, the person is detained, lookouts are posted in the databases, and appropriate actions are taken. The status management process manages the foreign national’s temporary presence in the United States, including the adjudication of benefits applications and investigations into possible violations of immigration regulations. As part of this process, commercial air and sea carriers transmit departure manifests electronically for each departing passenger. These manifests are transmitted through APIS and shared with the Arrival Departure Information System (ADIS). ADIS matches entry and exit manifest data (i.e., each record showing a foreign national entering the United States is matched with a record showing the foreign national exiting the United States). ADIS also receives status information from the Computer Linked Application Information Management System and the Student Exchange Visitor Information System on foreign nationals. The exit process includes the carriers’ submission of electronic manifest data to APIS. This biographic information is transmitted to ADIS, where it is matched against entry information. At the 11 POEs where the exit solution is being implemented, the departure is processed by one of three exit methods. Within each port, one or more of the exit methods may be used. The three methods are as follows: Kiosk: At the kiosk, the traveler, guided by a workstation attendant if needed, scans the machine-readable travel documents, provides electronic fingerprints, and has a digital photograph taken. A receipt is printed to provide documentation of compliance with the exit process and to assist in compliance on the traveler’s next attempted entry to the country. After the receipt prints, the traveler proceeds to his or her departure gate. At the conclusion of the transaction, the collected information is transmitted to IDENT. Mobile device: At the departure gate, and just before the traveler boards the departure craft, either a workstation attendant or law enforcement officer scans the machine-readable travel documents, scans the traveler’s fingerprints (right and left index fingers), and takes a digital photograph. A receipt is printed to provide documentation of compliance with the exit process and to assist in compliance on the traveler’s next attempted entry to the country. The device wirelessly transmits the captured data in real time to IDENT via the Transportation Security Administration’s Data Operations Center. If the device is being operated by a workstation attendant, he or she provides a printed receipt to the traveler, and the traveler then boards the departure craft. If the mobile device is being operated by a law enforcement officer, the captured biographic and biometric information is checked in near real time against watch lists. Any potential match is returned to the device and displayed visually for the officer. If no match is found, the traveler is allowed to board the departure craft. Validator: Using a kiosk, the traveler, guided by a workstation attendant if needed, scans the machine-readable travel documents, provides electronic fingerprints, and has a digital photograph taken. As with the kiosk, a receipt is printed to provide documentation of compliance with the exit process and to assist in compliance on the traveler’s next attempted entry to the country. However, this receipt has biometrics (i.e., the traveler’s fingerprints and photograph) embedded on the receipt. At the conclusion of the transaction, the collected information is transmitted to IDENT. The traveler presents his or her receipt to the attendant or law enforcement officer at the gate or departure area, who scans the receipt using a mobile device. The traveler’s identity is verified against the biometric data embedded on the receipt. Once the traveler’s identity is verified, he or she is allowed to board the departure craft. The captured data are not transmitted in real time back to IDENT. Instead, the data are periodically uploaded through the kiosk to IDENT. An analysis capability is to provide for the continuous screening against watch lists of individuals enrolled in US-VISIT for appropriate reporting and action. As more entry and exit information becomes available, it is to be used for analysis of traffic volume and patterns as well as for risk assessments. The analysis is also to be used to support resource and staffing projections across POEs, strategic planning for integrated border management analysis performed by the intelligence community, and determination of travel use levels and expedited traveler programs. In addition to the contact named above, the following people made key contributions to this report: Deborah Davis, Assistant Director; Hal Brumm; Tonia Brown; Joanna Chan; Barbara Collier; Neil Doherty; Jennifer Echard; James Houtz; Scott Pettis; Karen Richey; and Karl Seifert.

Summary: The Department of Homeland Security (DHS) has established a program--the U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)--to collect, maintain, and share information, including biometric identifiers, on selected foreign nationals entering and exiting the United States. US-VISIT uses these identifiers (digital fingerscans and photographs) to screen persons against watch lists and to verify that a visitor is the person who was issued a visa or other travel document. Visitors are also to confirm their departure by having their visas or passports scanned and undergoing fingerscanning at selected air and sea ports of entry (POE). GAO has made many recommendations to improve the program, all of which DHS has agreed to implement. GAO was asked to report on DHS's progress in responding to 18 of these recommendations. The current status of DHS's implementation of the 18 recommendations is mixed, but progress in critical areas has been slow. DHS has implemented 2 of the recommendations: it defined program staff positions, roles, and responsibilities, and it hired an independent verification and validation contractor. It has also taken steps to implement the other recommendations, partially completing 11 and beginning to implement another 5. In September 2003, GAO reported that the program had not assessed the costs and benefits of Increment 1 (which provides entry capabilities to air and sea POEs) and recommended that the program determine whether proposed increments will produce mission value commensurate with cost. In the latest cost-benefit analysis, dated June 23, 2005, the program identified potential costs and benefits for three alternatives for an air and sea exit solution. However, the analysis does not meet key Office of Management and Budget criteria; for example, it does not include a complete uncertainty analysis, which helps to provide decision makers with perspective on the potential variability of the cost and benefit estimates should circumstances change. GAO reported in May 2004 and February 2005 that system testing was not based on well-defined test plans and recommended that before testing begins, the program develop and approve test plans meeting certain criteria. However, although the latest test plan did cover many required areas (such as the tests to be performed), it did not adequately trace between test cases and the requirements to be verified by testing. Without complete and traceable test plans, the risk is increased that the deployed system will not perform as intended. In May 2004, GAO reported that the program had not assessed its workforce and facility needs for Increment 2B (which extends entry capabilities to the 50 busiest land POEs) and recommended that it do so. Since then, the program evaluated the processing times to issue and process entry/exit forms at 3 of the 50 busiest POEs and concluded that the results showed that no additional staff and only minor facilities modifications were required. However, the scope of the evaluation was limited. Since then, DHS has deployed and implemented Increment 2B capabilities to these 50 POEs, making the collection of predeployment baseline data for these sites impractical. Nonetheless, other alternatives, such as surveying site officials about the increment's impacts, have yet to be explored. Until they are, the program may not be able to accurately project resource needs or make any needed modifications to achieve its goals of minimizing US-VISIT's impact on POE operations, which was the impetus for GAO's recommendation. DHS attributed the pace of progress to competing demands on time and resources. The longer that US-VISIT takes to implement the recommendations, the greater the risk that the program will not meet its stated goals on time and within budget.