DHS satisfied or partially satisfied each of the applicable legislative conditions specified in the appropriations act. In particular, the plan, including related program documentation and program officials’ statements, satisfied or provided for satisfying all key aspects of federal acquisition rules, requirements, guidelines, and systems acquisition management practices. Additionally, the plan partially satisfied the conditions that specified (1) compliance with the capital planning and investment review requirements of the Office of Management and Budget (OMB), (2) compliance with DHS’s enterprise architecture, and (3) the plan’s review and approval by DHS’s Investment Review Board, the Secretary of Homeland Security, and OMB. DHS has completely implemented, has partially implemented, is in the process of implementing, or plans to implement all the remaining recommendations contained in our reports on the fiscal years 2002, 2003, and 2004 expenditure plans. Each recommendation, along with its current status, is summarized below: Develop a system security plan and privacy impact assessment. The department has partially implemented this recommendation. First, the US-VISIT program has developed a security plan that provides an overview of system security requirements, describes the controls in place or planned for meeting those requirements, and refers to the applicable documents that prescribe the roles and responsibilities for managing the US-VISIT component systems. However, a security risk assessment of the program has not been completed, and the plan does not include a date for the assessment’s completion. Second, the US-VISIT program has completed a privacy impact assessment for Increment 2. However, the assessment does not satisfy all aspects of OMB guidance for such an assessment, such as fully addressing privacy issues in relevant system documentation. Develop and implement a plan for satisfying key acquisition management controls, including acquisition planning, solicitation, requirements development and management, project management, contract tracking and oversight, evaluation, and transition to support, and implement the controls in accordance with the Software Engineering Institute’s (SEI) guidance. The department is in the process of implementing this recommendation. The US-VISIT Acquisition and Program Management Office has initiated a process improvement program and drafted a process improvement plan. The office has also developed processes or plans, some of which are approved and some of which are in draft, for all except one of SEI’s Software Acquisition Capability Maturity Model (SA-CMM) Level 2 key process areas. Ensure that future expenditure plans are provided to the department’s House and Senate Appropriations Subcommittees in advance of US- VISIT funds being obligated. With respect to the fiscal year 2005 expenditure plan, DHS implemented this recommendation by providing the plan to the Senate and House Subcommittees on October 19, 2004. Ensure that future expenditure plans fully disclose US-VISIT system capabilities, schedule, cost, and benefits to be delivered. The department has partially implemented this recommendation. The expenditure plan identifies high-level capabilities and high-level schedule estimates. It also identifies the amounts budgeted for each increment for fiscal years 2003 through 2005, but it does not associate this funding with specific capabilities and benefits. Further, while the plan identifies several benefits and associates these benefits with increments, it does not include any information on related metrics or on progress against achieving any of the benefits. Ensure that future expenditure plans fully disclose how the US-VISIT acquisition is being managed. The department is in the process of implementing this recommendation. The fiscal year 2005 plan describes some activities being employed to manage the US-VISIT acquisition, such as the governance structure, program office organizational structure, and staffing levels. However, the department does not describe how other important aspects of the program are being managed, such as testing, system capacity, and system configuration. Ensure that human capital and financial resources are provided to establish a fully functional and effective program office. The department has partially implemented this recommendation. As of October 2004, US-VISIT had filled 59 of its 115 government positions, with plans to fill about half the vacant positions once security clearances have been completed. As of November 2004, the program office had filled 88 of a planned 117 contractor positions. The expenditure plan indicates that DHS has budgeted $83 million to maintain the US-VISIT program management structure and baseline operations. Clarify the operational context in which US-VISIT is to operate. The department is in the process of implementing this recommendation. In September 2003, DHS released version 1.0 of its enterprise architecture. We reviewed version 1.0 and found that it is missing, either partially or completely, all the key elements expected in a well-defined architecture, such as descriptions of business processes, information flows among these processes, and security rules associated with these information flows. Since we reviewed version 1.0 of the architecture, DHS has drafted version 2.0. We have not reviewed version 2.0. Determine whether proposed US-VISIT increments will produce mission value commensurate with cost and risks. The department is in the process of implementing this recommendation. US-VISIT developed a cost-benefit analysis for Increment 2B, but it is unclear whether this increment will produce mission value commensurate with cost and risk. For example, the analysis addresses only government costs and does not address potential nongovernmental costs. Further, the analysis identifies three alternatives and identifies the third alternative as the preferred choice. However, US-VISIT is pursuing an alternative more closely aligned with alternative 2, because alternative 3 was considered too ambitious to meet statutorily required time lines. Define US-VISIT program office positions, roles, and responsibilities. The department has partially implemented this recommendation. US- VISIT has developed descriptions for positions within each office, and working with the Office of Personnel Management (OPM), it has drafted a set of core competencies that define the knowledge, skills, abilities, and other competencies needed for successful employee performance. Develop and implement a human capital strategy for the US-VISIT program office that provides for staffing positions with individuals who have the appropriate knowledge, skills, and abilities. The department has partially implemented this recommendation. The US- VISIT program office, in conjunction with OPM, has drafted a Human Capital Plan. The plan includes an action plan that identifies activities, proposed completion dates, and the organization responsible for completing these activities. The program office has completed some of the activities called for in the plan, including the designation of a liaison responsible for ensuring alignment between DHS and US-VISIT human capital policies. Develop a risk management plan and report all high risks and their status to the executive body on a regular basis. The department has partially implemented this recommendation. The US- VISIT program office has developed a risk management plan and process and has established a governance structure involving three primary groups—the Risk Review Board, Risk Review Council, and Risk Management Team. The Risk Review Board represents the highest level of risk management within the program and is composed of senior level staff, such as the program director and functional area directors. However, US- VISIT has not reported high risks beyond this board. Define performance standards for each US-VISIT program increment that are measurable and reflect the limitations imposed by relying on existing systems. The department is in the process of implementing this recommendation. The US-VISIT program office has defined some technical performance measures—such as availability, timeliness, and output quantity—for Increments 1 and 2B, but it has not defined others, such as reliability, resource utilization, and scalability. Additionally, US-VISIT systems documentation does not contain sufficient information to determine the limitations imposed by US-VISIT’s reliance on existing systems that have less demanding performance requirements, such as the 98.0 percent availability of the Treasury Enforcement Communications Systems. Develop and approve test plans before testing begins. These test plans should (1) specify the test environment; (2) describe each test to be performed, including test controls, inputs, and expected outputs; (3) define the test procedures to be followed in conducting the tests; and (4) provide traceability between test cases and the requirements to be verified by the testing. The department is in the process of implementing this recommendation. According to the US-VISIT Systems Assurance Director, the Increment 2B system acceptance test plan was approved on October 15, 2004. However, no documentation was provided that explicitly indicated the approval of the plan. Further, the test plan did not fully address the test environment, include descriptions of tests to be performed, or provide test procedures to be followed in conducting the tests. The plan also did not provide traceability between test cases and the requirements to be verified by the testing. For example, 15 of the 116 requirements did not have test cases, and 2 requirements were labeled “not testable.” Ensure the independence of the Independent Verification and Validation (IV&V) Contractor. The department is in the process of implementing this recommendation. The US-VISIT Information Technology (IT) Management Office is developing high-level requirements for IV&V, including a strategy and statement of work for acquiring an IV&V contractor. Implement effective configuration management practices, including establishing a US-VISIT change control board to manage and oversee system changes. The department plans to implement this recommendation. The US-VISIT program office has not yet developed or implemented US-VISIT-level configuration management practices or a change control board. The office has developed a draft configuration management plan that describes key configuration management activities that are to be defined and implemented, such as defining and identifying processes and products to be controlled and recording and monitoring changes to the controlled items. The draft plan also proposes a governance structure, including change control boards. Identify and disclose management reserve funding embedded in the fiscal year 2004 expenditure plan to the Appropriations Subcommittees. The department has implemented this recommendation. The US-VISIT program office reported management reserve funding of $33 million for fiscal year 2004 in a briefing to the Subcommittees on Homeland Security, Senate and House Committees on Appropriations. Ensure that all future US-VISIT expenditure plans identify and disclose management reserve funding. With respect to the fiscal year 2005 expenditure plan, DHS implemented this recommendation. The fiscal year 2005 plan specified management reserve funding of $23 million. Assess the full impact of Increment 2B on land ports of entry workforce levels and facilities, including performing appropriate modeling exercises. The department has partially implemented this recommendation. The US- VISIT program office conducted an analysis to help determine the impact of Increment 2B on workforce and travelers. According to program officials, additional staff will not be needed to implement this increment at the land borders. In addition, the US-VISIT program office has conducted space utilization surveys at all of the 166 land ports of entry and has completed survey reports at 16 of the 50 busiest land ports of entry, with the remaining 34 reports planned to have been completed in the fall of 2004. Although the survey reports indicated that most of the ports reviewed were at or near capacity and that facilities had no room for expansion, the program office maintains that Increment 2B will not require expansion of any facilities and will only require minor modifications. Develop a plan, including explicit tasks and milestones for implementing all our open recommendations and periodically report to the DHS Secretary and Under Secretary on progress in implementing this plan; also report this progress, including reasons for delays, in all future US-VISIT expenditure plans. The Department is in the process of implementing this recommendation. The US-VISIT program office has developed a report for tracking the status of our open recommendations. This report is shared with the program office director but is not shared with the Secretary and Under Secretary. Our observations recognize accomplishments to date and address the need for rigorous and disciplined program management practices relating to describing progress against commitments, managing the exit alternatives pilot, managing system capacity, and estimating cost, as well as collaborating with DHS’s Automated Commercial Environment (ACE) program. An overview of specific observations follows: The program office has acquired the services of a prime integration contractor to augment its ability to complete US-VISIT. On May 28, 2004, and on schedule, DHS awarded a contract for integrating existing and new business processes and technologies to a prime contractor and its related partners. The fiscal year 2005 expenditure plan does not describe progress against commitments made in previous plans. Although this is the fourth US-VISIT expenditure plan, it does not describe progress against commitments made in the previous three plans. For example, the fiscal year 2004 plan committed to analyzing, field testing, and initiating deployment of alternative approaches for capturing biometrics during the exit process at air and sea ports of entry. However, while the fiscal year 2005 plan states that US-VISIT was to expand its exit pilot sites during the summer and fall of 2004 and deploy the exit solution during fiscal year 2005, it does not explain the reason for the change or its potential impact. Additionally, the fiscal year 2004 plan stated that $45 million in fiscal year 2004 was to be used for exit activities. However, the fiscal year 2005 plan states that $73 million in fiscal year 2004 funds were to be used for exit activities, but it does not highlight this difference or address the reason for the change in amounts. The exit capability alternatives are faced with a compressed time line, missed milestones, and potentially reduced scope. In January 2004, US- VISIT deployed an initial exit capability as a pilot to two ports of entry, while simultaneously developing other exit alternatives. The May 2004 Exit Pilot Evaluation Plan stated that all exit pilot evaluation tasks were to be completed by September 2004. The plan allotted about 3 months to conduct the evaluation and report the results. However, an October 2004 schedule indicated that all exit pilot evaluation tasks were to be completed between late October 2004 and December 2004, which is about a 2-month evaluation and reporting period. As of early November 2004, exit alternatives were deployed and operating in only 5 of the 15 ports of entry that were scheduled to be operational by November 1, 2004. According to program implementation officials, this was because of delays in DHS granting security clearances to the civilian employees who would operate the equipment at the ports of entry. Additionally, the Evaluation Execution Plan describes the sample size of outbound passengers required to be evaluated at each port. This sample size will produce a specified confidence level in the evaluation results. Because of the reduced evaluation time frame, the program still plans to collect the desired sample size at each port by adding more personnel to the evaluation teams if needed. These changing facts and circumstances surrounding the exit pilot introduce additional risk concerning US-VISIT’s delivery of promised capabilities and benefits on time and within budget. US-VISIT and ACE collaboration is moving slowly. In February 2003, we recognized the relationship between US-VISIT and ACE and recommended steps to promote close collaboration between these two programs. Since then, US-VISIT and ACE managers have met to identify potential areas for collaboration between the two programs and to clarify how the programs can best support the DHS mission and provide officers with the information and tools they need. However, explicit plans have not been developed nor actions taken to understand US- VISIT/ACE dependencies and relationships. Because both programs are making decisions on how to further define, design, develop, and implement these systems, it is important that they exploit their relationships to reduce rework that might be needed to integrate the programs. US-VISIT system capacity is being managed in a compartmentalized manner. Currently, DHS does not have a capacity management program. Instead, the US-VISIT IT Management Office relies on the respective performance management activities of the pre-existing systems, such as those managed by U.S. Customs and Border Protection and U.S. Immigration and Customs Enforcement. Until US-VISIT has developed a comprehensive performance management and capacity planning program, the program will continue to be reactive in its efforts to ensure that US-VISIT system resources are sufficient to meet current workloads, increasing the risk that it may not be able to adequately support mission needs. The cost estimating process used for Increment 2B did not follow some key best practices. The US-VISIT cost estimate did not fully satisfy most of the criteria called for in SEI guidance. For example, costs related to development and integration tasks for US-VISIT component systems are specified, but information about estimated software lines of code is not. Additionally, no one outside the US-VISIT program office reviewed and concurred with the cost estimating categories and methodology. Without reliable cost estimates, the ability to make informed investment decisions and effectively manage progress and performance is reduced. The fiscal year 2005 expenditure plan (with related program office documentation and representations) either partially satisfies or satisfies the legislative conditions imposed by Congress. Further, steps are planned, initiated, under way, or completed to address all of our open recommendations. However, overall progress in addressing the recommendations has been slow, leaving considerable work to be done. Given that most of these open recommendations are aimed at correcting fundamental limitations in DHS’s ability to manage the program in a way that ensures the delivery of (1) mission value commensurate with costs and (2) promised capabilities on time and within budget, it is important that DHS implement the recommendations quickly and completely through effective planning and continuous monitoring and reporting. Until this occurs, the program will be at high risk of not meeting its stated goals on time and within budget. To its credit, the program office now has its prime contractor on board to support both near-term increments and to plan for and deliver the yet-to-be- defined US-VISIT strategic solution. However, it is important to recognize that this accomplishment is a beginning and not an end. The challenge for DHS is now to effectively and efficiently work with the prime contractor in achieving desired mission outcomes. To accomplish this, it is important that DHS move swiftly in building its program management capacity, which is not yet in place, as shown by the status of our open recommendations and our recent observations about (1) economic justification of US-VISIT Increment 2B, (2) completion of the exit pilot evaluation, (3) collaboration with a closely related import/export processing and border security program, (4) system capacity management activities, and (5) cost estimating practices. Moreover, it is important that DHS improve its measurement and disclosure to its Appropriations Subcommittees of its progress against commitments made in prior expenditure plans, so that the Subcommittees’ ability to effectively oversee US-VISIT’s plans and progress is not unnecessarily constrained. Nevertheless, the fact remains that the program continues to invest hundreds of millions of dollars for a mission-critical capability under circumstances that introduce considerable risk that cost-effective mission outcomes will not be realized. At a minimum, it is incumbent upon DHS to fully disclose these risks, along with associated mitigation steps, to executive and congressional leaders so that timely and informed decisions about the program can be made. To better ensure that the US-VISIT program is worthy of investment and is managed effectively, we reiterate our prior recommendations and further recommend that the Secretary of Homeland Security direct the Under Secretary for Border and Transportation Security to ensure that the US- VISIT program director takes the following five actions: Fully and explicitly disclose in all future expenditure plans how well DHS is progressing against the commitments that it made in prior expenditure plans. Reassess its plans for deploying an exit capability to ensure that the scope of the exit pilot provides for adequate evaluation of alternative solutions and better ensures that the exit solution selected is in the best interest of the program. Develop and implement processes for managing the capacity of the US- VISIT system. Follow effective practices for estimating the costs of future increments. Make understanding the relationships and dependencies between the US-VISIT and ACE programs a priority matter, and report periodically to the Under Secretary on progress in doing so. In written comments on a draft of this report, signed by the Acting Director, Departmental GAO/IG Liaison Office (reprinted in app. II), DHS concurred with our findings and recommendations. DHS also stated that it appreciated the guidance that the report provides for future efforts and described actions taken and progress made in implementing the US-VISIT program. We are sending copies of this report to the Chairmen and Ranking Minority Members of other Senate and House committees and subcommittees that have authorization and oversight responsibilities for homeland security. We are also sending copies to the Secretary of Homeland Security, Secretary of State, and the Director of OMB. Copies of this report will also be available at no charge on our Web site at www.gao.gov. Should you or your offices have any questions on matters discussed in this report, please contact me at (202) 512-3439 or at hiter@gao.gov. Another contact and key contributors to this report are listed in appendix III. facilitate legitimate travel and trade, ensure the integrity of the U.S. immigration system, and protect the privacy of our visitors. meets the capital planning and investment control review requirements established by the Office of Management and Budget (OMB), including OMB Circular A-11, part 7;2 complies with DHS’s enterprise architecture; complies with the acquisition rules, requirements, guidelines, and systems acquisition management practices of the federal government; is reviewed and approved by the DHS Investment Review Board, the Secretary of Homeland Security, and OMB; and is reviewed by GAO. Pub. L. 108-334 (Oct. 18, 2004). OMB Circular A-11 establishes policy for planning, budgeting, acquisition, and management of federal capital assets. 1. determine whether the US-VISIT fiscal year 2005 expenditure plan satisfies the 2. determine the status of our US-VISIT open recommendations, and 3. provide any other observations about the expenditure plan and DHS’s management of US-VISIT. We conducted our work at US-VISIT offices in Rosslyn, Virginia, from June 2004 through November 2004, in accordance with generally accepted government auditing standards. Details of our scope and methodology are described in attachment 1 of this briefing. Satisfies or provides for satisfying many, but not all, key aspects of the condition that we reviewed. Satisfies or provides for satisfying every aspect of the condition that we reviewed. Actions are planned to implement the recommendation. Actions are under way to implement the recommendation. Actions have been taken that fully implement the recommendation. The Software Acquisition Capability Maturity Model (SA-CMM®) developed by Carnegie Mellon University’s Software Engineering Institute (SEI) defines acquisition process management controls for planning, managing, and controlling software-intensive system acquisitions. With respect to the fiscal year 2005 expenditure plan. The purpose of independent verification and validation is to provide an independent review of processes and products throughout the acquisition and deployment phase. Results in Brief: Objective 3 Observations The program office has acquired the services of a prime integration contractor to augment its ability to complete US-VISIT. The fiscal year 2005 Expenditure Plan does not describe progress against commitments (e.g., capabilities, schedule, cost, and benefits) made in previous plans. The exit capability alternatives evaluation is faced with a compressed time line, missed milestones, and potentially reduced scope. US-VISIT and Automated Commercial Environment (ACE)3 collaboration is moving slowly. US-VISIT system capacity is being managed in a compartmentalized manner. The cost estimating process used for Increment 2B did not follow some key best practices. ACE is a new trade processing system planned to support the movement of legitimate imports and exports and strengthen border security. Results in Brief: Objective 3 Observations To assist DHS in managing US-VISIT, we are making five recommendations to the Secretary of DHS. In their comments on a draft of this briefing, US-VISIT program officials stated that they generally agreed with our findings, conclusions, and recommendations. collecting, maintaining, and sharing information on certain foreign nationals who enter and exit the United States; identifying foreign nationals who (1) have overstayed or violated the terms of their visit; (2) can receive, extend, or adjust their immigration status; or (3) should be apprehended or detained by law enforcement officials; detecting fraudulent travel documents, verifying traveler identity, and determining traveler admissibility through the use of biometrics; and facilitating information sharing and coordination within the border management community. An indefinite-delivery/indefinite-quantity contract provides for an indefinite quantity, within stated limits, of supplies or services during a fixed period of time. The government schedules deliveries or performance by placing orders with the contractor. Accenture’s partners include, among others, Raytheon Company, the Titan Corporation, and SRA International, Inc. 8 C.F.R. 235.1(d)(1)(iv) and 215.8(a)(2) state that lasses of travelers that are not subject to US-VISIT are foreign nationals admitted on A-1, A-2, C-3 (except for attendants, servants, or personal employees of accredited officials), G-1, G-2, G-3, G-4, NATO-1, NATO-2, NATO-3, NATO-4, NATO-5, or NATO-6 visas; certain Taiwan officials who hold E-1 visas and members of their immediate families who hold E-1 visas, unless the Secretary of State and the Secretary of Homeland Security jointly determine that a class of such aliens should be subject to the rule; children under the age of 14; persons over the age of 79; classes of aliens to whom the Secretary of Homeland Security and the Secretary of State jointly determine it shall not apply; and an individual alien to whom the Secretary of Homeland Security, the Secretary of State, or the Director of Central Intelligence determines shall not be subject to the rule. At that time, the pilot employed a self-serve kiosk to capture biographic information and biometric data (two index fingerprints). The pilots are deployed to Miami Royal Caribbean seaport and the Baltimore/Washington International Airport. Chicago O’Hare International Airport, Denver International Airport, and Dallas/Ft. Worth International Airport. electronic fingerprints and photographs. The hybrid combines the enhanced kiosk, which is used to generate a receipt, with the mobile device, which scans the receipt and the electronic fingerprint of the traveler at the gate to verify exit. As of November 18, 2004, US-VISIT had processed about 13 million foreign nationals, including about 2 million from visa waiver countries. According to US- VISIT, it had positively matched over 1,500 persons against watch list databases. Pub. L. 108-299 (Aug. 9, 2004) extended the deadline from October 26, 2004, to October 26, 2005. Secondary inspection is used for more detailed inspections that may include checking more databases, conducting more intensive interviews, or both. As required by the Immigration and Naturalization Service Data Management Improvement Act of 2000, 8 U.S.C. 1365a(d)(2). The three sites are Laredo, Texas; Port Huron, Michigan; and Douglas, Arizona. Radio frequency (RF) technology relies on proximity cards and card readers. RF devices read the information contained on the card when the card is passed near the device and can also be used to verify the identity of the cardholder. As required by the Immigration and Naturalization Service Data Management Improvement Act of 2000, 8 U.S.C. 1365a(d)(3). maintains lookout (i.e., watch list) data,15 interfaces with other agencies’ databases, and is currently used by inspectors at POEs to verify traveler information and update traveler data. Within TECS are several databases, including the following: Advance Passenger Information System (APIS) includes arrival and departure manifest information provided by air and sea carriers. Crossing History includes information about individuals’ crossing histories. Lookout data sources include DHS’s Customs and Border Protection and Immigration and Customs Enforcement; the Federal Bureau of Investigation (FBI); legacy DHS systems; the U.S. Secret Service; the U.S. Coast Guard; the Internal Revenue Service; the Drug Enforcement Agency; the Bureau of Alcohol, Tobacco, & Firearms; the U.S. Marshals Service; the U.S. Office of Foreign Asset Control; the National Guard; the Treasury Inspector General; the U.S. Department of Agriculture; the Department of Defense Inspector General; the Royal Canadian Mounted Police; the U.S. State Department; Interpol; the Food and Drug Administration; the Financial Crimes Enforcement Network; the Bureau of Engraving and Printing; and the Department of Justice Office of Special Investigations. interest. Secondary includes the results of prior secondary inspections performed on an individual, including if the person was admitted or denied entry. US-VISIT Biometric Information File (BIF) includes keys or links to other databases in TECS, IDENT, and ADIS and includes such information as fingerprint identification numbers, name, and date of birth. Addresses includes addresses of individuals. I-94/Non-Immigrant Information System (NIIS) includes information from I- 94 forms. US-Visa (Datashare) includes Department of State records of visa applications, such as photographs, biographic information, and fingerprint identification number. arrival and departure data and that provides query and reporting functions. Automated Biometric Identification System (IDENT) is a system that collects and stores biometric data about foreign visitors.16 Student Exchange Visitor Information System (SEVIS) is a system that contains information on foreign students. Computer Linked Application Information Management System (CLAIMS 3) is a system that contains information on foreign nationals who request benefits, such as change of status or extension of stay. Consular Consolidated Database (CCD) is a system that includes information on whether a visa applicant has previously applied for a visa or currently has a valid U.S. visa. Includes data such as FBI information on all known and suspected terrorists, selected wanted persons (foreign-born, unknown place of birth, previously arrested by DHS), and previous criminal histories for high-risk countries; DHS Immigration and Customs Enforcement information on deported felons and sexual registrants; and DHS information on previous criminal histories and previous IDENT enrollments. Information from the FBI includes fingerprints from the Integrated Automated Fingerprint Identification System. The I-94 form is used to track the arrival and departure of nonimmigrants. It is divided into two parts. The first part is an arrival portion, which includes, for example, the nonimmigrant’s name, date of birth, and passport number. The second part is a departure portion, which includes the name, date of birth, and country of citizenship. At the kiosk, the traveler, guided by a WSA if needed, scans the machine- readable travel documents, provides electronic fingerprints, and has a digital photograph taken. A receipt is printed to provide documentation of compliance with the exit process and to assist in compliance on the traveler’s next attempted entry to the country. After the receipt prints, the traveler proceeds to his/her departure gate. At the conclusion of the transaction, the collected information is transmitted to IDENT. If the device is being operated by a WSA, the WSA provides a printed receipt to the traveler, and the traveler then boards the departure craft. If the mobile device is being operated by a law enforcement officer, the captured biographic and biometric information is checked in near real time against watch lists. Any potential match is returned to the device and displayed visually for the officer. If no match is found, the traveler boards the departure craft. scans the machine-readable travel documents, provides electronic fingerprints, and has a digital photograph taken. As with the enhanced kiosk alternative, a receipt is printed to provide documentation of compliance with the exit process and to assist in compliance on the traveler’s next attempted entry to the country. However, this receipt has biometrics (i.e., the traveler’s fingerprints and photograph) embedded on the receipt. At the conclusion of the transaction, the collected information is transmitted to IDENT. The traveler presents his or her receipt to the WSA or law enforcement officer at the gate or departure area, who scans the receipt using a mobile device. The traveler’s identity is verified against the biometric data embedded on the receipt. Once the traveler’s identity is verified, he/she is allowed to board the departure craft. The captured information is not transmitted in real time back to IDENT. Data collected on the mobile device are periodically uploaded through the kiosk to IDENT. the traveler arrives for inspection. Travelers subject to US-VISIT are to be processed at secondary inspection, rather than at primary inspection. Inspectors’ workstations are to use a single screen, which eliminates the need to switch between the TECS and IDENT screens. Datashare includes a data extract from State’s CCD system and includes the visa photograph, biographical data, and the fingerprint identification number assigned when a nonimmigrant applies for a visa. Chronology of US-VISIT Expenditure Plans Since November 2002, four US-VISIT expenditure plans have been submitted. On November 15, 2002, the Immigration and Naturalization Service (INS)21 submitted to its appropriations subcommittees its first expenditure plan, which outlined $13.3 million in expenditures for contract activities; design, development, and deployment of the Visa Waiver Support System; facilities assessments; biometric standards development; prototyping; IBIS support activities; travel; program office operations; and fingerprint scanner procurements. On June 5, 2003, the second expenditure plan outlined $375 million in expenditures for system enhancements and infrastructure upgrades, POE information technology (IT) and communication upgrades, facilities planning analysis and design, program management support, proof of concept demonstrations, operations and system sustainment, and training. Effective March 1, 2003, INS became part of DHS. On January 27, 2004, the third expenditure plan outlined $330 million in expenditures for exit pilots; capability to read biometrically enabled travel documents; land infrastructure upgrades; system development and testing; radio frequency technology deployment to the 50 busiest land POEs; technical infrastructure planning and development; program management; and operations and maintenance. The current and fourth expenditure plan, submitted on October 19, 2004, outlines $340 million in expenditures (see table, next slide). Background Review of Current Expenditure Plan Fiscal Year 2005 Expenditure Plan Summary (see next slides for descriptions) Objective 1: Legislative Conditions Condition 1 The US-VISIT expenditure plan satisfies or partially satisfies each of the legislative conditions. Condition 1. The plan, including related program documentation and program officials’ statements, partially satisfies the capital planning and investment control review requirements established by OMB, including OMB Circular A-11, part 7, which establishes policy for planning, budgeting, acquisition, and management of federal capital assets. The table that follows provides examples of the results of our analysis. Examples of A-11 conditions Provide justification and describe acquisition strategy. Results of our analysis US-VISIT has completed an Acquisition Plan, dated November 2003. The plan provides a high-level justification and description of the acquisition strategy for the system. US-VISIT completed a cost/benefit analysis for Increment 2B on June 11, 2004. Summarize life-cycle costs and cost/benefit analysis, including the return on investment. Provide performance goals and measures. Address security and privacy. Provide risk inventory and assessment. The plan includes benefits, but does not identify corresponding metrics. The plan states that performance measures are under development. US-VISIT has developed a security plan that partially satisfies OMB and the National Institute of Standards and Technology security guidance. US-VISIT has not yet conducted a security risk assessment on the overall US-VISIT program. While the plan states the intention to do the assessment, it does not specify when it will be completed. The US-VISIT program published a privacy policy and privacy impact assessment for Increment 2. US-VISIT has developed a risk management plan and process for developing, implementing, and institutionalizing a risk management program. Risks are currently tracked using a risk- tracking database. Objective 1: Legislative Conditions Condition 2 Condition 2. The plan, including related program documentation and program officials’ statements, partially satisfies the condition that it provide for compliance with DHS’s enterprise architecture (EA). DHS released version 1.0 of the architecture in September 2003.22 We reviewed the initial version of the architecture and found that it was missing, either partially or completely, all the key elements expected in a well-defined architecture, such as a description of business processes, information flows among these processes, and security rules associated with these information flows.23 Since we reviewed version 1.0, DHS has drafted version 2.0 of its EA. We have not reviewed this draft. Department of Homeland Security Enterprise Architecture Compendium Version 1.0 and Transitional Strategy. GAO, Homeland Security: Efforts Under Way to Develop Enterprise Architecture, but Much Work Remains, GAO-04-777 (Washington, D.C.: Aug. 6, 2004). Objective 1: Legislative Conditions Condition 2 According to officials from the Office of the Chief Strategist, concurrent with the development of the strategic vision, the US-VISIT program office has been working with the DHS EA program office in developing version 2.0 to ensure that US-VISIT is aligned with DHS’s evolving EA. According to these officials, US-VISIT representatives participate in both the DHS EA Center of Excellence and the DHS Enterprise Architecture Board.24 In July 2004, the Center of Excellence reviewed US-VISIT’s submission for architectural alignment with some EA components, but not all. Specifically, the submission included information intended to show compliance with business and data components, but not, for example, the application and technology components. According to the head of DHS’s EA Center of Excellence, the application and technical components were addressed by this center, which found that US-VISIT was in compliance. The Center of Excellence supports the Enterprise Architecture Board in reviewing component documentation. The purpose of the Board is to ensure that investments are aligned with the DHS EA. Objective 1: Legislative Conditions Condition 2 Based on its review, the DHS Enterprise Architecture Board recommended that the US-VISIT program be given conditional approval to proceed for investment, provided that the program resubmit its documentation upon completion of its strategic plan, which is anticipated in January 2005. DHS has not yet provided us with sufficient documentation to allow us to understand DHS architecture compliance methodology and criteria, or verifiable analysis justifying the conditional approval. Objective 1: Legislative Conditions Condition 3 Condition 3. The plan, including related program documentation and program officials’ statements, satisfies the condition that it comply with the acquisition rules, requirements, guidelines, and systems acquisition management practices of the federal government. The plan provides for satisfying this condition, in part, by describing efforts to develop Software Engineering Institute (SEI) Software Acquisition Capability Maturity Model (SA-CMM) key process areas, such as requirements development and management and contract tracking and oversight. The plan also states that the program intends to achieve SA-CMM Level 225 by establishing a process improvement program based on SEI-identified industry best practices. As part of establishing this program, US-VISIT has developed a draft process improvement plan that specifies process improvement goals, objectives, assumptions, and risks, and which describes a process improvement time line and phase methodology. If these processes are implemented effectively, they will help US-VISIT meet federal acquisition rules, requirements, and guidelines and comply with systems acquisition management practices. The SA-CMM ranks organizational maturity according to five levels. Maturity levels 2 through 5 require verifiable existence and use of certain key process areas. Objective 1: Legislative Conditions Condition 4 Condition 4. The plan, including related program documentation and program officials’ statements, partially satisfies the requirement that it be reviewed and approved by the DHS Investment Review Board (IRB), the Secretary of Homeland Security, and OMB. The DHS Under Secretary for Management26 reviewed and approved the fiscal year 2005 expenditure plan on October 14, 2004, and OMB approved the plan on October 15, 2004. According to the US-VISIT Budget and Finance Director, the IRB reviewed the fiscal year 2005 expenditure plan but did not approve it because DHS management determined that review of the expenditure plan was not in the scope of the IRB review process. According to DHS Delegation Number 0201.1, the Secretary of Homeland Security delegated authority to the Under Secretary for Management for, among other things, the budget, appropriations, and expenditure of funds. Objective 1: Legislative Conditions Condition 5 Condition 5. The plan satisfies the requirement that it be reviewed by GAO. Our review was completed on November 23, 2004. Objective 2: Open Recommendations Recommendation 1 Open Recommendation 1: Develop a system security plan and privacy impact assessment. Security Plan. US-VISIT has developed a security plan.27 OMB and the National Institute of Standards and Technology (NIST) have issued security planning guidance28 requiring, in part, the completion of system security plans that (1) provide an overview of the system security requirements, (2) include a description of the controls in place or planned for meeting the security requirements, and (3) delineate roles and responsibilities of all individuals who access the system. US-VISIT Program, Security Plan for US-VISIT Program Version 1.1 (Sept. 13, 2004). OMB Circular A-130, Revised (Transmittal Memorandum No. 4), Appendix III, Security of Federal Automated Information Resources (Nov. 28, 2000) and NIST, Guide for Developing Security Plans for Information Technology Systems, NIST Special Publication 800-18 (December 1998). Objective 2: Open Recommendations Recommendation 1 According to the guidance, the plan should also describe the methodology used to identify system threats and vulnerabilities and to assess risks, and it should include the date the assessment was conducted. If no system risk assessment has been completed, the plan is to include a milestone date for completion. The US-VISIT security plan provides an overview of the system security requirements, describes the controls in place or planned for meeting those requirements, and references the applicable documents that contain roles and responsibilities for the US-VISIT component systems. However, the plan states that although a security risk assessment on the US-VISIT program will be completed in accordance with NIST guidelines, it has not yet been completed, and the plan does not indicate a date for doing so. Objective 2: Open Recommendations Recommendation 1 Privacy Impact Assessment. The US-VISIT program has conducted a privacy impact assessment for Increment 2, and according to the US-VISIT Privacy Officer, a privacy impact assessment will be completed for the exit portion of Increment 1 in early 2005. According to OMB guidance,29 the depth and content of such an assessment should be appropriate for the nature of the information to be collected and the size and complexity of the system involved. The assessment should also, among other things, (1) be updated when a system change creates new privacy risk, (2) ensure that privacy is addressed in the documentation related to system development, (3) address the impact the system will have on an individual’s privacy, (4) analyze the consequences of collection and flow of information, and (5) analyze alternatives to collection and handling as designed. OMB, Guidance for Implementing the Privacy Provisions of the E-Government Act of 2002, OMB M-03-22 (Sept. 26, 2003). Objective 2: Open Recommendations Recommendation 1 The Increment 2 assessment satisfies some, but not all, of the above OMB guidance areas. To DHS’s credit, the assessment, which was completed in September 2004, states that the DHS Chief Privacy Officer directed that the assessment be updated as necessary to reflect future changes to Increment 2. The assessment also discusses the impact that Increment 2 will have on an individual’s privacy and analyzes the consequences of collection and flow of information. However, privacy is only partially addressed in the Increment 2 system documentation. For example, privacy is used in the Increment 2B cost-benefit analysis to evaluate the weighted risk of Increment 2B alternative solutions. Additionally, the ADIS functional requirements specify that access to information contained in the system, which is protected by the Privacy Act,30 must be limited to authorized users. However, the IDENT Server 2.0 requirements do not consider privacy at all. Additionally, the assessment’s only discussion of design is a statement that a major choice for US-VISIT was whether to develop an entirely new system, develop a largely new system, or build upon existing systems. The assessment does not analyze these options. The timing of the planned privacy impact assessment for the exit portion of Increment 1 is consistent with plans for completing the exit pilots. Objective 2: Open Recommendations Recommendation 2 Open Recommendation 2: Develop and implement a plan for satisfying key acquisition management controls—including acquisition planning, solicitation, requirements development and management, project management, contract tracking and oversight, evaluation, and transition to support—and implement the controls in accordance with SEI guidance. The US-VISIT program plans to achieve SEI SA-CMM Level 2 status in October 2006. According to SEI, a process improvement effort should involve building a process infrastructure, establishing current levels of process maturity, and completing an action plan. The plan should include, among other things, process improvement assumptions and risks, goals, objectives, and criteria for success. The US-VISIT Acquisition and Program Management Office (APMO) has initiated a process improvement program and drafted a process improvement plan. Objective 2: Open Recommendations Recommendation 2 The draft US-VISIT plan discusses assumptions, such as the improvement program being sponsored and supported by senior US-VISIT management, and risks, such as not meeting the process improvement time line if the process improvement effort is not fully staffed. The plan also lists both process improvement goals and short- and long-term objectives. However, the goals and objectives are generally not defined in measurable terms. For example, the plan identifies the following goal and objective: Goal: ensure that US-VISIT is in compliance with federal mandates, making future funding more likely. Objective: define a strategy for attaining SEI SA-CMM Level 2 as soon as possible within the existing constraints—limited contractor and government staff resources and centralized facility. The plan also does not address criteria for success. Objective 2: Open Recommendations Recommendation 2 APMO has developed processes or plans, some of which are approved and some of which are in draft, for all key process areas except “transition to support.”31 The Director of APMO could not say when APMO plans to develop the documentation for this key process area, but noted that US-VISIT is considering a transition from the SA-CMM to SEI’s Capability Maturity Model Integration (CMMI) model.32 No time line was provided as to when this decision might be made. The Director of APMO acknowledges that a transition to the CMMI will likely change the previously mentioned time line for CMM certification. The purpose of transition to support is to provide for the effective and efficient “handing off” of the acquired software products to the support organization responsible for software maintenance. CMU/SEI-2004-TR-001 (February 2004). Objective 2: Open Recommendations Recommendation 3 Open Recommendation 3: Ensure that future expenditure plans are provided to the DHS’s House and Senate Appropriations Subcommittees on Homeland Security in advance of US-VISIT funds being obligated. On October 18, 2004, the President signed the Department of Homeland Security Appropriations Act, 2005, which included $340 million in fiscal year 2005 funds for the US-VISIT program.33 The act states that $254 million of the $340 million is subject to the expenditure plan requirement. On October 19, 2004, DHS provided its fiscal year 2005 expenditure plan to the Senate and House Appropriations Subcommittees on Homeland Security. Department of Homeland Security Appropriations Act, 2005, Pub. L. 108-334 (Oct. 18, 2004). Objective 2: Open Recommendations Recommendation 4 Open Recommendation 4: Ensure that future expenditure plans fully disclose US- VISIT system capabilities, schedule, cost, and benefits to be delivered. The expenditure plan identifies high-level capabilities by increments. However, the capabilities are not consistently presented. For example, in one section of the plan, Increment 2B capabilities are identified as collect biometric data and verify identity at the 50 busiest land POEs, develop global enrollment system capability, and support facilities delivery. However, later in the plan, Increment 2B capabilities are identified as Increment 1 functionality at the top 50 land POEs, biometric data collection, and infrastructure upgrades. Objective 2: Open Recommendations Recommendation 4 Further, some of the capabilities are described in vague and ambiguous terms. For example, the plan describes such Increment 2C capabilities as integration of Border Crossing Cards with US-VISIT, test, model, and deploy technology to preposition biographic and biometric data of enrolled travelers, and desktop upgrades. The plan identifies specific milestones for some increments, but not for others. For example, it states that Increment 2B is to be implemented by December 31, 2004, and Increment 3 by December 31, 2005. However, it states that Increment 1 exit and Increment 2C are to be implemented in fiscal year 2005. Objective 2: Open Recommendations Recommendation 4 The plan identifies the amounts budgeted for each increment for fiscal years 2003 through 2005. For example, the plan states that US-VISIT plans to obligate $55 million in fiscal year 2005 funds for Increment 2C. However, the plan does not associate the $55 million with specific Increment 2C capabilities and benefits. Rather, it states that this amount will be used to support Increment 2C by funding the installation of technology in entry and exit lanes at land borders and supporting facility delivery. Further, the plan does not identify any estimated nongovernmental costs, such as the social costs associated with any potential economic impact at the border. Objective 2: Open Recommendations Recommendation 4 The plan identifies several benefits and associates these benefits with increments. For example, for Increment 1, the plan identifies such benefits as prevention of entry of high-threat or inadmissible individuals through improved and/or advanced access to data before the foreign national’s arrival, improved enforcement of immigration laws through improved data accuracy and completeness, reduction in foreign nationals remaining in the country under unauthorized circumstances, and reduced threat of terrorist attack and illegal immigration through improved identification of national security threats and inadmissible individuals. As we previously reported,34 these benefits were identified in the fiscal year 2004 expenditure plan, although they were not associated with Increment 1. GAO, Homeland Security: First Phase of Visitor and Immigration Status Program Operating, but Improvements Needed, GAO-04-586 (Washington, D.C.: May 11, 2004). Objective 2: Open Recommendations Recommendation 4 Further, the fiscal year 2004 plan included planned metrics for the first two benefits identified above and stated that US-VISIT was developing metrics for measuring the projected benefits, including baselines by which progress can be assessed. However, the fiscal year 2005 plan does not include any information on these metrics or on progress against any of the benefits. The fiscal year 2005 plan again states that performance measures are still under development. While the plan does not associate any measures with the defined benefits, it does identify several measures and links them to the US-VISIT processes—pre-entry, entry, status management, exit, and analysis. The plan also identifies examples of how US-VISIT is addressing its four stated goals. The examples, however, largely describe US-VISIT functions rather than measures of goal achievement. For example, in support of the stated goal of ensuring the integrity of our immigration system, the plan states that through US- VISIT, officers at primary inspection are able to instantly search databases of known criminals and known and suspected terrorists. It does not, however, identify how this ensures immigration system integrity. Objective 2: Open Recommendations Recommendation 5 Open Recommendation 5: Ensure that future expenditure plans fully disclose how the US-VISIT acquisition is being managed. The expenditure plan describes some activities being employed to manage the US- VISIT acquisition. For example, the plan describes the US-VISIT governance structure, as well as the program office organizational structure and staffing levels. The plan also describes certain management processes currently being used. For example, the plan states that US-VISIT program officials hold formal weekly meetings to discuss program risks/issues, schedule items, and critical path items. In addition, it states that formal points of contact for risk issues have been designated across the Increment Integrated Project teams, and the US-VISIT program organization and the plan states that US-VISIT is establishing a formal risk review board to review and manage risk. However, the plan does not describe how other important aspects of the program are being managed, several of which are discussed in this briefing. For example, it does not describe how testing, system capacity, and systems configuration are being managed. Objective 2: Open Recommendations Recommendation 6 Open Recommendation 6: Ensure that human capital and financial resources are provided to establish a fully functional and effective program office. DHS established the US-VISIT program office in July 2003 and determined the office’s staffing needs to be 115 government and 117 contractor personnel. As of October 2004, DHS had filled 59 of the 115 government positions. Of those positions that have not been filled, 5 have reassignments in progress and 51 have competitive announcements pending. According to US-VISIT, about half of these positions are to be filled when security clearances are completed. In addition, US-VISIT has changed its organizational structure, and some positions were moved to other offices within US-VISIT. For example, the number of positions in the Office of Mission Operations Management decreased from 23 to 18, and the number of positions in the Office of Chief Strategist increased from 10 to 14. Also, the number of positions in the Office of Administration and Management—now called the Office of Administration and Training—increased from 10 to 11. Objective 2: Open Recommendations Recommendation 6 The graphic on the next page shows the US-VISIT program office organization structure and functions, the number of positions needed by each office, and the number of positions filled. This graphic reflects the recent changes to the US-VISIT organizational structure. Objective 2: Open Recommendations Recommendation 6 In addition to the 115 government staff that were anticipated, the program anticipated 117 contractor support staff. As of November 2004, program officials told us they had filled 88 of these 117 positions. The expenditure plan also states that DHS has budgeted $83 million to maintain the program management structure and baseline operations, including, among other things, salaries and benefits for government full-time equivalents, personnel relocation costs, rent, and supplies. Objective 2: Open Recommendations Recommendation 7 Open Recommendation 7: Clarify the operational context in which US-VISIT is to operate. DHS is in the process of defining the operational context in which US-VISIT is to operate. In September 2003, DHS released version 1.0 of its enterprise architecture.35 We reviewed the initial version of the architecture and found that this architecture was missing, either partially or completely, all the key elements expected in a well-defined architecture, such as descriptions of business processes, information flows among these processes, and security rules associated with these information flows.36 Since we reviewed version 1.0, DHS has drafted version 2.0 of its architecture. We have not reviewed the draft, but DHS EA program officials told us this version focuses on departmental operations, and that later versions will incrementally focus on the national homeland security picture. This is important to the US-VISIT operational context because US-VISIT is a governmentwide program, including entities outside DHS, such as the Departments of State and Justice. Department of Homeland Security Enterprise Architecture Compendium Version 1.0 and Transitional Strategy. GAO, Homeland Security: Efforts Under Way to Develop Enterprise Architecture, but Much Work Remains, GAO-04-777 (Washington, D.C.: Aug. 6, 2004). Objective 2: Open Recommendations Recommendation 8 Open Recommendation 8: Determine whether proposed US-VISIT increments will produce mission value commensurate with cost and risks. US-VISIT developed a cost-benefit analysis (CBA) for Increment 2B, dated June 11, 2004. However, the CBA’s treatment of both benefits and costs raises several issues, making it unclear whether Increment 2B will produce mission value commensurate with cost and risks. First, the CBA primarily addresses government costs and is silent on some potential nongovernmental costs. For example, the CBA does not consider potential social costs like the economic impact on border communities. Objective 2: Open Recommendations Recommendation 8 operational performance benefits, such as improvement of traveler identification and validation of traveler documentation. Moreover, the CBA does not explain why these benefits cannot be quantified. Also, the CBA states that none of the proposed alternatives result in a positive net present value or return on investment, which it attributes to the limited scope of Increment 2B. Third, the CBA includes three alternatives and identifies alternative 3 as the preferred alternative. However, US-VISIT is not pursuing alternative 3, but rather is pursuing an alternative more aligned with alternative 2. According to the Program Director, this is because alternative 3 was considered too ambitious to meet the statutory requirement that US-VISIT be implemented at the 50 busiest land POEs by December 31, 2004. Objective 2: Open Recommendations Recommendation 9 Open Recommendation 9: Define US-VISIT program office positions, roles, and responsibilities. US-VISIT has developed descriptions for positions within each office. In addition, US-VISIT has worked with the Office of Personnel Management (OPM) to draft a set of core competencies that define the knowledge, skills, abilities, and other characteristics (competencies) needed for successful employee performance. According to US-VISIT’s draft Human Capital Plan, these core competencies will form the foundation for recruitment and selection, training and development, and employee performance evaluations. Currently, US-VISIT is using some of these draft core competencies in its employee performance appraisal process. Objective 2: Open Recommendations Recommendation 10 Open Recommendation 10: Develop and implement a human capital strategy for the US-VISIT program office that provides for staffing positions with individuals who have the appropriate knowledge, skills, and abilities. The US-VISIT program office awarded a contract to OPM to develop a draft Human Capital Plan. Our review of the draft plan showed that OPM developed a plan for US-VISIT that employed widely accepted human capital planning tools and principles. OPM’s recommendations to US-VISIT include the following: Develop and adopt a competency-based system and a corresponding human capital planning model that illustrate the alignment of US-VISIT’s mission with individual and organizational performance. Conduct a comprehensive workforce analysis to determine diversity trends, retirement and attrition rates, and mission-critical and leadership competency gaps. Objective 2: Open Recommendations Recommendation 10 Develop a leadership competency model and establish a formal leadership development program to ensure continuity of leadership. Link the competency-based human capital management system to all aspects of human resources, including recruitment, assessment, training and development, and performance. The draft human capital plan includes an action plan that identifies activities, proposed completion dates, and the office (OPM or US-VISIT) responsible for completing these activities. According to OPM, it has completed its work under the draft plan. As of October 2004, US-VISIT had completed some of the activities called for in the draft plan. For example, US-VISIT’s Office of Administration and Training has designated a liaison responsible for ensuring alignment between DHS and US-VISIT human capital policies. However, it remains to be seen how full implementation of the plan will impact the US-VISIT program office. For example, the workforce analysis called for in the draft plan could result in a change in the number and competencies of the staff needed to implement US-VISIT. Objective 2: Open Recommendations Recommendation 11 Open Recommendation 11: Develop a risk management plan and report all high risks and their status to the executive body on a regular basis. The US-VISIT program office has developed a risk management plan (dated June 2, 2004) and process (dated June 9, 2004). The plan addresses, among other things, the process for identifying, analyzing, mitigating, tracking, and controlling risks. As part of its process, US-VISIT has developed a risk management database. The database includes, among other things, a description of the risk, its priority (e.g., high, medium, low), and mitigation strategy. US-VISIT has also established the governance structure for managing risks. The governance structure includes three primary groups—the Risk Review Board, Risk Review Council, and Risk Management Team. Objective 2: Open Recommendations Recommendation 11 The Risk Review Board provides overall decision making, communication, and coordination in regard to risk activities. The board is composed of senior-level staff, such as the program director and functional area directors. The Risk Review Council reviews initially reported risks, validates their categorizations, and ensures that a mitigation approach has been developed. It also serves as a filter for the Board by deciding which risks can be mitigated without being elevated to the Board. The Risk Management Team provides risk management expertise and institutional knowledge. This group is staffed by APMO. According to the Director, APMO, US-VISIT has not reported high risks beyond the Review Board. Objective 2: Open Recommendations Recommendation 12 Open Recommendation 12: Define performance standards for each US-VISIT increment that are measurable and reflect the limitations imposed by relying on existing systems. Available documentation shows that some technical performance measures for Increments 1 and 2B have been defined. For example: Availability.37 The system will be available 99.5 percent of the time. Timeliness.38 Login, visa query, and TECS/NCIC default query will be less than 5 seconds; TECS optional queries will be less than 60 seconds; and IDENT watch list queries will be less than 10 seconds (matcher time only). Output quantity.39 70,000 primary inspection transactions per user, per day, with a maximum of 105,000 transactions during peak times. The time the system is operating satisfactorily, expressed as a percentage of time that the system is required to be operational. The time needed to perform a unit of work correctly and on time. The number of transactions processed. Objective 2: Open Recommendations Recommendation 12 However, other measures, such as reliability,40 resource utilization,41 and scalability,42 are not defined in the documentation. Further, the documentation does not contain sufficient information to determine the limitations imposed by US- VISIT’s reliance on existing systems that have less demanding performance requirements, such as TECS availability of 98.0 percent. Such information would include, for example, the processing sequencing and dependencies among the existing systems. The probability that a system, including all hardware, firmware, and software, will satisfactorily perform the task for which it was designed. A ratio representing the amount of time a system or component is busy divided by the time it is available. Ability of a system to function well when it is changed in size or volume. Objective 2: Open Recommendations Recommendation 13 Open Recommendation 13: Develop and approve test plans before testing begins. These test plans should (1) specify the test environment; (2) describe each test to be performed, including test controls, inputs, and expected outputs; (3) define the test procedures to be followed in conducting the tests; and (4) provide traceability between test cases and the requirements to be verified by the testing. According to the US-VISIT Systems Assurance Director, the Increment 2B system acceptance test (SAT) plan was approved during an October 15, 2004, test readiness review (TRR). However, no documentation was provided that explicitly indicated the approval of the plan, and the results of the TRR were not approved until October 28, 2004, which is 11 days after the date we were told that acceptance testing began. Objective 2: Open Recommendations Recommendation 13 The test plan does not fully address the test environment. For example, the plan does not describe the scope, complexity, and completeness of the test environment or identify necessary training. The plan does include generic descriptions of testing hardware, such as printers and card readers. The plan does not include descriptions of tests to be performed. However, officials from the IT Management Office provided us with other documentation describing the tests to be performed that included expected outputs, but it did not include inputs or controls. The plan does not provide test procedures to be followed in conducting the tests. Objective 2: Open Recommendations Recommendation 13 The plan does not provide traceability between test cases and the requirements to be verified by the testing. Our analysis of the 116 requirements identified in the consolidated requirements document showed that 39 requirements mapped to test cases that lacked sufficient detail to determine whether the test cases are testable, 15 requirements did not have test cases, 2 requirements were labeled “not testable,” and 1 requirement was identified as “TBD,” but was mapped to an actual test case. Open Recommendation 14: Ensure the independence of the Independent Verification and Validation (IV&V) contractor. According to the US-VISIT Program Director, the US-VISIT IT Management Office is developing high-level requirements for IV&V. In particular, it is developing a strategy and statement of work for acquiring an IV&V contractor. Objective 2: Open Recommendations Recommendation 15 Open Recommendation 15: Implement effective configuration management practices, including establishing a US-VISIT change control board to manage and oversee system changes. According to US-VISIT’s draft configuration management (CM) plan, dated July 2004, and US-VISIT officials, US-VISIT has not yet developed or implemented US- VISIT-level configuration management practices or a change control board. In the interim, for Increments 1, 2A and 2B, US-VISIT continues to follow relevant IDENT, ADIS, and TECS configuration management procedures, including applicable change control boards and system change databases. According to the US-VISIT System Assurance Director, for Increment 2B, US-VISIT is using the TECS change requests database for US-VISIT change requests, including those for IDENT and ADIS. Objective 2: Open Recommendations Recommendation 15 The draft configuration management plan describes key configuration activities that are to be defined and implemented, including (1) defining and identifying processes and products to be controlled; (2) evaluating, coordinating, and approving/rejecting changes to controlled items; (3) recording and monitoring changes to the controlled items; and (4) verifying that the controlled items meet their requirements and are accurately documented. The draft plan also proposes a governance structure, including change control boards. The proposed governance structure includes the following: A US-VISIT CM team is responsible for implementing, controlling, operating, and maintaining all aspects of configuration management and administration for US-VISIT. The team is to be composed of a CM manager, CM team staff, DHS system CM liaisons, prime integrator CM liaison, and testers and users. A change control board is to serve as the ultimate authority on changes to any US-VISIT system baseline, decide the content of system releases, and approve the schedule of releases. Objective 2: Open Recommendations Recommendation 16 Open Recommendation 16: Identify and disclose management reserve funding embedded in the fiscal year 2004 expenditure plan to the Appropriations Subcommittees. The US-VISIT program office reported the management reserve funding of $33 million for fiscal year 2004 to the Appropriations Subcommittees. According to the Deputy Program Manager, US-VISIT provided this information in a briefing to the Subcommittee staff. Open Recommendation 17: Ensure that all future US-VISIT expenditure plans identify and disclose management reserve funding. The fiscal year 2005 expenditure plan specified management reserve funding of $23 million. Objective 2: Open Recommendations Recommendation 18 Open Recommendation 18: Assess the full impact of Increment 2B on land POE workforce levels and facilities, including performing appropriate modeling exercises. US-VISIT conducted an Increment 2B baseline analysis to help determine the impact of Increment 2B on workforce and travelers. The analyses included three sites and addressed the Form I-94 issuance process and the Form I-94W43 process in secondary inspection. According to program officials, additional staff will not be needed to implement 2B at the border. Instead, US-VISIT has developed a plan to train existing Customs and Border Protection officers on the collection of traveler entry data, has completed the “train the trainer” classes at the training academy, and has begun training at three land POEs. I-94W is used for foreign nationals from visa waiver countries. Objective 2: Open Recommendations Recommendation 18 In addition, US-VISIT has conducted space utilization surveys at all of the 166 land POEs and completed survey reports at 16 of the 50 busiest land POEs. US-VISIT expects to have completed survey reports for the remaining 34 busiest land POEs during the fall of 2004. According to the 16 completed survey reports, existing traffic at most of these facilities was at or near capacity and the facilities had no room for expansion. However, US-VISIT officials said that Increment 2B will not require expansion at any facilities; rather, it will require mostly minor modifications, such as the installation of new or updated countertops and electrical power outlets to accommodate new equipment. Objective 2: Open Recommendations Recommendation 19 Open Recommendation 19: Develop a plan, including explicit tasks and milestones, for implementing all our open recommendations and periodically report to the DHS Secretary and Under Secretary on progress in implementing this plan; also report this progress, including reasons for delays, in all future US-VISIT expenditure plans. The US-VISIT program office has developed a report for tracking the status of our open recommendations. This report is shared with the program office director, but according to the Deputy Program Director, it is not shared with the Secretary and Under Secretary. In addition, he stated that the program office meets weekly with the Under Secretary, but the status of our recommendations are not discussed. The fiscal year 2005 expenditure plan summarizes our recommendations, but it does not identify tasks and milestones for implementing them or discuss progress in implementing them. Observation 1: The program office has acquired the services of a prime integration contractor to augment its ability to complete US-VISIT. DHS reported in its fiscal year 2004 US-VISIT expenditure plan that it had intended to award a contract by the end of May 2004 to a prime contractor for integrating existing and new business processes and technologies. US-VISIT awarded the contract on time. Specifically, on May 28, 2004, DHS awarded its prime contract to Accenture LLP and its related partners. Objective 3: Observations Progress Observation 2: The fiscal year 2005 Expenditure Plan does not describe progress against commitments (e.g., capabilities, schedule, cost, and benefits) made in previous plans. Given the immense importance of the US-VISIT program to the security of our nation’s borders and the need to acquire and implement it efficiently and effectively, the Congress has placed limitations on the use of appropriations for the US-VISIT program until DHS submits periodic expenditure plans. As we had previously reported,44 to permit meaningful congressional oversight, it is important that expenditure plans describe how well DHS is progressing against the commitments made in prior expenditure plans. GAO, Information Technology: Homeland Security Needs to Improve Entry Exit System Expenditure Planning, GAO-03-563 (Washington, D.C.: June 9, 2003). Objective 3: Observations Progress The fiscal year 2005 expenditure plan does not describe progress against commitments made in prior expenditure plans. For example, in its fiscal year 2004 expenditure plan, US-VISIT committed to, among other things, analyzing, field testing, and initiating deployment of alternative approaches for capturing biometrics during the exit process at air and sea POEs and implementing entry and exit capabilities at the 50 busiest land POEs by December 31, 2004, including delivering the capability to read radio frequency enabled documents at the 50 busiest land POEs for both entry and exit processes. The fiscal year 2005 plan does not address progress against these commitments. For example, the plan does not describe the status of the exit pilot testing or deployment, such as whether it has met its target schedule or whether the schedule has slipped. While the plan does state that US-VISIT will expand its pilot sites during the summer and fall of 2004 and deploy the exit solution during fiscal year 2005, it does not explain the reason for the change or its potential impact. The following graphic provides our analysis of the commitments made in the fiscal year 2003 and 2004 plans, compared with currently reported and planned progress. Objective 3: Observations Progress Further, the fiscal year 2004 plan states that $45 million in fiscal year 2004 funds were to be used for exit activities. However, the fiscal year 2005 plan states that $73 million in fiscal year 2004 funds were to be used for exit activities, but does not highlight this difference or address the reason for the change in budget amounts. Also, the fiscal year 2005 expenditure plan includes benefits stated in the fiscal year 2004 plan, but it does not provide progress in addressing those benefits, despite the fact that, in the fiscal year 2004 plan, US-VISIT stated that it was developing metrics for measuring the projected benefits, including baselines by which progress could be assessed. The fiscal year 2005 plan again states that performance measures are under development. This information is needed to allow meaningful congressional oversight of plans and progress. Objective 3: Observations Exit Deployment Observation 3: The exit capability alternatives are faced with a compressed time line, missed milestones, and potentially reduced scope. On January 5, 2004, US-VISIT deployed an initial exit capability in pilot status to two POEs. At that time, the Program Director stated that US-VISIT was developing other exit alternatives, along with criteria for evaluating and selecting one or more of the alternatives by December 31, 2004. Planned evaluation time line compressed In May 2004, US-VISIT issued an Exit Pilot Evaluation Execution Plan. This plan states that three alternative exit solutions are to be evaluated while deployed to a total of 15 air and sea POEs. The plan allotted about 3 months to conduct the evaluation and report the results. Specifically, the deployment was to be completed by August 1, 2004, and all exit pilot evaluation tasks were to be completed by September 30, 2004, with an evaluation report finished by October 28, 2004. Objective 3: Observations Exit Deployment However, according to the exit master schedule provided to us on October 26, 2004, the three alternatives were scheduled to be fully deployed by October 29, 2004, and all evaluation tasks are to be completed on December 6, 2004, with delivery of the evaluation report on December 30, 2004, which is about a 2-month evaluation and reporting period. The following graphic illustrates how the exit pilot schedule has been shortened from the originally planned 3 months to the currently planned 2 months and compares the original plan with the current plan. Objective 3: Observations Exit Deployment As of November 8, 2004, the three alternatives were deployed and operational in only 5 of the 15 POEs that were to be operational by November 1. According to the Exit Implementation Manager, all ports had received and installed the exit equipment. However, the requisite number of contract employees (WSAs) is not yet available to make all 15 POEs operational because of delays in DHS granting security clearances to the attendants. The manager stated that a recent meeting with DHS security officials has helped to improve the pace of finalized security clearances, but the manager did not know when the remaining 10 ports would become operational. Objective 3: Observations Exit Deployment The Evaluation Execution Plan describes the evaluation methodology that is to be employed for the three alternatives. An important element of that methodology is the targeted sample size per port. For each port, a targeted number of outbound passengers will be processed by the three alternatives and data gathered on these encounters. The plan’s specified sample sizes are described as sufficient to achieve a 95 percent confidence level with a margin of error of 5 percent. According to the Exit Implementation Manager, the desired sample size will be collected at each port, despite the compressed time frame for conducting the evaluations, by adding additional personnel to the evaluation teams if needed. These changing facts and circumstances surrounding the exit pilot introduce additional risk concerning US-VISIT’s delivery of promised capabilities and benefits on time and within budget. Objective 3: Observations Exit Deployment On November 12, 2004, US-VISIT issued a revised draft Exit Pilot Evaluation Plan. However, the plan does not address any of the concerns cited, in part because it does not include a planned completion date. Instead, the plan states that the evaluation period is planned for October 31, 2004, until completion. Without a planned completion date, it is not possible to determine the length of the evaluation period or any impact that the length of the evaluation may have on the evaluation’s scope. Observation 4: US-VISIT and Automated Commercial Environment (ACE) collaboration is moving slowly. The US-VISIT EA alignment analysis document describes a port of entry/exit management conceptual project that is to establish uniform processes at POEs and the capability to inspect and categorize people and goods and act upon the information collected. The document recognizes that both US-VISIT and ACE45 support this project because they have related missions and a planned presence at the borders, including the development and deployment of infrastructure and technology. We recognized the relationships between these two programs in February 2003,46 when we recommended that future ACE expenditure plans specifically address any proposals or plans, whether tentative or approved, for extending and using ACE infrastructure to support other homeland security applications. ACE is a new trade processing system planned to support the movement of legitimate imports and exports and strengthen border security. GAO, Customs Service Modernization: Automated Commercial Environment Progressing, but Further Acquisition Management Improvements Needed, GAO-03-406 (Washington D.C.: Feb. 28, 2003). Objective 3: Observations Collaboration people, processes, and technology, which includes establishing a team to review deployment schedules and establishing a team and process to review and normalize business requirements. In August 2004, the US-VISIT and ACE programs tasked their respective contractors to form collaboration teams to address the three areas. Nine teams have been formed: business; organizational change management; facilities; information and data; technology; privacy and security; deployment, operations, and maintenance; and program management. Objective 3: Observations Collaboration The teams met in September 2004 to develop team charters, identify specific collaboration opportunities, and develop time lines and next steps. In October 2004, US-VISIT and ACE contractors met US-VISIT and ACE management to present their preliminary results. According to a US-VISIT official, the team charters have not yet been formally approved. Since we recommended steps to promote close collaboration between these two programs, about 20 months have passed, and explicit plans have not been developed nor actions taken to understand US-VISIT/ACE dependencies and relationships so that these can be exploited to optimize border operations. During this time and in the near future, the management of both programs have been and will be making and acting on decisions to further define, design, develop, and implement their respective programs. The longer it takes for the programs to exploit their relationships, the more rework will be needed at a later date to integrate the two programs. According to the US-VISIT Program Director, the pace of collaboration activities has been affected by scheduling and priority conflicts, as well as staff availability. Observation 5: US-VISIT system capacity is being managed in a compartmentalized manner. Capacity management is intended to ensure that systems are properly designed and configured for efficient performance and have sufficient processing and storage capacity for current, future, and unpredictable workload requirements. Capacity management includes (1) demand forecasting, (2) capacity planning, and (3) performance management. Demand forecasting ensures that the future business requirement workloads are considered and planned. Capacity planning involves determining current and future resource requirements and ensuring that they are acquired and implemented in a timely and cost-effective manner. Performance management involves monitoring the performance of system resources to ensure required service levels are met. The US-VISIT system, as noted earlier, is actually a system made up of various pre-existing (or legacy) systems that are operated by different DHS organizational components and that have been enhanced and interfaced. Objective 3: Observations Capacity Management Currently, DHS does not have a capacity management program. Instead, the US- VISIT IT Management Office relies on the performance management activities of the respective pre-existing DHS systems. For example: A quarterly report provided by the Customs and Border Protection Systems Engineering Branch Performance Engineering Team tracks such system measures as transaction volume, central processing unit utilization, and workload growth. Immigration and Customs Enforcement tracks such system measures as hourly and daily transaction rates and response times. According to the program office, the system-of-systems nature of US-VISIT does not lend itself to easily tracking systemwide performance. Nevertheless, program officials told us that the US-VISIT program has tasked two of its contractors with developing a comprehensive performance management and capacity planning effort. Until this is developed, the program will continue to rely on component system performance management activities to ensure that US-VISIT system resources are sufficient to meet current US-VISIT workloads, which increases the risk that they may not be able to adequately support US-VISIT mission needs. Objective 3: Observations Cost Estimate Observation 6:The cost estimating process used for Increment 2B did not follow some key best practices. SEI recognizes the need for reliable cost-estimating processes in managing software-intensive system acquisitions. To this end, SEI has issued a checklist47 to help determine the reliability of cost estimates. Our analysis found that US-VISIT did not fully satisfy most of the criteria on SEI’s checklist. The US-VISIT Increment 2B estimate met two of the checklist items that we evaluated, partially met six, and did not meet five. For example, US-VISIT provided no evidence that Increment 2B was appropriately sized. Specifically, costs related to development and integration tasks for the TECS, IDENT, and ADIS systems are specified, but estimated software lines of code to be reused, modified, added, or deleted are not. As another example, no one outside the US-VISIT program office reviewed and concurred with the cost estimating categories and methodology. The table on the following slides summarizes our analysis of the extent to which US-VISIT’s cost-estimating process for Increment 2B met SEI’s criteria. Carnegie Mellon University Software Engineering Institute, A Manager’s Checklist for Validating Software Cost and Schedule Estimates, CMU/SEI-95-SR-004 (January 1995). Criterion 1. The objectives of the estimate are stated in writing. 2. The life cycle to which the estimate applies is clearly defined. 3. The task has been appropriately sized (e.g., software lines of code). 4. The estimated cost and schedule are consistent with demonstrated accomplishments on other projects. 5. A written summary of parameter values and their rationales accompanies the estimate. 6. Assumptions have been identified and explained. 7. A structured process such as a template or format has been used to ensure that key factors have not been overlooked. Criterion 8. Uncertainties in parameter values have been identified and quantified. 9. 10. If a dictated schedule has been imposed, an estimate of the normal schedule has been compared to the additional expenditures required to meet the dictated schedule. If more that one cost model or estimating approach has been used, any differences in results have been analyzed and explained. 11. Estimators independent of the performing organization concurred with the reasonableness of the parameter values and estimating methodology. 12. Estimates are current. 13. The results of the estimate have been integrated with project planning and tracking. Objective 3: Observations Cost Estimate Without reliable cost estimates, the ability to make informed investment decisions and effectively measure progress and performance is reduced. progressing against the commitments that it made in prior expenditure plans. Reassess its plans for deploying an exit capability to ensure that the scope of the exit pilot provides for adequate evaluation of alternative solutions, and better ensures that the exit solution selected is in the best interest of the program. Develop and implement processes for managing the capacity of the US-VISIT system. Follow effective practices for estimating the costs of future increments. VISIT and ACE programs a priority matter, and report periodically to the Under Secretary on progress in doing so. To accomplish our objectives, we performed the following tasks: We analyzed the expenditure plan against legislative conditions and other relevant federal requirements, guidance, and best practices to determine the extent to which the conditions were met. We analyzed key acquisition management controls documentation and interviewed program officials to determine the status of our open recommendations. We analyzed supporting documentation and interviewed DHS and US-VISIT program officials to determine capabilities in key program management areas, such as enterprise architecture and capacity management. We analyzed Increment 2B systems and software testing documentation and compared them with relevant guidance to determine completeness. We attended program working group meetings. We assessed the reliability of US-VISIT’s Increment 2B cost estimate by selecting 13 criteria from the SEI checklist48 that, in our professional judgment, represent the minimum set of criteria necessary to develop a reliable cost estimate. We analyzed the Increment 2B cost-benefit analysis and supporting documentation and interviewed program officials to determine how the estimate was derived. We then assessed each of the criteria as satisfied (US- VISIT provided substantiating evidence for the criterion), partially satisfied (US- VISIT provided partial evidence, including testimonial evidence, for the criterion), and not satisfied (no evidence was found for the criterion). We did not review the State Department’s implementation of machine- readable, tamper-resistant visas that use biometrics. For DHS-provided data that our reporting commitments did not permit us to substantiate, we have made appropriate attribution indicating the data’s source. Carnegie Mellon University Software Engineering Institute, A Manager’s Checklist for Validating Software Cost and Schedule Estimates, CMU/SEI-95-SR-004 (January 1995). We conducted our work at US-VISIT program offices in Rosslyn, Virginia, from June 2004 through November 2004, in accordance with generally accepted government auditing standards. Attachment 2 Recent US-VISIT Studies Border Security: State Department Rollout of Biometric Visas on Schedule, but Guidance Is Lagging. GAO-04-1001. Washington, D.C.: September 9, 2004. Border Security: Joint, Coordinated Actions by State and DHS Needed to Guide Biometric Visas and Related Programs. GAO-04-1080T. Washington, D.C.: September 9, 2004. Homeland Security: First Phase of Visitor and Immigration Status Program Operating, but Improvements Needed. GAO-04-586. Washington, D.C.: May 11, 2004. DHS Office of Inspector General. An Evaluation of the Security Implications of the Visa Waiver Program. OIG-04-26. Washington, D.C.: April 2004. Homeland Security: Risks Facing Key Border and Transportation Security Program Need to Be Addressed. GAO-04-569T. Washington, D.C.: March 18, 2004. Homeland Security: Risks Facing Key Border and Transportation Security Program Need to Be Addressed. GAO-03-1083. Washington, D.C.: September 19, 2003. Information Technology: Homeland Security Needs to Improve Entry Exit System Expenditure Planning. GAO-03-563. Washington, D.C.: June 9, 2003. In addition to the individual named above, Barbara Collier, Neil Doherty, David Hinchman, James Houtz, Carolyn Ikeda, Anh Le, John Mortin, David Noone, Karen Richey, Karl Seifert, and Randolph Tekeley made key contributions to this report.

Summary: The Department of Homeland Security (DHS) has established a program--the U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)--to collect, maintain, and share information, including biometric identifiers, on selected foreign nationals who travel to the United States. By congressional mandate, DHS is to develop and submit for approval an expenditure plan for US-VISIT that satisfies certain conditions, including being reviewed by GAO. Among other things, GAO was asked to determine whether the plan satisfied these conditions and to provide observations on the plan and DHS's program management. DHS's fiscal year 2005 expenditure plan and related documentation at least partially satisfied all conditions established by the Congress, including meeting the capital planning and investment control requirements of the Office of Management and Budget (OMB). For example, DHS has developed a plan and a process for developing, implementing, and institutionalizing a program to manage risk. In its observations about the expenditure plan and DHS's management of the program, GAO recognizes accomplishments to date and addresses the need for rigorous and disciplined program practices. For example, US-VISIT has acquired the services of a prime integration contractor to augment its ability to complete US-VISIT. However, DHS has not employed rigorous, disciplined processes typically associated with successful programs, such as tracking progress against commitments. More specifically, the fiscal year 2005 plan does not describe progress against commitments made in previous plans (e.g., capabilities, schedule, cost, and benefits). According to GAO's analysis, delays have occurred in delivering capability to track the entry and exit of persons entering the United States at air, land, and sea ports of entry. Such information is essential for oversight. Additionally, the effort to pilot alternatives for delivering the capability to track the departure of persons exiting the United States is faced with a compressed time line, missed milestones, and potentially reduced scope. In particular, the pilot evaluation period has been reduced from 3 to 2 months, and as of early November 2004, the alternatives were deployed and operating in only 5 of the 15 ports of entry scheduled to be operational by November 1, 2004. According to US-VISIT officials, this is largely due to delays in DHS granting security clearances to the civilian employees who would operate the equipment at the ports of entry. These changing facts and circumstances surrounding the pilot introduce additional risk concerning US-VISIT's delivery of promised capabilities and benefits on time and within budget.