Biosurveillance: Observations on BioWatch Generation-3 and Other Federal Efforts
Released: 2012-09-13T00:00:00

Letter

DHS and the White House Have Taken Action to Enhance Biosurveillance

In December 2009, we published a report assessing DHS’s efforts to establish the National Biosurveillance Integration Center (NBIC). We reported that NBIC was not fully equipped to carry out its mission because it lacked key resources—data and personnel—from its partner agencies, a situation that could be at least partially attributed to collaboration challenges NBIC faced. We recommended that NBIC work with its federal partners to develop a strategy to enhance collaboration— including sharing data, personnel, and other resources—and to establish effectiveness measures for that collaboration. DHS generally concurred with our findings and recommendations and stated that NBIC would work with its partners to develop a collaboration strategy to clarify both the mission space and roles and responsibilities for all partners. In August 2012, DHS issued the National Biosurveillance Integration Center Strategic Plan. According to DHS officials, the plan articulates a clear approach with a series of measurable steps and initiatives to enhance the nation’s biosurveillance capability. In late August 2012, when providing us with a copy of the strategy, officials stated that they believe it satisfies the intent of our recommendations. Officials said the plan was written in coordination with NBIC’s federal partners and is the result of a deliberative process examining NBIC’s current capabilities and capability gaps. We are currently assessing the extent to which the plan fully responds to the recommendations.

In June 2010, we reported on federal efforts that support a national biosurveillance capability and the extent to which mechanisms were in place to guide the development of a national biosurveillance capability. We reported that a national biosurveillance capability would largely rely on an interagency effort because the activities and accompanying resources that support the capability—personnel, training, equipment, and systems—are dispersed across a number of federal agencies.

However, we found that the federal government did not have a unifying framework and structure for integrating dispersed capabilities and responsibilities and no federal agency had authority to guide and oversee the development and implementation of a national effort that encompassed all stakeholders with biosurveillance responsibilities. We concluded that without such a framework and an entity with the authority, resources, time, and responsibility for guiding its implementation, it would be very difficult to create an integrated approach to building and sustaining a national biosurveillance capability. We recommended that the Homeland Security Council within the White House direct the National Security Staff to identify, in consultation with relevant federal agencies, a focal point to lead the development of such a strategy.

Our June 2010 report also noted that a national biosurveillance capability depends upon participation from state, local, and tribal governments, because few of the resources required to support the capability are wholly owned by the federal government. In October 2011, we reported on how the federal government worked with its nonfederal partners to support biosurveillance, activities those partners identified as essential to their biosurveillance efforts, and particular challenges those partners faced. We recommended that the strategy we called for in June 2010 incorporate a means to leverage existing efforts that support nonfederal biosurveillance capabilities, consider challenges that nonfederal jurisdictions face, and include a framework to develop a baseline and gap assessment of nonfederal jurisdictions’ biosurveillance capabilities. The White House did not comment on these recommendations.

In July 2012, the White House released the National Strategy for Biosurveillance to describe the U.S. government’s approach to strengthening biosurveillance. The strategy describes guiding principles, core functions, and enablers for strengthening biosurveillance. The strategy states that its approach emphasized teamwork between and within federal departments, across all layers of government, and with private sector partners. A strategic implementation plan is to be completed within 120 days of the strategy issuance. The strategy does not fully meet the intent of our June 2010 and October 2011 recommendations, as discussed later in this statement, but it is possible that it will when the implementation plan is complete.

DHS Did Not Develop Critical Knowledge before Proceeding with the Gen-3 Acquisition

DHS Proceeded with the Gen-3 Acquisition before Establishing a Mission Need

DHS approved the Gen-3 acquisition in October 2009 without fully developing critical knowledge that would help ensure sound investment decision making, pursuit of optimal solutions, and reliable performance, cost, and schedule information. Specifically, DHS did not engage the initial phase of its Acquisition Life-cycle Framework, which is designed to help ensure that the mission need driving the acquisition warrants investment of limited resources. In the Acquisition Life Cycle Framework design, it is not the purpose of the Mission Needs Statement to specify a technical solution. Rather it is to serve as a touchstone for subsequent acquisition efforts by focusing on the capability gap to help articulate and build consensus around the goals and objectives for a program.

However, DHS began to pursue a specific autonomous detection solution well before completing a Mission Needs Statement. Specifically, DHS’s Integrated Planning Guidance (IPG) for fiscal years 2010-2014, which was finalized in March 2008, included specific goals for the next generation of BioWatch—to deploy in all major cities an autonomous BioWatch detection device reducing the operating cost per site by more than 50 percent and warning time to less than 6 hours. The purpose of DHS’s IPG is to communicate the Secretary’s policy and planning goals to component-level decision makers to inform their programming, budgeting, and execution activities. As such, this specific set of goals for BioWatch Gen-3 demonstrates that DHS leadership had established a course for the acquisition by March 2008, in advance of efforts to define the mission need through the Mission Needs Statement process, which was finalized more than a year and a half later.

DHS officials in multiple departments described a climate, in the wake of the September 11, 2001, terrorist attacks and the subsequent Amerithrax attacks, in which the highest levels of the administration expressed interest in quickly deploying the early generation BioWatch detectors and improving their functionality—as quickly as possible—to allow for faster detection and an indoor capability. BioWatch officials stated that they were aware that the Mission Needs Statement prepared in October 2009 did not reflect a systematic effort to justify a capability need, but stated that the department directed them to proceed because there was already departmental consensus around the solution. Accordingly, the utility of the Mission Needs Statement as a foundation for subsequent acquisition efforts was limited.

DHS Did Not Systematically Analyze Alternatives

Additionally, DHS did not use the processes established by its Acquisition Life-cycle Framework to systematically ensure that it was pursuing the optimal solution—based on cost, benefit, and risk—to mitigate the capability gap identified in the Mission Needs Statement. The DHS Acquisition Life-cycle Framework calls for the program office to develop an Analysis of Alternatives that systematically identifies possible alternative solutions that could satisfy the identified need, considers cost- benefit and risk information for each alternative, and finally selects the best option from among the alternatives.

However, the Analysis of Alternatives prepared for the Gen-3 acquisition did not reflect a systematic decision-making process. For example, in addition to—or perhaps reflecting—its origin in the predetermined solution from the Mission Needs Statement, the Analysis of Alternatives did not fully explore costs or consider benefits and risk information as part of the analysis. Instead, the Analysis of Alternatives focused on just one cost metric that justified the decision to pursue autonomous detection—cost per detection cycle—to the exclusion of other cost and benefit considerations that might have informed decision makers.the Analysis of Alternatives examined only two alternatives, though the Additionally, guidance calls for at least three. The first alternative was the currently deployed Gen-2 technology with a modified operational model (which by definition was unable to meet the established goals). The second alternative was the complete replacement of the deployed Gen-2 program with an autonomous detection technology and expanded deployment.

BioWatch program officials acknowledged that other options—including but not limited to deploying some combination of both technologies, based on risk and logistical considerations—may be more cost-effective. As with the Mission Needs Statement, program officials told us that they were advised that a comprehensive Analysis of Alternatives would not be necessary because there was already departmental consensus that autonomous detection was the optimal solution.

Because the Gen-3 Analysis of Alternatives did not evaluate a complete solution set, did not consider complete cost information, did not consider benefits, and did not include a cost-benefit analysis, it does not provide information on which to base trade-off decisions. For example, it does not provide information about the extent to which various aspects of the solution—such as the number of participating jurisdictions—results in a reduction of risk and at what cost. Given the uncertainty related to Gen- 3’s costs, benefits, and risk mitigation potential, DHS does not have reasonable assurance that the strategy of expanding and completely replacing the existing Gen-2 program with autonomous detection technology is the most cost-effective solution.

DHS Did Not Fully Develop Performance, Cost, and Schedule Information

In October 2009, DHS approved the Gen-3 acquisition at Acquisition Decision Event (ADE) 2A—one of the key formal decision points in DHS’s Acquisition Life-cycle Framework—based on information contained in acquisition documents provided by the BioWatch program. One critical purpose of the ADE-2A documentation set required by DHS’s acquisition guidance is to describe the expected performance, cost, and schedule parameters for an acquisition. However, the ADE-2A Acquisition Decision Memorandum stated that significant data necessary for the proper adjudication of an ADE-2A decision were missing. Further, we reported that some performance, cost, and schedule expectations presented at ADE-2A were not developed in accordance with DHS guidance and good acquisition practices—like accounting for risk in schedule and cost estimates.

On the basis of the Gen-3 documentation submitted at ADE-2A, DHS expected to acquire a system that would cost $2.1 billion, be fully deployed by fiscal year 2016, and meet certain performance requirements. However, the performance, cost, and schedule parameters for the Gen-3 acquisition have changed. Specifically, certain performance requirements have been revised, the estimated date for full deployment has been delayed from fiscal year 2016 to fiscal year 2022, and the expected life cycle cost has changed from the $2.1 billion point estimate prepared for ADE-2A to a risk-adjusted $5.8 billion estimate, calculated at the 80 percent confidence level.

BioWatch program officials told us that they had to prepare ADE-2A documentation quickly because ADE-2A had been accelerated by more than a year. Additionally, DHS officials from multiple offices described a climate around the time of ADE-2A in which the department’s business processes—including acquisition practices—were maturing and thus were less rigorous in their adherence to best practices for cost and schedule estimating. However, in the absence of complete and reliable information, DHS had limited assurance that the acquisition would successfully deliver the intended capability within cost and on schedule. Comprehensive and systematic information developed using good practices for cost and schedule estimating could help ensure that more reliable performance, cost, and schedule information is available for future acquisition decision making.

We recommended that before continuing the acquisition, DHS reevaluate the mission need and alternatives and develop performance, cost, and schedule information in accordance with guidance and good acquisition practices. DHS concurred with the recommendations but plans to proceed with the next step in the acquisition—performance testing—while implementing them. We are pleased that DHS plans to implement the recommendation but are concerned by DHS’s intention to continue the acquisition efforts before ensuring that it has fully developed the critical knowledge a comprehensive Acquisition Life-cycle Framework effort is designed to provide.

Several Steps Remain before Gen-3 Is Ready for Deployment

The BioWatch program completed initial testing and evaluation on a Gen- 3 prototype technology in June 2011, but several steps remain before For example, the BioWatch Gen-3 can be deployed and operational.program must complete additional testing. The characterization testing conducted in 2010 and 2011 was intended to assess the state of available technology. This testing sought to demonstrate the performance of available candidate Gen-3 technologies against the requirements established by the BioWatch program, and consisted primarily of laboratory testing of individual system components. This testing did not demonstrate the performance of the full system in detecting live pathogens in the operational environment. It also did not test the information technology network that will transmit results for public health officials. Now the program plans to conduct the next phase of testing— performance testing in three independent laboratories and operational test and evaluation in four BioWatch jurisdictions. On the basis of the June 2011 Life-cycle Cost Estimate, the BioWatch program estimates this testing will take approximately 3 years and cost approximately $89 million (risk adjusted at the 80 percent confidence level).

The Deputy Secretary of Homeland Security and other senior officials met on August 16, 2012 for an Acquisition Review Board, during which the BioWatch program was seeking approval to initiate the next phase of the acquisition. DHS did not make a final decision, but authorized release of a solicitation for performance testing under the next testing phase. In response to the recommendations we made in the Gen-3 report, DHS officials stated that before awarding a performance testing contract— which would allow the program to acquire a small number of test units— the program office is directed to return to the Acquisition Review Board for approval.

Before undertaking the remaining steps in the acquisition, the program office is directed to return for Acquisition Decision Event-2B (ADE-2B)— the next formal decision point in DHS’s Acquisition Life-cycle Framework—with updated information, including an Analysis of Alternatives and Concept of Operations, as we recommended. No timeframe for completing these actions has been specified, but according to DHS officials, it may take up to 1 year to update the Analysis of Alternatives. In preparation for the August 16, 2012, meeting, the BioWatch program had updated key acquisition documents—including the Life-cycle Cost Estimate and Acquisition Program Baseline—as required by the Acquisition Decision Authority in a February 2012 memo. However, in order to inform the ADE-2B decision, these documents must accurately reflect changes to Gen-3 performance requirements and updated cost and schedule estimates for the acquisition and therefore may require further revisions.

If approved at ADE-2B, the BioWatch program plans to conduct operational testing of Gen-3 units in four BioWatch jurisdictions. Following operational testing, DHS intends to decide whether to authorize the production and deployment of Gen-3. If Gen-3 is approved, the BioWatch program plans to prepare for deployment by working with BioWatch jurisdictions to develop location-specific plans to guide Gen-3 operations. DHS estimates based on the June 2011 Life-cycle Cost estimate show that about $5.7 billion of the $5.8 billion life-cycle cost (risk adjusted at the 80 percent confidence level) remains to be spent to test, produce, deploy, and operate Gen-3 through fiscal year 2028.

Observations about Prior Strategy Recommendations and the July 2012 National Strategy for Biosurveillance

In the report on Gen-3 released today, we noted that beyond the uncertainty related to the costs and benefits of the planned Gen-3 approach, there is additional uncertainty about the incremental benefit of this kind of environmental monitoring as a risk mitigation activity because of its relatively limited scope. As the study committee for a 2011 National Academies evaluation of BioWatch noted, there is considerable uncertainty about the likelihood and magnitude of a biological attack, and how the risk of a release of an aerosolized pathogen compares with risks from other potential forms of terrorism or from natural diseases. The National Academies report also notes that while the BioWatch program is designed to detect certain biological agents (currently five agents) that could be intentionally released in aerosolized form, detecting a bioterrorism event involving other pathogens or routes of exposure requires other approaches.

In the report we released today, we stated that given the total estimated operating cost for the Gen-3 program, it is important, especially in an increasingly resource-constrained environment, to consider the benefit— in terms of its ability to mitigate the consequences of a potentially catastrophic biological attack—that the investment provides. We noted that the scope limitations of this kind of environmental monitoring provide context in both the consideration of mission need and in analyzing cost effectiveness.

However, it was not within the scope of our BioWatch Gen-3 study nor was it our intention to reach a firm conclusion about the value of this kind of activity as part of a layered biosurveillance strategy. Rather, we believe the need to consider value within the larger biosurveillance enterprise as part of an effort to define mission need for a single federal program like Gen-3 provides a timely and concrete illustration of the kind of issues we sought to address with our June 2010 recommendation. The recommendation for the Homeland Security Council to direct the National Security Staff to identify a focal point to lead the development of a national biosurveillance strategy was grounded in previous work on desirable strategy characteristics for complex homeland security missions. We recognized the difficulty that decision makers and program managers in individual federal agencies face prioritizing resources to help ensure a coherent effort across a vast and dispersed interagency, intergovernmental, and intersectoral network. Therefore, we called for a strategy that would, among other things, (1) define the scope and purpose of a national capability; (2) provide goals, objectives and activities, priorities, milestones, and performance measures; and (3) assess the costs and benefits and identify resource and investment needs, including investment priorities.

We stated that one of the aims of a national biosurveillance strategy should be to help prioritize where resources and investments should be targeted and guide agencies to allocate resources accordingly. Further, we reported that a national strategy could begin to address the difficult but critical issues of who pays and how funding for biosurveillance will be sustained in the future. Finally, we noted that in an environment with competing priorities, a strategy could help address situations where investments must be carefully weighed and sound judgments made about the most cost-effective approaches, but doing so would require information about the cost, benefits, and risks associated with the whole biosurveillance enterprise.

The National Strategy for Biosurveillance includes four guiding principles that are designed to serve as a foundation for enterprisewide efforts, four core functions that are designed to promote a deliberate and shared approach, and four enabling capabilities that are designed to represent areas for ongoing focus. These planks of the strategy align with our call for a strategy that would help to clarify the scope and purpose of a national biosurveillance capability and the goals of that capability. Our June 2010 report described several categories of federal efforts to improve the personnel, training, and systems and equipment that support a national capability. These included responding to workforce needs, facilitating information sharing, and applying technologies to enhance surveillance. Among the planks of the National Strategy for Biosurveillance, it is possible to discern support for each these categories. For example, the enabling capability called build capacity, discusses both workforce and information sharing issues. The four guiding principles that serve as the strategy’s foundation encourage broad-based and cross-cutting actions to leverage constrained resources, responding, in part, to our call for the strategy to help identify the resources currently being used, additional resources that may be needed, and opportunities for leveraging resources.

However, the strategy does not yet offer a mechanism to identify resource and investment needs, including investment priorities among these various efforts. Accordingly, the enterprise is still without a framework to guide the systematic identification of risk, assessment of resources needed to address those risks, and the prioritization and allocation of investment across the entire biosurveillance enterprise, as we recommended in June 2010. For example, in the case of the broader contextual information needed to inform the BioWatch Gen-3 mission need, the strategy has language indicating that advances in science and technology are a priority. In fact, the capability enabler called fostering innovation specifically calls for science and technology capabilities, including new detection approaches. However, the strategy does not facilitate analysis or provide tools to assess the risks to be addressed—in the context of enterprisewide goals—by such science and technology approaches or the value they should offer the enterprise relative to their costs. Without such a framework and tool set, it remains difficult for decision makers—in both the executive and legislative branches—to help ensure that their resource allocation decisions contribute to a coherent enterprisewide approach.

We are encouraged by the National Strategy for Biosurveillance and the work the White House has done to date to provide a platform for achieving a well-integrated national biosurveillance enterprise. We are hopeful that the forthcoming strategic implementation plan which promises to include specific actions and activity scope, designated roles and responsibilities, and a mechanism for evaluating progress will help to address the ongoing need for mechanisms to help prioritize resource allocation.

Chairmen Bilirakis and Lungren, this concludes my prepared statement. I would be happy to respond to any questions you or the other committee members may have.

Contact Information

For further information on this testimony, please contact Bill Jenkins, (202) 512-8757 or by e-mail at jenkinswo@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Other contributors include; Hal Brumm, Nirmal Chaudhary, Michelle Cooper, Edward George, Kathryn Godfrey, Allyson Goldstein, Tracey King, Amanda Miller, Jan Montgomery, Katy Trenholme, and Katherine Trimble.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.