Biosurveillance: Developing a Collaboration Strategy Is Essential to Fostering Interagency Data and Resource Sharing
Released: 2009-12-18T00:00:00

Letter

Background

We conducted this performance audit from February 2008 through November 2009, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

NBIC Is Responsible for Facilitating Collaboration across Multiple Agencies to Enhance Biosurveillance Effectiveness by Integrating Data and Expert Analysis

Biosurveillance is the process of gathering, analyzing, and interpreting data in order to achieve early detection and warning and overall situational awareness of biological events with the potential to have catastrophic human and economic consequences. In August 2007, the 9/11 Commission Act established NBIC to contribute to the nation’s biosurveillance capability by enhancing the ability of the federal government to rapidly identify, characterize, localize, and track biological events of national concern through integration and analysis of data relating to human health, animal, plant, food, and environmental monitoring systems (both national and international). Once a potential event is detected, NBIC is to disseminate alerts to enable response to a biological event of national concern. To achieve these objectives, NBIC is to coordinate with federal and other stakeholders that have information that can be used to enhance the safety and security of the United States against potential biological events of national significance. This community of federal stakeholders is known as the NBIS.

The NBIS community predated the enactment of the 9/11 Commission Act. Beginning in 2004, DHS managed the NBIS and developed an IT system to manage other agencies’ biosurveillance information, an effort that was moved among several DHS Directorates, including DHS’s Science and Technology Directorate and the Preparedness Directorate. In 2007, DHS created the Office of Health Affairs, headed by the DHS Chief Medical Officer, to lead DHS’s biodefense activities and provide timely incident- specific management guidance for the medical consequences of disasters. At that time, DHS placed NBIS in the Office of Health Affairs. Shortly after that, the 9/11 Commission Act created NBIC and gave it responsibility for managing the NBIS, which has remained in the Office of Health Affairs. Since fiscal year 2008, NBIC has operated with an annual budget of $8 million dollars. Biosurveillance activities at NBIC are carried out by its Operations Division, which is headed by the Deputy Director and Chief Scientist and supported by 10 contract employees that serve as the analytic core for NBIC’s daily operations. These staff members have various backgrounds related to biodefense, including public health, veterinary, environmental, and intelligence training.

NBIC Draws Its Early Detection and Situational Awareness Missions from Multiple Presidential Directives, in Addition to the 9/11 Commission Act

As shown in table 1, the 9/11 Commission Act outlines certain requirements for NBIC and NBIS member agencies, and most of these relate to how NBIC is to coordinate NBIS member agency data and information management resources. Generally, there are four elements that are critical for NBIC to achieve its early detection and situational awareness missions established in the 9/11 Commission Act: (1) acquire data from NBIS partners that can be analyzed for indications of new or ongoing biological events, (2) leverage scientific and event-specific expertise from across the NBIS, (3) obtain strategic and operational guidance from NBIS partners, and (4) develop and maintain information technologies to support data collection, analysis, and communication.

Although the act does not specify any member agency that must participate in the NBIS, it defines a member agency as any agency that signifies agreement to participate by signing a memorandum of understanding (MOU) and establishes for them specific requirements— generally related to sharing information and human assets. For example, as shown in table 1, the act provides that each member agency shall use its best efforts to integrate biosurveillance information into the NBIC, with the goal of promoting information sharing between federal, state, local, and tribal governments to detect biological events of national concern. NBIC has identified 11 NBIS partners at the federal level—the Departments of Health and Human Services (HHS), Agriculture (USDA), Commerce, Defense, Interior, Justice, State, Transportation, and Veterans Affairs, as well as the Environmental Protection Agency and the United States Postal Service. In some departments, more than one component has been identified for participation. Some of these departments, such as HHS and USDA, have major mission responsibilities for collecting health data that may indicate an outbreak of a disease or other biological event. Other departments may collect data or have subject matter expertise that may be used during the course of a biological event. For example, the National Oceanic and Atmospheric Agency within the Department of Commerce collects meteorological data that may be used by NBIC to help inform the progression of an outbreak based on weather patterns.

Around the same time as the enactment of the 9/11 Commission Act, the President issued Homeland Security Presidential Directive-21 (HSPD-21), as a high-level biodefense strategy. HSPD-21 is built on the principles of earlier directives—HSPD-9 and HSPD-10—which collectively describe the role of the federal government in building a national capability to detect a biological event. For example, HSPD-21 lays out goals for addressing each of four biodefense elements for human health, one of which is surveillance. In this respect, HSPD-21 calls for the United States to develop a nationwide, robust, and integrated biosurveillance capability to provide early warning and ongoing characterization of disease outbreaks in near real-time. Consistent with this goal, HSPD-21 directs the Secretary of Health and Human Services to establish a national epidemiologic surveillance system for human health, in part, to integrate federal, state, and local data into a national biosurveillance common operating picture. Although HSPD-21 does not specify a role for DHS in biosurveillance, the earlier directives did, and creation and maintenance of an electronic biosurveillance common operating picture has been an NBIS goal since its inception.

Biosurveillance Involves Data Gathering and Analysis at Multiple Levels of Government for the Purposes of Earlier Warning and Enhanced Situational Awareness

The data needed to detect an infectious disease outbreak or bioterrorism may come from a variety of sources, and aggregating and integrating data across multiple sources is intended to help recognize the nature of a disease event or understand its scope. Combining and comparing data streams from different sectors to detect or interpret indications of a potential health emergency is called biosurveillance integration. Both HSPD-21 and the 9/11 Commission Act seek enhanced integration of disparate systems and programs that collect data with the aim of providing early warning and ongoing characterization of biological events. HSPD-21 and the 9/11 Commission Act each also seek to enhance the situational awareness for the detection of and response to biological events. Much of the information gathered for these biosurveillance purposes is generated at the state government level. For example, state health departments collect and analyze data on notifiable diseases submitted by health care providers and others. In addition, state-run laboratories conduct testing of samples for clinical diagnosis and participate in special clinical or epidemiologic studies. Finally, state public health departments verify cases of notifiable diseases, monitor disease incidence, and identify possible outbreaks within their states. At the federal level, agencies and departments generally collect and analyze surveillance data gathered from the states and from international sources, although some federal agencies and departments also support their own national surveillance systems and laboratory networks.

When an issue crosses federal agency lines, as biosurveillance integration does, the agencies involved must collaborate to deliver results more efficiently and effectively. Due to NBIC’s role as an integrator of information across the biosurveillance community, it is important for NBIC to ensure that it effectively collaborates with the NBIS to obtain the cooperation of this interagency community. One reason that it is important that NBIC effectively collaborate with federal partners is that agencies are not required by law to support NBIC or participate in the NBIS community. We have previously reported that for collaborating agencies to enhance and sustain collaboration, they need to, among other things, (1) have a clear and compelling rationale for working together; (2) establish joint strategies, policies, and procedures for aligning core processes and resources; (3) identify resources needed to initiate or sustain their collaborative effort; (4) work together to define and agree on their respective roles and responsibilities; and (5) develop accountability mechanisms to help implement and monitor their efforts to achieve collaborative results.

NBIC Has Undertaken Efforts to Coordinate the NBIS and Acquire Resources, but Lacks Key Mission-Critical Elements

NBIC has made some efforts to put mission-critical elements in place, such as requesting data from other federal partners, initiating relationship- building activities among NBIC analysts and subject matter experts at other agencies, and establishing governance bodies to oversee and guide the NBIS. However, NBIC currently relies on publicly available data because it receives limited data from NBIS partners and generally lacks assignments of personnel from other agencies to leverage analytical expertise from across the NBIS partners.

NBIC Generally Has Not Acquired Data from Other Agencies to Support the Early Detection Mission and Instead Relies on Nonfederal, Open-Source Data

NBIC’s ability to acquire and consolidate data from NBIS partners as well as from nonfederal sources is central to achieving its mission. Current and initial drafts of the NBIS Concept of Operations reinforce this notion, noting that the identification of relevant and timely data sources, which act in combination to provide actionable information for decisionmaking, is essential to accomplishing early detection. NBIC has taken some action to acquire these types of data from NBIS partners, for instance, by requesting that NBIS partners identify the types of data they collect or generate that might aid in NBIC’s early detection mission. However, as of October 2009, NBIC was generally not receiving the types of data best suited to early detection of biological events of national concern. NBIC officials acknowledge that they lack key data, and NBIC and other NBIS member officials described numerous challenges to sharing such information, including but not limited to scant availability of such data throughout the federal government and concerns about trust and control over sensitive information before it is vetted and verified.

Based on our discussions with NBIS agency officials and review of NBIC documents, we have defined and verified with NBIC officials three categories of electronic data that are critical for NBIC to achieve its mission and might be available from federal agencies or other sources. As described in table 2, these data categories are (1) raw structured data, (2) raw unstructured data, and (3) final products which are typically briefings produced by other agencies in the course of monitoring routine and emerging disease. As of October 2009, NBIC was receiving some final products from NBIS partners, but was not receiving any raw data— particularly data that are generated at the earliest stages of a biological event.

According to officials, receipt of all three types of electronic data is important to help NBIC achieve its mission of detecting and warning of a biological event because detection of events that are novel, from multiple sources, or widespread requires analysis of multiple independent data streams. However, the officials told us that they do not receive from NBIS partners the raw structured or unstructured data that best support the early detection goal of biosurveillance. In particular, NBIC identified data that are generated at the earliest stages of a biological event—which can include raw data collected by federal agencies as part of their biosurveillance responsibilities—as being among the highest value for enabling the earlier detection of biological events of national concern. For instance, structured data, such as medical codes corresponding to diagnoses that are entered into databases, as well as some sources of unstructured data, such as written observations noted on medical forms, are generated at the earliest stages of a biological event and have been identified by NBIC as a high priority for early detection. These data can be collected or generated by federal agencies with responsibilities for biosurveillance and which are participating in NBIS. For example, HHS has developed a surveillance system that collects data on symptoms of patients entering emergency departments, that when analyzed with statistical tools, may be able to indicate the presence of an outbreak in less time than it takes to perform diagnostic lab tests.

NBIC seeks to finalize three types of agreements with NBIS partners to articulate and establish protocols and legal authority for resource sharing: (1) MOUs, (2) interagency agreements (IAA), and (3) interagency security agreements (ISA). To date, 7 of the 11 agencies have signed MOUs, but only 1 has a finalized ISA in place for data sharing, according to NBIC officials. As of October 2009, the federal agency that signed an ISA agreed to provide a single data source related to food safety. NBIC officials told us that although the agreement and the technology allowing the electronic data exchange are in place, the agency has not yet begun transferring the data to NBIC, and they did not know when to expect the transfer to begin. NBIC’s inability to finalize agreements can be attributed in part to challenges it faces in ensuring effective collaboration, which will be discussed later in this report.

Five NBIS partners provide NBIC with written final products, such as briefings produced on a routine basis that provide information on outbreaks of diseases or special alerts of potentially dangerous biological events issued as needed. However, NBIC officials noted that there are limitations on the value of final reports for supporting early detection. These finished products represent the agency’s final analysis and interpretation of the raw data that it collects and have been reviewed and approved by the agency leadership for general dissemination to interested parties. According to NBIC officials, these products are generally useful for providing context but not for early detection of a biological event because they are not generated in a timely enough fashion to be valuable for detecting new biological events and focus on biological events that have already been detected.

In the absence of proprietary information from NBIS partners, NBIC relies on mostly nonfederal sources of data, such as media reports of illness, to attempt to identify biological events. The bulk of data—according to NBIC officials more than 98 percent—NBIC currently uses to pursue its mission is unstructured and comes from nonfederal, open sources, including an international information gathering service called Global Argus, a federally-funded program in partnership with Georgetown University. The service searches and filters over 13,000 overseas media sources, in more than 34 languages. The practice of monitoring and translating local news articles has the potential to provide information about undiagnosed and other suspicious disease activity before it is reported through more official channels. NBIC officials stated that continuous monitoring of global news media sources and publicly available Web sites would be important to round out potential gaps in coverage, even if other data are available from federal agencies.

NBIC Has Had Limited Success Obtaining Expertise from NBIS Partners through Interagency Personnel Assignments

NBIC officials told us that regardless of the quantity and quality of data types shared by collaborating agencies, effective biosurveillance depends on human analysts to interpret events and place them in context. For example, determining whether an outbreak of a new emerging infectious disease has occurred and further assessing whether this event is one of national concern are analytic judgments that require not only data but also the expertise of an experienced, knowledgeable analyst. According to these officials, analyst-to-analyst communication in a trusted environment is absolutely essential for rapid vetting, verification, and contextualization of events.

The 9/11 Commission Act calls for member agencies to provide personnel to NBIC under an interagency personnel agreement and consider the qualifications of such personnel necessary to provide human, animal, and environmental data analysis and interpretation support. However, for the most part, NBIC has not consistently received this kind of support from NBIS partners. Personnel detailed (that is, personnel employed by a federal agency and temporarily assigned to NBIC for a specified period of time) from other federal agencies enable analysis and interpretation of data by serving as subject matter experts for specific issues that are part of their home agencies’ missions and as conduits of information from their respective home agencies. NBIC has signed MOUs with seven agencies, but only two have provided a personnel detail to the NBIC headquarters in Washington, D.C., and as of October 2009, only one of those personnel details was active, because one of those agencies did not replace personnel after the initial detail ended. NBIC officials told us that daily interaction with officials who had been on detail at NBIC not only enhanced their ability to interpret the information immediately on hand but also contributed to ongoing contextual learning for NBIC’s analytical corps.

Although most of the NBIS partners have not detailed their subject matter experts to NBIC, the integration center officials have used other means to obtain expertise and information from other agency analysts. NBIC officials told us that they have co-located the NBIC analysts at other collaborating agencies where they spend up to 2 weeks working with analysts from these other agencies both to learn more about their operations and to help forge ongoing relationships. NBIC officials stated they have also established a daily process to engage the NBIS in sharing information and analytic insights with each other. During this process— which NBIC calls the daily production process—NBIC analysts compile information on reports of outbreaks that may be of concern, and then this information is disseminated to the NBIS community for discussion at a daily teleconference. The participants in the teleconference determine whether the events merit further monitoring or evaluation and share any relevant information they may have about the event. NBIC analysts then use the information gathered, as refined by the daily teleconference, to finalize NBIC daily reports and update its electronic Biosurveillance Common Operating Picture, which is a manually updated electronic picture of current worldwide biological events being tracked.

For example, NBIC analysts might identify local news reports that suggest food contamination in a region. During the daily conference call, one or more of the agencies with responsibility for monitoring food safety or foodborne illness might contribute more information, such as a history of similar issues in the same geographical region, that gives more context to the reports. Then, collectively, the responsible agencies might decide that the event, first uncovered in open source media, warrants further investigation and monitoring. NBIC analysts would then post all known information to its electronic Biosurveillance Common Operating Picture for all interested parties to follow. Meanwhile, the agencies with missions of jurisdiction would conduct their investigations and report any new findings during the following day’s teleconference. NBIC officials told us that this process requires a wide range of expertise from across the agencies. These officials said that they may also communicate directly with an agency prior to the daily teleconference if NBIC plans to discuss an item relevant to the agency’s mission at the meeting.

Another means NBIC uses to obtain expertise and information from other agency analysts is through participation in the Biosurveillance Indications and Warnings Analytic Community (BIWAC). The BIWAC is a self- governing interagency body composed of federal officials who are actively responsible for pursuing a biosurveillance mission. The agencies represented include: the Department of Defense, HHS’s Centers for Disease Control and Prevention, USDA, DHS, and the intelligence community. The mission of the BIWAC is to provide a secure, interagency forum for the collaborative exchange of critical information regarding biological events that may threaten U.S. national interests. On behalf of the BIWAC, the Department of Defense’s National Center for Medical Intelligence hosts an encrypted information sharing portal called Wildfire. According to NBIC’s Chief Scientist and Deputy Director, in addition to engaging in the information exchange through Wildfire, she is an active supporter and participant in BIWAC meetings and teleconferences.

According to NBIC officials, although these efforts to obtain the analytical insights of subject matter experts from collaborating agencies may be valuable, they do not provide a substitute for personnel details to the integration center itself. For example, with the daily teleconference, NBIC may have limited access to NBIS agency subject matter experts because analysts from only a few of the various agencies may be available for immediate communication on any given day, and not all agencies regularly participate in the daily teleconference. In addition, apart from the daily teleconference, NBIC officials said that agencies may limit NBIC’s ability to communicate with their subject matter experts, particularly in the early stages of responding to a biological event when the agency is prioritizing its response needs. Finally, NBIC analysts may also communicate through federal agencies’ operations centers during the course of an ongoing biological event, but NBIC officials noted that this channel of communication is not always an effective means to get meaningful input from agencies’ subject matter experts. The lack of sustained personnel detailed to NBIC from other NBIS partner agencies can be attributed, in part, to challenges it faces with ensuring effective interagency collaboration, which will be discussed later in this report.

NBIC Established Governance Structures to Develop and Oversee the NBIS

In order to support the ability for NBIS partners to engage in overseeing and guiding the NBIS, NBIC has established and administers two governance bodies. NBIC sponsors meetings of the two groups on a regular basis. The NBIS Interagency Oversight Council (NIOC) is composed of representatives at the assistant secretary level from each NBIS agency. The NIOC is to act as the senior oversight body to provide guidance and direction for the operation, implementation, and maintenance of the NBIS, as well as to resolve interagency or intradepartmental issues that cannot be resolved at lower levels. The NBIS Interagency Working Group (NIWG) is a senior, director-level working body created to share information on NBIC activities, such as the status of developing draft documents and standard operating procedures including procedures undertaken during ongoing biological events of national concern. The NIWG can also establish sub-working groups to conduct specific work as necessary to provide support to the NBIC and the NIOC. For example, NIWG established a sub-working group to propose procedures for resolving conflict during the daily production cycle.

NBIC Uses an IT System to Manage Publicly Available Data and to Communicate Alerts, but Generally Lacks the Ability to Apply Analytical Tools to Data

One of the elements that is critical for NBIC to carry out its mission is development and maintenance of information technologies to support data collection, analysis, and communication of alerts. The 9/11 Commission Act also specifically mentions the need for statistical tools to analyze data to identify and characterize trends of biological events of national concern. NBIC has taken steps to develop an IT system that can manage data from NBIS partners and can help identify open source reports of potential biological events, but NBIC largely lacks data from federal agencies. Given this condition, rather than a system designed to electronically process structured data received directly from NBIS partners, NBIC has configured its IT system—the Biosurveillance Common Operating Network (BCON)—primarily to identify and assemble unstructured data from public sources on the Internet that it will later vet with other NBIS analysts in the daily production process. Therefore, NBIC relies on the NBIS community and member agency subject matter experts for analysis and interpretation of publicly available data rather than providing the NBIS community with an analysis of integrated, raw, structured data from the NBIS partners. According to NBIC officials, they anticipate using BCON to manage any agency data streams that they may eventually acquire.

BCON is a system of systems that is built on multiple commercial-off-the- shelf software packages. Currently, the central feature of BCON is its use of a set of keywords within a language algorithm to search the Internet for media articles that may contain biosurveillance-relevant information and compile them for NBIC analysts to review. As part of this function, BCON also flags events for immediate analyst attention. Additionally, the information from BCON is the basis for the NBIC Biosurveillance Common Operating Picture, which is a manually updated Google Maps application of current worldwide biological events being tracked. NBIS agency officials can view the Biosurveillance Common Operating Picture on the Homeland Security Information Network. According to NBIC officials, in the future NBIS agency officials will also have the ability to create and update event information.

Although NBIC generally lacks direct-feed, raw, structured data from NBIS partners to apply statistical and analytical tools, according to our observations and review of documents supporting the development of the system, BCON is designed to locate and log information associated with the events contained in the open source media that it searches. This information includes the geographic coordinates and the date and time of occurrence for each event. This data is archived and, according to NBIC officials, can be used to conduct cross-domain analysis for trends, historical context, associated events, anomaly detection, and hypothesis generation. Among the applications planned for inclusion in BCON is a tool that is designed to perform historical analysis of this archived data to help monitor and refine the effectiveness of the algorithm. According to NBIC officials, the goal of this analysis is to help ensure that NBIC analysts will be able to identify events that merit attention by refining the algorithm to limit results that are less relevant for monitoring for biological events of national concern. However, these officials told us that this aspect of BCON has been put on hold due to budget constraints.

To advance information sharing among federal agencies, NBIC is also pursuing $90 million dollars in supplemental funding for a broader information sharing initiative. This initiative is intended to enable greater information sharing capabilities among federal, state, and local agencies and to have the necessary data security to house classified data. According to NBIC officials, this initiative is being led by the National Security Council.

To communicate alerts to member agencies and the larger NBIS community regarding any incident that could develop into a biological event of national concern, NBIC has developed an IT system to provide alerts and warnings, based on an existing system that had been developed for another DHS component. However, according to NBIC officials, the system has not yet been fully implemented because they recently acquired it, and NBIC is still testing protocols for using it. According to our observations of the system and review of operational protocols, the system provides NBIC with the capability to tailor alerts and warnings to specific recipients via distribution lists. These officials said that in spring 2008 the protocols were approved by the NIWG and briefed to the NIOC. NBIC officials said they are currently testing the protocols but have not yet needed to employ the system during a biological event.

Use of Key Collaboration Practices Could Help NBIC Strengthen Collaboration and Promote Fuller Participation

Our analysis and interviews with NBIS partners suggest that NBIC could strengthen its use of collaborative practices. Because participation in the NBIS is voluntary, effective use of collaborative practices is essential to NBIC’s ability to successfully develop and oversee the NBIS in a way that enhances federal biosurveillance capabilities. However, we found (1) widespread uncertainty and skepticism around the value of participating in the NBIS and the purpose of NBIC; (2) incomplete joint strategies, policies, and procedures for operating across agency boundaries; (3) an inability or unwillingness of NBIS members to respond to plans for leveraging resources; (4) confusion and dissatisfaction around the definitions of mission, roles, and responsibilities of NBIC and its NBIS partners; and (5) a lack of mechanisms to monitor and account for collaborative results.

Clearly Defining the Mission and Purpose, the Value of Participation, and Joint Strategies and Procedures Could Promote More Effective Interagency Cooperation

Biosurveillance integration is an inherently interagency enterprise, requiring expertise and resources from various federal agencies, such as information on human and zoonotic diseases monitored by HHS and USDA. Indeed, NBIC officials acknowledged that NBIC cannot provide national-level capability for cross-domain biosurveillance relying solely on DHS resources. As a result, it is crucial for NBIC to ensure stakeholder buy-in and participation in clearly defining the value of NBIS participation and NBIC’s mission or purpose, as well as establishing the strategies and procedures for how the partners will work together. Our prior work states that effective collaboration requires agencies to have a clear and compelling rationale for working together, which can be achieved by defining and articulating a common federal outcome or purpose. The rationale can be imposed externally through legislation or other directives or can come from the agencies’ own perceptions of the value of working together. In either case, agency staff can accomplish this by working across agency lines to define and articulate the common purpose they are seeking to achieve that is consistent with their respective agency goals and mission. Because there is no legal requirement for agencies to participate in NBIS, agencies must have a clear and compelling rationale to work together as a community of federal partners by joining the NBIS and providing data and personnel to the integration center. In the case of an agency like NBIC, for which collaboration is essential, clearly defining and communicating its purpose and mission can help to ensure that partners share a vision of the desired outcomes. In addition, our work has shown that to enhance and sustain collaboration, it is important to establish joint strategies, policies, and procedures for operating across agency boundaries. Establishing joint strategies and compatible policies and procedures helps align collaborating agencies’ activities, processes, and resources to, among other things, bring together diverse organizational cultures to enable a cohesive working relationship across agency boundaries and create the mutual trust required to sustain the collaborative effort.

However, we found in interviews with agency officials from 14 components of the 11 NBIS partners, widespread uncertainty and skepticism around the value of and rationale for participation in the NBIS and incomplete strategies, policies, and procedures for operating across agency boundaries that lack key stakeholder buy-in. Twelve of the 14 NBIS-partner components expressed uncertainty about the value of participating in the NBIS community or confusion about the purpose of NBIC. For example, officials from one component stated that they were uncertain whether sharing resources with the integration center, something that is required of members of the NBIS community, would further their agency’s missions. Officials from another component expressed concerns about the rationale for participating in the NBIS and supporting the integration center, stating they were unsure whether NBIC contributed anything to the federal biosurveillance community that other agencies were not already accomplishing in the course of carrying out their biosurveillance-relevant missions. Officials from five of these components noted that their uncertainty about the value of participation in the NBIS was a factor in not assigning personnel to NBIC. Further, officials from 7 of the 14 components we interviewed indicated that their experience with a recent tabletop exercise and real life events had contributed to their concerns about the value of participating in NBIS and the purpose of NBIC. For example, officials from one component said that the tabletop exercise showcased agencies’ reluctance to share information and underscored that there was no role for NBIC; while officials from another component said that during 2009 H1N1 activities, NBIC was not able to demonstrate that it had unique value to add. Officials from seven of the components indicated that they lacked a concrete understanding of the purpose for which NBIC was requesting their agencies’ data, which was, in part, the reason they had not been able to identify appropriate data sources or to work out data sharing agreements with NBIC.

NBIC officials told us that they regularly reminded NBIS partners of NBIC’s mission as the coordinator of the NBIS and the value of sharing data and personnel to achieve the goal of earlier detection and enhanced situational awareness. However, officials from 8 of the 14 components told us that during negotiations with NBIC, they had raised concerns about the purpose of the data or the value of detailing personnel to NBIC, and NBIC had not followed up in a timely and consistent manner to resolve those concerns. NBIC officials also stated that they have taken actions to demonstrate the value of participating in NBIS and of sharing resources with the integration center. For example, NBIC co-located the integration center’s analysts with analysts at other agencies, such as the Centers for Disease Control and Prevention, for brief periods of time to enhance mutual understanding between NBIC and NBIS partner agencies. Further, NBIC officials have attempted to demonstrate the value of participating in NBIS and supporting the integration center by encouraging agencies to participate in NBIC’s daily production process. NBIC officials said that through daily engagement in the production process and during recent real life events like food borne illness outbreaks they have been able to demonstrate the value of NBIC. However, agency officials told us that their experiences with NBIC during real life events and the tabletop exercise created questions about the value of participating in the NBIS and NBIC’s purpose.

NBIC officials have drafted but not completed a strategic plan for NBIC that includes a mission statement, which could help clarify NBIC’s purpose. The plan is also to provide strategic and operational guidance to NBIC officials for achieving that mission. According to NBIC officials, however, they have not shared the draft strategic plan with NBIS officials or solicited their input, and it is not currently their plan to do so because it is an internal document. Officials have not set a deadline for completing the NBIC strategic plan because they are still in the process of vetting the initial draft internally.

In addition to uncertainty about the value of participating in the NBIS and the purpose of NBIC, we also found that NBIC has not completed and achieved buy-in for joint strategies, policies, and procedures for operating across agency boundaries. NBIC has drafted a Concept of Operations, which is intended to communicate joint strategies, policies, and procedures for operating across the NBIS. According to NBIC officials, they have solicited and considered comments from NBIS partners as they developed the draft, which is currently on its third version. However, NBIC has not yet achieved agreement around strategies, policies, and procedures that would support effective collaboration across the NBIS. For example, one key partner agency—one for which biosurveillance is a mission critical function and is thus essential to a strong and effective NBIS—shared with us a memo they had written to NBIC expressing their lack of concurrence with the current Concept of Operations. The memo cited several concerns that related largely to lack of clarity in the document about the desired common federal outcome and the role of the different partners in achieving it. NBIC officials told us they plan to finalize the Concept of Operations by the end of 2009.

Clearly defining its mission, as well as articulating the value of participation in the NBIS, could help NBIC overcome challenges convincing agencies to work collectively as part of the NBIS. In addition, establishing and clarifying joint strategies, policies, and procedures with buy-in across the NBIS, could help address barriers to collaboration.

NBIC Has Not Clearly Identified How to Leverage Resources or Effectively Defined Roles and Responsibilities with NBIS Partners

Two of the collaborative practices we recommend speak to how agencies will share human and other assets to achieve the desired outcomes— identifying and addressing needs by leveraging resources and agreeing on roles and responsibilities. According to NBIC officials, the concept of a national center for integrating biosurveillance data from multiple agencies depends on the willingness of the collaborating agencies to detail their experts to the center for a period of time to interpret the data for signs of an outbreak or biological attack; consequently, effectively identifying what resources are available and how to leverage them is important.

In our work on practices to enhance and sustain collaboration, we call for agencies to assess relative strengths and weaknesses to identify opportunities to leverage each other’s resources, thus obtaining additional benefits that would not be available were the agencies working separately. However, agency officials we met with stated that NBIC did not recognize the different levels of resources and capacities that each agency brought to this effort. Seven of the 14 groups of agency officials we interviewed noted that the NBIC made personnel requests that were not compatible with the resources agencies had available. For example, one of the comments officials made to us regarding NBIC’s request for personnel details was that they did not have available or could not spare personnel that matched NBIC’s request for senior-level officials with sufficient analytical knowledge and authority to make immediate decisions about sharing information across the NBIS. Officials from one of the components without a direct biosurveillance mission told us that they only have one such person on staff and needed to keep that person in house to be able to carry out their mission-critical activities. Officials at two agencies described methods they had devised for human-resource sharing arrangements that did not involve locating senior staff at NBIC for several months. However, NBIC officials told us that this is no substitute for the value of a member agency personnel detail that is physically located at NBIC. NBIC officials noted that the Secretary of Homeland Security had sent a memo to other NBIS agency leadership requesting help in securing personnel details on May 23, 2008. In addition, they stated that the issue is regularly addressed in NIWG and NIOC meetings. The officials also provided several examples of outreach to NBIS officials at all 11 agencies, such as through discussions with NBIS partner agency representatives at NIWG meetings.

Similarly, 5 of 14 groups of officials we interviewed reported that they had experienced confusion about how NBIC planned to use personnel details if they were provided. For example, one such agency expressing this confusion said that NBIC’s guidance on what it is looking for in a personnel detail had changed frequently. NBIC officials told us that initially they requested individuals with strong scientific backgrounds to assist with data analysis and interpretation (analyst model). However, they later determined that they could use senior-level agency officials who were knowledgeable about their home organization to act as liaisons by identifying specific subject matter experts to consult with NBIC, as needed (liaison model). According to these NBIC officials, they have communicated to the NBIS partners that if they detail personnel to NBIC, they can follow either the analyst model or the liaison model. Nevertheless, during our interviews a lack of clarity about personnel detail roles and responsibilities was among the reasons cited for not finalizing MOUs or interagency agreements for personnel details.

Of the two NBIS partners that placed personnel at NBIC, officials from one agency told us that although they still were not entirely clear on NBIC’s needs, they were committed to the NBIS concept. Therefore, they committed to send two half-time detailees each fitting one of the two types of detailees NBIC had alternately requested. These personnel details were ongoing as of October 2009. According to agency officials, they committed to a shorter detail than NBIC requested because they intend to use the current detail placement to help clarify for themselves what NBIC’s needs are and the extent to which the detail arrangement might be valuable to their agency. However, officials at the only other agency that had detailed personnel to NBIC told us that they had not renewed the detail agreement when it ended, in part because of budgetary challenges, but also because of a general perception at their agency that the detail had not been particularly valuable for the individual or for their agency. According to NBIC officials, the personnel details from this agency assisted NBIC immeasurably in both the analysis work and in thinking through how to grow and shape the personnel detail program.

We also discuss in our work on practices for enhancing and sustaining collaboration the importance of defining and agreeing on roles and responsibilities, to allow each agency to clarify who will do what, organize their joint and individual efforts, and facilitate decision making. Our analysis of NIOC and NIWG post meeting reports, NBIS tabletop exercise results, and interviews with NBIS agency officials reveals some ambiguity about NBIC’s mission, roles, and responsibilities, particularly during a crisis. Officials from 8 of 14 components we interviewed expressed uncertainty about NBIC’s role during a response relative to the biosurveillance capability provided by other agencies in the course of their routine, mission-critical duties. In large part, these officials said that if they had information to share that might involve a biological emergency, they would be more likely to interact with DHS’s National Operations Center (NOC), at which NBIC has representation, than directly with NBIC. The after action report, as well as comments from these officials, show that such questions about NBIC’s response role manifested during a recent tabletop exercise. In our interviews, officials from seven components expressed concerns about NBIC’s role in the exercise or real life events, ranging from lack of clarity about what role NBIC played or should play to statements that the exercise showed clearly that NBIC has no proper role in event response. According to the memo that the moderator prepared after the tabletop exercise, although the NOC did not participate, some participants thought NBIC would have been bypassed in favor of the NOC.

They said the NOC would perform the essential biosurveillance integration roles of coordinating and disseminating information across agencies, states, and the private sector. In addition, the memo notes that exercise participants were not in agreement about the proper role for NBIC in ongoing collection and dissemination of data specific to an identified event. Among the recommendations in the after-action memo was for NBIC to work internally with the appropriate DHS parties, including the NOC, to write protocols defining the NBIC role inside DHS. According to NBIC officials, they have followed up on this recommendation, by among other things, exploring it through the NIWG. Additionally, NBIC, the NOC, and other stakeholders have initiated discussions about how to develop appropriate protocols.

A related issue that came to light during the tabletop exercise and was a theme in interviews with NBIS officials is the extent to which NBIS partners trust NBIC to use their information and resources appropriately. According to the exercise after-action memo, participants repeatedly raised concerns about trusting NBIC with data, and participants also expressed concern that NBIC would reach the wrong conclusions or disseminate erroneous data or reports. Similarly, in our semistructured interviews, officials from 5 of 14 components said they were cautious about sharing data or information with NBIC because they lack confidence that NBIC will either interpret it in the appropriate context or reach back to the agency to clarify before sharing the data across the whole interagency community. These comments generally noted concerns that NBIC’s lack of contextual sophistication could lead to confusion, a greater volume of unnecessary communication in the biosurveillance environment, or even panic. NBIC officials acknowledged that subject matter expertise from the agencies with frontline responsibility for disease surveillance is essential for drawing appropriate conclusions about emerging situations. However, they also noted that analysts at NBIC have experience with public health and have been building their expertise as the program matures. Clearly identifying how NBIS resources, including personnel details, will be leveraged and establishing institutional roles and responsibilities, could strengthen NBIC’s efforts to obtain buy-in for agencies to fully participate in the NBIS, including by committing to personnel detail arrangements.

Creating a Mechanism to Monitor Performance and Accountability Could Help NBIC Enhance and Sustain Collaboration

We have previously reported that federal agencies can use their strategic and annual performance plans as tools to drive collaboration with other agencies and partners. Such plans can also reinforce accountability for collaboration by establishing performance measures and aligning agency goals and strategies with those of the collaborative efforts. Using established performance measures to evaluate and report on the effectiveness of collaboration could identify ways to improve it. NBIC’s draft strategic plan outlines milestones, goals, objectives, and key tasks needed for NBIC to meet its mission. These tasks include, among other things, defining an information-sharing strategy among its stakeholders, deploying IT to support its mission, and establishing standard operating procedures. However, despite acknowledging that interagency cooperation and collaboration remain a concern to resolve, the strategic plan does not address how NBIC will improve collaboration among current and potential NBIS member agencies or how it will measure collaborative results. NBIC’s draft strategic plan includes one proposed performance metric related to collaboration with NBIS partners—to assess current collaboration activities for relevance and contribution to NBIS mission requirements. However, the plan lacks a discussion of strategic objectives to achieve collaboration and, correspondingly, lacks associated measures and targets to monitor efforts to achieve collaborative results. Strategic objectives for collaboration and associated targets and measures could provide NBIC with a critical tool to help ensure that it appropriately focuses its efforts on enhancing collaboration with NBIS members and that the desired results are achieved.

Leveraging NBIS Governing Bodies to Develop a Strategy for Collaboration Could Help NBIC More Effectively Meet Its Mission

NBIC has the means to engage NBIS partners through the organizations that help organize and manage the NBIS community—the NIOC and the NIWG—but our analysis shows the integration center has not yet fully leveraged these groups to develop effective collaboration strategies. The purpose of the NIOC and NIWG governance bodies is to provide strategy and policy advice on the operation of the NBIS. Information on the status of NBIC’s efforts to achieve its mission has been provided to the NIOC, an oversight council serving the NBIS community, but substantive discussion of strategies for overcoming barriers to collaboration that impact NBIC’s execution of its mission did not occur during meetings with the NIOC. For example, post meeting reports from the NIOC—the higher level strategic governance body for the community of NBIS partners—show that the NBIC director routinely gave a status update of the MOUs and interagency agreements for each agency, during which agencies report the status from their perspective. However, in these segments of the NIOC meetings, the post meeting reports reflect little, if any, discussion of the reasons NBIS agency officials cited in our interviews for not finalizing the agreements. Neither do the reports show any focused effort to discuss barriers to participation or solutions to working across agency boundaries.

The NIWG—operational level working group—post meeting reports between March 2008 and May 2009 reflect only one discussion during which the need to finalize agreements was addressed. Although the NIWG has formed a sub-working group specifically to address collaboration, our review of the post meeting reports shows that neither the full NIWG nor the sub-working group has been effectively engaged in a focused effort to identify, discuss, and address challenges to working across agency boundaries. According to NBIC officials, they place contentious issues before the NBIS governance structure in a way that may not be clearly captured in post meeting reports. NBIC officials noted that the post meeting reports do not clearly reflect the numerous times they have made proposals for solutions to problems and have been met with silence from the attendees. However, they acknowledge that they have approached the NBIS governance bodies seeking buy-in for their proposals for tactical and operational approaches rather than an open-ended discussion seeking strategic solutions to the broader barriers to information and resource sharing. Leveraging these bodies to get meaningful input from NBIS- partner leadership could help NBIC ensure that it is able to identify commonly accepted solutions to working across agency boundaries.

Conclusions

Enhancing the federal government’s ability to detect and warn of biological events of national concern and to provide better situational awareness for response to those events depends on multiple actors inside and outside the federal government to work together effectively. The 9/11 Commission Act charged NBIC with early detection and situational awareness, but both the act and the operational guidance NBIC has developed acknowledges that this is to be done, in large part, through the NBIS—a multi-agency collaborative community. Despite the critical role of this collaborative community in achieving the act’s charge, the act does not require any specific agency to participate in the NBIS or to support the integration center. Therefore, it is imperative that NBIC employ collaborative practices to enhance and sustain collaboration across the NBIS so that this community of federal partners are fully and effectively engaged in pursuit of the overarching missions of early detection and enhanced situational awareness.

Although NBIC has made some efforts to strengthen relationships with and solicit participation from NBIS partners, working with the NBIS to develop a strategy for collaboration that includes key collaboration practices identified in our previous work could help the integration center promote more effective collaboration. During the course of our review, officials from the NBIS community recounted a number of constraints on their participation, including concerns about the clarity of NBIC’s mission and the ends to which shared information and resources would be used. We have previously reported that having a mission statement helps to clarify an agency’s focus and purpose. Moreover, our prior work on enhancing and sustaining collaboration in the federal government advises that practices such as articulating common outcomes, identifying appropriate resources to be shared, clarifying roles and responsibilities, and developing mechanisms to monitor performance and accountability could help NBIC address barriers to collaboration. However, NBIC has not formulated goals and objectives for overcoming barriers to collaboration and has no supporting performance and accountability mechanisms—such as performance measures—to help ensure that they are pursuing those goals effectively. In addition, although NBIC has created the NIOC and NIWG to provide strategic and operational advice on how the NBIS should function, NBIC had not effectively engaged them in a focused effort to identify shared solutions for overcoming barriers to collaboration and creating buy-in for joint strategies, policies, procedures, roles, and responsibilities. A strategy for helping ensure that NBIC applies key collaborative practices effectively and consistently, that draws on the existing intellectual resources of its strategic partners in the NIOC, and that includes mechanisms to monitor performance and accountability for collaborative results, may help NBIC and NBIS partners to identify and overcome challenges to sharing data and personnel for the purposes of earlier detection and enhanced situational awareness of potentially catastrophic biological events.

Recommendations for Executive Action

In order to help NBIC ensure that it effectively applies practices to enhance and sustain collaboration, including the provision of data, personnel, and other resources, we are making the following two recommendations to the Director of NBIC: In conjunction with the NIOC, finalize a strategy for more effectively collaborating with current and potential NBIS members, by (1) clearly defining NBIC’s mission and purpose, along with the value of NBIS membership for each agency; (2) addressing challenges to sharing data and personnel, including clearly and properly defining roles and responsibilities in accordance with the unique skills and assets of each agency; (3) developing and achieving buy-in for joint strategies, procedures, and policies for working across agency boundaries.

Establish and use performance measures to monitor and evaluate the effectiveness of collaboration with current and potential NBIS partners.

Agency Comments and Our Evaluation

We provided a draft of this report for review and comment to the following agencies: DHS, HHS, USDA, and the Departments of Commerce, Defense, Interior, Justice, State, Transportation, and Veterans Affairs, as well as the Environmental Protection Agency and the United States Postal Service. DHS provided written comments on December 10, 2009, which are summarized below and presented in their entirety in appendix I of this report. HHS, USDA, and the Departments of Commerce, Defense, Interior, Justice, Transportation, and Veterans Affairs, as well as the Environmental Protection Agency and the United States Postal Service did not provide written comments. We incorporated technical comments from DHS, USDA, and the United States Postal Service where appropriate.

DHS generally concurred with our findings and recommendations and stated that NBIC will work with the NIOC and all NBIS partners to develop a collaboration strategy to clarify both the mission space and roles and responsibilities of all NBIS partners. DHS has taken initial steps to implement our recommendations. For example, DHS noted that at the December 9, 2009, quarterly NIOC meeting, the Assistant Secretary of Health Affairs and Chief Medical Officer for DHS, Dr. Alex Garza, referenced this report’s findings and challenged NIOC members to work to resolve and address confusion regarding NBIS and NBIC. We are encouraged by DHS’s efforts to engage the NIOC to identify and overcome barriers to collaboration; continuing to work with the NIOC to develop and finalize a strategy for collaboration could help NBIC overcome challenges to sharing data and personnel. In addition, monitoring the effectiveness of collaboration through the use of performance metrics could help NBIC ensure they are progressing towards their goal of obtaining the resources necessary to accomplish its mission of early detection and situational awareness of biological events of national concern.

While DHS stated that we clearly identify the challenges faced by NBIC in carrying out its mission, the department also commented that the lack of a legal requirement for other federal agencies to participate in the NBIS prevents DHS from compelling the cooperation that is needed to ensure success of the NBIC mission. As we noted in our report, the lack of a legal requirement is what makes the effective use of collaboration best practices crucial for NBIC to be successful.

We are sending copies of this report to the Secretary of Homeland Security, Secretary of Health Human and Services, Secretary of Agriculture, Secretary of Commerce, Secretary of Defense, Secretary of Interior, Attorney General, Secretary of State, Secretary of Transportation, and the Secretary of Veterans Affairs, as well as the Administrator of the Environmental Protection Agency, the Postmaster General, the Director of NBIC, and interested congressional committees. The report is also available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions about this report please contact me at (202) 512-8777 or JenkinsWO@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix II.

William O. Jenkins, Jr Director, Homeland Security and Justice Issues .

Appendix I: Comments from the Department of Homeland Security

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Anne Laffoon, Assistant Director; Michelle Cooper; Clare Dowdle; Kathryn Godfrey; and Andrea Yohe made significant contributions to the work. Keira Dembowski, Susanna Kuebler, Alberto Leff, and Juan Tapiavidela also provided support. Amanda Miller assisted with design, methodology, and analysis. Tracey King provided legal support. Linda Miller provided communications expertise.