BIOFORENSICS

DHS Needs to 

Conduct a Formal 

Capability Gap 

Analysis to Better 

Identify and Address 

Gaps

Report to Congressional Requesters

January 2017

GAO-17-177

United States Government Accountability Office
Highlights of GAO-17-177, a report to 

congressional requesters

January 2017

BIOFORENSICS

DHS Needs to Conduct a Formal Capability Gap 

Analysis to Better Identify and Address Gaps

What GAO Found

The Department of Homeland Security (DHS) and the Federal Bureau of 

Investigation (FBI) have identified some gaps in their bioforensics capabilities, 

but DHS has not performed a formal bioforensics capability gap analysis. It is 

therefore not clear whether DHS and the FBI have identified all of their capability 

gaps. A capability gap analysis can help identify deficiencies in capabilities and 

can help support the validation and prioritization of how to address the gaps. 

DHS and the FBI have identified capability gaps using an informal 

undocumented process. For example, DHS held informal meetings to seek FBI 

input on capability gaps associated with recent casework. Gaps identified 

through this informal process include the inability to (1) characterize unique, 

novel, and engineered agents and “unknowns” (emerging or synthetic 

organisms) and (2) understand and communicate uncertainty associated with 

analyzing complex biological samples, among other things. In the absence of a 

well-documented bioforensics capability gap analysis, the rationale for DHS’s 

resource allocations, or its plans for future enhancements to existing capabilities

are not clear and thus cannot ensure that resources are being targeted to the 

highest priority gaps. 

In addition to DHS and the FBI, other organizations, such as the National 

Research Council (NRC) of the National Academy of Sciences (NAS), and the 

National Science and Technology Council (NSTC) of the Office of Science and 

Technology Policy (OSTP), have identified potential bioforensics capability 

needs. These needs can generally be grouped into three areas: science, 

technology and methods, and bioinformatics and data. GAO also convened a 

meeting of experts, with the help of NAS, and these experts updated a list of 

potential bioforensics capability needs that NAS and OSTP had previously 

identified within each of these areas. Some of the needs these experts confirmed 

as still relevant were similar to those DHS and FBI officials have identified, while 

others were different. For example, like DHS and the FBI, the experts agreed 

that an ability to characterize genetically engineered agents was needed, but 

they also suggested that evaluating existing protocols, such as those for DNA 

sequencing, to determine whether they were validated, was needed. GAO 

believes that this information may be helpful to DHS and the FBI as part of any 

future bioforensics capability gap analysis they undertake.

Since 2010, DHS has enhanced some of its bioforensics capabilities, with FBI 

input, by focusing on developing methods-based capabilities while maintaining 

agent-based capabilities. DHS has funded research and development projects

addressing areas such as genome sequencing approaches, which underpin 

many methods-based bioforensics capabilities. DHS is also developing an inhouse reference collection for use in investigations. In addition, DHS is 

developing the ability to characterize unique, novel agents as well as 

“unknowns,” such as synthetic organisms. DHS projects that some enhanced 

capabilities will be complete in about 2025. However, in pursuing enhancements, 

DHS faces several challenges, including establishing a statistical framework for 

interpreting bioforensics analyses and associated inferences and communicating 

them in a court setting, as well as obtaining suitable biological agents and DNA 

sequences to ensure quality references for use in investigations.

View GAO-17-177. For more information, 

contact Tim Persons at (202) 512-6412 or 

personst@gao.gov.

Why GAO Did This Study

The ability to attribute the source of an 

intentionally released biological threat 

agent and quickly apprehend and 

prosecute the perpetrator is essential 

to our nation’s safety. However, 

questions remain about whether DHS’s 

and the FBI’s capabilities have 

improved since the 2001 anthrax 

attack. GAO was asked to report on 

DHS’s and the FBI’s bioforensics 

capabilities. 

This report examines the (1) extent to 

which DHS and the FBI have identified 

gaps in their bioforensics capabilities 

since 2010, (2) bioforensics needs 

experts have identified, and 

(3) actions, if any, DHS and the FBI 

have taken to enhance their ability to 

attribute the source of a biological 

attack, and to identify any challenges 

to enhancing bioforensics capabilities. 

GAO’s review focused on the 

agencies’ efforts since 2010, when the 

FBI’s investigation of the 2001 anthrax 

attack was closed. GAO analyzed 

relevant agency documents and 

interviewed agency officials and 

scientists on issues related to 

bioforensics. GAO also convened a 

meeting of experts with NAS’s 

assistance to discuss potential 

bioforensics needs.

What GAO Recommends

GAO recommends that DHS—in 

consultation with the FBI—conduct a 

formal bioforensics capability gap 

analysis and update it periodically. 

DHS concurred with GAO’s 

recommendation.
Page i GAO-17-177 Bioforensics

Letter 1

Background 6

DHS Has Not Performed a Capability Gap Analysis to Help Focus 

Its Resources for Addressing Bioforensics Capability Gaps 9

Experts Have Identified Several Capability Needs for Bioforensics 14

DHS and the FBI Have Acted to Enhance Bioforensics 

Capabilities but Face Numerous Challenges 17

Conclusions 44

Recommendation for Executive Action 45

Agency Comments and Our Evaluation 45

Appendix I Objectives, Scope, and Methodology 47

Appendix II DHS R&D to Enhance Bioforensics Capabilities, 2008 – 2016 50

Appendix III List of Experts 52

Appendix IV Comments from the Department of Homeland Security 54

Appendix V GAO Contacts and Staff Acknowledgments 58

Tables

Table 1: Capability Needs for Bioforensics Identified by Experts 15

Table 2: Agent-Based and Methods-Based Capabilities for 

Bioforensics Analyses 20

Table 3: Broad and Technical Topic Areas in Broad Area 

Announcements (BAA), 2008 – 2016 50

Figures

Figure 1: DHS 2014 Bioforensics Roadmap 13

Figure 2: Broad Topic Areas for DHS R&D, 2008 – 2016 22

Figure 3: The Possible Composition of a Metagenomics Sample 29

Contents
Page ii GAO-17-177 Bioforensics

Figure 4: Genetic Engineering with Restriction Enzymes 34

Figure 5: The Dependence of Probative Value on Confidence in 

Analysis, Interpretation, and the Meaning of the Evidence 37

Abbreviations

BAA Broad Area Announcement 

BRC Bioforensic Repository Collection 

BSL Biological safety level

CBD Chemical and Biological Defense Division 

CDC Centers for Disease Control and Prevention

CFT cell free translational assay

CRISPR clustered, regularly interspaced, short palindromic repeats

DHS U.S. Department of Homeland Security

DNA deoxyribonucleic acid 

DOJ U.S. Department of Justice

ELISA enzyme-linked immunosorbent assay 
Page iii GAO-17-177 Bioforensics

FAR Federal Acquisition Regulation

FBI Federal Bureau of Investigation 

HHS U.S. Department of Health and Human Services

HSPD Homeland Security Presidential Directive

IEC International Electrotechnical Commission

ISO International Organization for Standardisation 

MALDI-TOF matrix-assisted laser desorption and ionization 

time-of-flight

NAS National Academy of Sciences 

NBACC National Biodefense Analysis and Countermeasures 

Center

NBFAC National Bioforensics Analysis Center

NRC National Research Council 

NSTC National Science and Technology Council 

OBAA Open Broad Area Announcements 

OSTP Office of Science and Technology Policy

PCR polymerase chain reaction 

R&D research and development 

SEM scanning electron microscopy

SNP single nucleotide polymorphisms 

SOP standard operating procedure

SWGMGF Scientific Working Group on Microbial Genetics and 

Forensics

TEM transmission electron microscopy 

WGAMF Whole Genome Approach to Microbial Forensics 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-17-177 Bioforensics

441 G St. N.W.

Washington, DC 20548

January 11, 2017

The Honorable Charles Grassley

United States Senate

The Honorable Ron Johnson

United States Senate

The Honorable Claire McCaskill

United States Senate

The Honorable Daniel M. Donovan, Jr.

House of Representatives

According to the Department of Homeland Security (DHS), the threat of a 

terrorist or criminal use of pathogenic organisms and their toxins is a 

great concern in the United States. Further, with the advent of synthetic 

biology, concerns have been raised that more virulent genetically 

engineered pathogens may be created and that completely new 

pathogens are being created synthetically, to be used nefariously.1 While 

not all are convinced that such threats will result in an attack, DHS’s 

mission requires it to be prepared for one. Therefore, an ability to attribute 

the source of a biological attack and quickly apprehend and prosecute the 

perpetrator is essential to our nation’s safety.2 However, concerns have 

been raised about whether that ability has improved since the 2001 case 

concerning the anthrax-contaminated letters sent through the postal 

facilities to members of Congress and the media (Amerithrax), in which 

attribution took several years.3 DHS and the Federal Bureau of 

 1Researchers routinely generate pathogens containing recombinant or synthetic nucleic 

acid molecules for a variety of purposes, including the creation of vaccines using 

recombinant material. “Recombinant pathogens” refers to pathogens that contain 

molecules that are constructed by joining different nucleic acid molecules together 

(recombinant) or by creating completely new nucleic acid molecules (synthetic).

2By August 2016, 65 select agents and toxins had been determined to have the potential 

to pose a severe threat to human, animal, or plant health and safety, or to animal or plant 

products. For the purpose of this report, the term biological agent encompasses select 

agents, such as bacteria, viruses, and toxins. The list of agents and toxins is at 

http://www.selectagents.gov/SelectAgentsandToxinsList.html

3U.S. Department of Justice (DOJ), Amerithrax Investigative Summary (Washington, D.C., 

Feb. 19, 2010). 

Letter
Page 2 GAO-17-177 Bioforensics

Investigation (FBI) continue to play a role not only in responding to 

bioterrorism but also biocrimes, including the FBI’s investigation of 

multiple biocrimes involving the use of ricin—one of the most poisonous, 

naturally occurring substances—such as a case in 2013 in which ricin 

was sent to the U.S. President.

Attribution relies on many facets of an investigation—one of which is 

bioforensics.4 In its recommended guidelines for laboratories engaged in 

microbial forensic analyses, the Scientific Working Group on Microbial 

Genetics and Forensics (SWGMGF) defines attribution as “the 

information obtained on the identification or source of a material to the 

degree that it can be ascertained.”5 The FBI has also described attribution 

as “the act of attributing, the ascribing of a crime or act of terrorism to an 

actor.” It requires the totality of investigative information—one aspect of 

which may be forensic evidence—according to the FBI. However, when 

using scientific analyses in an investigation, “scientific attribution” has 

been defined as the assignment of a sample of questioned origin to a 

source, or sources of known origin, to the highest possible degree of 

scientific certainty—while excluding origination from other sources.6

Therefore, attribution to a perpetrator—the ultimate goal—is likely to be 

supported by both traditional forensics (for example, fingerprints) and 

bioforensics.

Attribution for the 2001 anthrax case took about 9 years. On February 19, 

2010, the FBI announced that that it was closing the case, having 

concluded that a scientist at the United States Army Medical Research 

Institute for Infectious Diseases had perpetrated the attack alone. The 

scientific analyses were a key investigative lead, according to the FBI.7

 4Microbial forensics characterizes, analyzes, and interprets microbial evidence for 

attribution purposes. The field has grown from the multidisciplinary fields of genomics, 

microbiology, and forensics, among others. Microbial forensics has also been referred to 

as “bioforensics” and “forensic microbiology.” In this report, we use “bioforensics.” 

5Scientific Working Group on Microbial Genetics and Forensics (SWGMGF), “Quality 

Assurance Guidelines for Laboratories Performing Microbial Forensic Work,” FBI 

Laboratory, Quantico, Virginia, June 20, 2003, in Forensic Science Communications 5:4 

(October 2003). http://www.fbi.gov/about-us/lab/forensic-sciencecommunications/fsc/archives. 

6National Research Council (NRC), Science Needs for Microbial Forensics:Developing 

Initial International Research Priorities (Washington, D.C.: National Academies Press, 

2014), p. 14.

7U.S. Department of Justice, Amerithrax.
Page 3 GAO-17-177 Bioforensics

However, experts have noted that U.S. bioforensics capabilities at that 

time were “initially limited to detection and identification and did not 

include detailed characterization and comparative analyses.”8 Therefore, 

because the case was never tried in a court of law, it is not clear whether 

the scientific analyses the FBI conducted were sound enough to support 

its conclusions.

In 2008, the FBI asked the National Research Council (NRC) of the 

National Academy of Sciences (NAS) to review the scientific approaches 

it had used to support is conclusions. NRC issued its report in 2011. NRC 

found that “it is not possible to reach a definitive conclusion about the 

origins of the B. anthracis in the mailing based on the available scientific 

evidence alone,” and it also details many methodological and 

organization problems in the scientific portion of the FBI’s investigation.9

Our prior work also revealed several gaps in the FBI’s validation of a key 

set of genetic assays for the 2001 investigation as well as challenges 

related to characterizing microbial deoxyribonucleic acid (DNA) in an 

effort to identify its source—which could be important in future 

investigations.10 Specifically, we reported on the need for a statistical 

framework for analyzing the results the assays generated. We also 

identified challenges related to the use of signatures, or genetic markers, 

and their significance in an investigation.

Agency responsibilities changed after the 2001 anthrax attack with DHS’s 

establishment. Today, DHS and the FBI coordinate tasks to attribute the 

source of a released biological agent—while the FBI alone investigates 

actual criminal acts and DOJ prosecutes alleged perpetrators. DHS’s 

National Bioforensics Analysis Center (NBFAC)—a dedicated 

 8Bruce Budowle and others, Microbial Forensics, 2nd ed. (Burlington, Mass.: Academic 

Press, 2011), p. xix.

9NRC, Review of the Scientific Approaches Used during the FBI’s Investigation of the 

2001 Anthrax Letters (Washington, D.C.: National Academies Press, Feb. 2011), p. 144.

10See GAO, Anthrax: Agency Approaches to Validation and Statistical Analyses Could Be 

Improved, GAO-15-80 (Washington, D.C.: Dec. 19, 2014).
Page 4 GAO-17-177 Bioforensics

bioforensics laboratory—is responsible for analyzing evidentiary samples 

in a bioforensics investigation.11

In addition, capabilities that could enhance bioforensics continue to 

evolve. The Bioforensics Research and Development (R&D) program 

supports both customers within DHS (for example, Customs and Border 

Patrol) as well as other agencies, including the FBI, and collaborates with 

federal and international partners. DHS has funded R&D of new 

technologies and associated processes that are intended to achieve a 

range of homeland security goals, including those for NBFAC’s 

bioforensics analyses for FBI casework. These include the ability to 

examine biological agents and nonbiological materials submitted as 

evidence to provide relevant information and intelligence that supports an 

investigation. DHS also works with the FBI and others to identify potential 

gaps in NBFAC’s bioforensics capabilities, in an attempt to ensure that it 

is prepared for FBI’s casework needs.

In this context, you asked us to evaluate issues related to the status of 

DHS’s and the FBI’s bioforensics capabilities for attributing a biological 

attack and whether those capabilities have any scientific and technical 

gaps. For this report, we evaluated (1) the extent to which DHS and the 

FBI have identified gaps in their bioforensics capabilities since 2010, 

(2) bioforensics needs experts have identified, and (3) any actions DHS 

and the FBI have taken to enhance their bioforensics capabilities, 

including those for characterizing a novel synthetic biological weapon, 

and any challenges they have experienced in enhancing bioforensics 

capabilities.

To determine the extent to which DHS and the FBI have identified gaps in 

their bioforensics capabilities, we reviewed agency documents and 

interviewed relevant agency officials about their efforts to identify such 

gaps since 2010, when the Department of Justice closed the FBI’s 

investigation of the 2001 anthrax attack. We examined agency planning 

documents, such as DHS’s Strategic Plan 2015-2019 and NBFAC’s 

Bioforensics Roadmap for research, among others. We reviewed DHS 

policy and guidance, such as DHS’s Joint Requirements Integration and 

 11Under a 2004 Presidential Directive, NBFAC is the lead federal facility to conduct and 

facilitate the technical forensic analysis and interpretation of materials from biocrime and 

bioterror investigations or those recovered following a biological attack in support of the 

lead federal agency. Analysis of evidentiary samples may result in evidence that will be 

admissible in court or used as an investigative lead.
Page 5 GAO-17-177 Bioforensics

Management System, which formed the basis for the criteria we used to 

compare and assess the extent to which DHS had identified capability 

gaps or conducted a capability gap analysis of its bioforensics 

capabilities.

To develop a list of bioforensics needs that experts have identified, we 

identified capabilities that might be needed for bioforensics purposes from 

a 2014 NRC publication entitled Science Needs for Microbial Forensics: 

Developing Initial International Research Priorities and the 2009 National 

Research and Development Strategy for Microbial Forensics from the 

National Science and Technology Council (NSTC). We extracted potential 

capability needs from these publications. We grouped the remaining 

bioforensics capability needs into three broad areas: (1) science, 

(2) technologies and methods, and (3) bioinformatics and data. We then 

convened, with the assistance of the National Academy of Sciences, a 

meeting of experts to discuss and update the capability needs we 

identified, including identifying issues related to these needs. These 

experts represented industry, academia, and government, and had 

experience in, among other things, microbiology, molecular genetics, nongenetic methods, genetic engineering, bioinformatics and statistics, and 

legal issues related to bioforensics. We also interviewed agency officials, 

including those with DOD to determine whether any gaps had been 

identified that related to bioforensics and their interactions with DHS in 

this regard.

To determine the actions DHS and the FBI have taken to enhance their 

bioforensics capabilities since 2010 and any challenges they 

encountered, we reviewed agency documents, including planning 

documents and R&D efforts. We also examined DHS’s actions to 

enhance NBFAC’s capabilities for the long term as well as the FBI’s 

casework needs. We reviewed DHS’s Broad Area Announcements (BAA) 

and Open Broad Area Announcements (OBAA) from 2008 to 2016. These 

are the mechanisms by which DHS solicits research to develop its 

bioforensics capabilities. We obtained details on contracted external R&D 

efforts. These included statistical models, standard operating procedures 

(SOP), and genetic sequences from external researchers. To determine 

any challenges to enhancing bioforensics capabilities, we reviewed 

agency documentation, related literature, and our prior work on 

bioforensics. We also interviewed agency officials and scientists, and 

obtained the opinions of experts in the United Kingdom, which 

collaborates with DHS and the FBI on bioforensics-related issues, as well 

as those in the United States, including those at our expert meeting. 

These included officials from the U.K. Home Office, Public Health 
Page 6 GAO-17-177 Bioforensics

England at Porton Down, and academia, regarding challenges related to 

bioforensics capabilities, including synthetic biology.

We conducted this performance audit from July 2015 to January 2017 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

DHS coordinates the federal government’s overall response to or 

recovery from terrorist attacks. The Centers for Disease Control and 

Prevention (CDC) within the U.S. Department of Health and Human 

Services is the primary agency for the public health response to a 

biological terrorism attack or naturally occurring outbreak.12 The FBI 

within DOJ is the primary agency for the criminal investigation of incidents 

of bioterrorism.13 In its recommended guidelines for laboratories engaged 

in microbial forensic analyses, the Scientific Working Group on Microbial 

Genetics and Forensics (SWGMGF) defines attribution as “the 

information obtained regarding the identification or source of a material to 

the degree that it can be ascertained.”14

As part of the effort to deter biological terrorism and strengthen the law 

enforcement response to such an act, Homeland Security Presidential 

Directive (HSPD) 10, “Biodefense for the 21st Century” established within 

DHS a dedicated central microbial forensic laboratory known as the 

NBFAC to provide bioforensics analysis of evidence associated with the 

 12Other agencies, such as the U.S. Environmental Protection Agency, could also be 

involved.

13According to Homeland Security Presidential Directive 5 (HSPD-5), the U.S. attorney 

general has lead responsibility for criminal investigations of terrorist acts or terrorist 

threats made by individuals or groups inside the United States or directed at U.S. citizens 

or institutions abroad, where such acts are within the federal criminal jurisdiction of the 

United States. The criminal investigation of biological incidents or bioterrorism is under the 

purview of Justice, and DHS is designated to coordinate overall response and recovery 

activities.

14SWGMGF Quality Assurance Guidelines.

Background 
Page 7 GAO-17-177 Bioforensics

event.15 This Presidential Directive established the NBFAC as “the lead 

federal facility to conduct and facilitate technical forensic analysis and 

interpretation of materials recovered from biocrime and bioterror 

investigations in support of the appropriate lead federal agency.”

DHS Science and Technology (S&T) is to accelerate the delivery of 

enhanced technological capabilities to meet the requirements and fill 

capability gaps to support DHS agencies in accomplishing their mission.16

Pursuant to this mission the DHS Chemical and Biological Defense (CBD) 

Division seeks technologies to defend against a chemical and biological 

attack.17 In addition, the division is charged with pursuing research to 

improve response and restoration, conduct threat risk assessments, and 

invest in bioforensics R&D. In this regard, the Bioforensics R&D Program, 

according to DHS, supports NBFAC operational threat agent identification 

and characterization through investments in bioforensics research and 

next generation technologies to include molecular biology, genomic 

comparison techniques, genotyping assays and physical and chemical 

analysis of sample matrix to better understand the origin, evolutionary 

history, production method and dissemination mechanism associated with 

the malicious use of biological agents.

Bioforensics has been defined as an interdisciplinary field of microbiology 

devoted to the development, evaluation, validation, and application of 

methods to detect and fully characterize microbial samples containing a 

biological agent or its components for the purpose of making statistically 

meaningful comparative analyses.18 Attributing something to a perpetrator 

requires different types of information and analysis—both traditional and 

 15To achieve its mission under HSPD-10, NBFAC, a component of NBACC, maintains 

dedicated biological safety level (BSL) 2, BSL-3 and BSL-4 biocontainment laboratories, 

equipment, trained and vetted staff, and internationally accredited methods and assays to 

conduct continuously available bioforensics analyses 24 hours a day, 7 days a week.

16The Homeland Security Act of 2002 (Public Law 107-296, sec. 302) states that DHS 

S&T is responsible for “establishing priorities for, directing, funding, and conducting 

national research, development, test and evaluation, and procurement of technology and 

systems for ... detecting, preventing, protecting against, and responding to terrorist 

attacks.”

17The Homeland Security Advanced Research Projects Agency executes S&T’s R&D 

programs. In late 2010, S&T realigned itself to “enhance its ability to strategically 

contribute to the DHS Homeland Security Enterprise mission, operations and strategy.”

18The National Science and Technology Council, National Research and Development 

Strategy for Microbial Forensics, (Washington, D.C.: 2009). 
Page 8 GAO-17-177 Bioforensics

bioforensics. Information produced by forensic examination can result in 

an investigative lead or provide support for the investigation.

Bioforensics capabilities used to provide analyses of evidence may show 

how, when, and where microorganisms were grown and potential 

methods for dissemination, which assists attribution.19

• Bioforensics evidence could include the agent that was released, 

toxins, nucleic acids, and protein signatures. It could also include 

contaminants, additives, and evidence of preparation methods.

• Traditional evidence could include fingerprints, hair, fibers, 

documents, photos, firearms, and body fluids.

• In a bioforensics case, the intent would likely be to gather sufficient 

information to allow a comparison of an evidentiary sample with a 

known reference sample to assist in supporting source attribution. 

Evidence from a bioforensics investigation must also meet the 

scientific community’s standards for evidence as well as a criminal 

court’s standards for legal admissibility.20

 19According to one expert we contacted, it would be difficult to determine a specific 

dissemination method from evidence left behind after biological weapons were 

aerosolized although it might be possible to differentiate between wet and dry 

dissemination and maybe gain some additional general information but determining the 

specific methods would be challenging.

20Under the Federal Rules of Evidence, Rule 702, an expert witness is considered 

qualified to testify if, among other things, the testimony is the product of reliable principles 

and methods. The 1993 Supreme Court case, Daubert v Merrell Dow Pharmaceuticals, 

Inc. (509 U.S. 579), significantly changed the admissibility of scientific evidence for 

Federal trial courts, making trial judges responsible for acting as gatekeepers to exclude 

unreliable scientific expert testimony. The Daubert case listed factors for judges to use in 

assessing the reliability of scientific expert testimony, including (1) whether the expert’s 

technique or theory can be or has been tested, (2) whether the technique or theory has 

been subjected to peer review, (3) the known or potential rate of error of the technique or 

theory when applied, (4) the existence and maintenance of standards and controls, and 

(5) whether the technique or theory has been generally accepted by a relevant scientific 

community.
Page 9 GAO-17-177 Bioforensics

DHS has developed strategic plans and goals related to bioforensics 

attribution and identified some key bioforensics capability needs.21

However, according to DHS officials, DHS did not perform a bioforensics 

capability gap analysis, but, rather, used an informal approach to identify 

capability needs and gaps. DHS officials stated that they did not 

document the process DHS used or the results of its informal approach. 

Further, DHS officials told us that there is not a complete list of the gaps 

identified using the informal approach. Finally, although they indicated 

that DHS had focused resources toward addressing those gaps, they 

could not provide documentation of the bioforensics capability 

requirements and other relevant information to support the capability gap 

identification and resource allocation decisions that were made. In the 

absence of documentation of the processes, discussions, analyses, 

decisions, or any other activities performed to identify and prioritize 

bioforensics gaps, DHS’s rationale for the identification and prioritization 

of needs and gaps on which to focus its resources is unclear.

According to DHS officials, DHS relies on the DHS S&T-managed

NBFAC and bioforensics R&D programs to identify bioforensics capability 

needs and gaps. However, DHS does not have a complete list of its 

bioforensics capability gaps because it has not performed a bioforensics 

capability gap analysis. According to the DHS Systems Engineering Life 

Cycle guide, a gap analysis is a best practice that is essential to 

understanding whether capabilities exist that can meet requirements, or if 

they must be developed.22 In addition, some DHS officials told us that 

performing a capability gap analysis is a best practice that DHS programs 

should follow, even in the absence of DHS guidance to do so. In 

interviews and written responses, DHS officials described generally how 

DHS identified and documented capability needs and gaps. They told us 

that they identify priorities each fiscal year, develop projects to meet 

these priorities, and develop the NBFAC Annual Plan to address these 

 21DHS defines a capability as the means to accomplish a mission, function, or objective. A 

capability need is a capability necessary to achieve an organization’s mission. A capability 

gap is a capability need for which there is no existing capability. Generally, a capability 

gap analysis involves (1) identifying the scope and basis of the analysis based on the 

strategic context, mission, and scenarios, (2) identifying necessary capabilities, 

(3) assessing current capabilities, (4) identifying the gaps between necessary and current 

capabilities, (5) assessing the risk of the capability gaps, (6) assessing alternative 

solutions to address the gaps, (7) and documenting the results of the analysis.

22DHS, DHS Systems Engineering Lifecycle, (Washington, DC: Acquisition Program 

Management Division and the Office of the Chief Information Officer, September 2010).

DHS Has Not 

Performed a 

Capability Gap 

Analysis to Help 

Focus Its Resources 

for Addressing 

Bioforensics 

Capability Gaps

DHS Did Not Use a 

Systematic, Documented 

Approach to Identify Its 

Bioforensics Capability 

Needs and Gaps
Page 10 GAO-17-177 Bioforensics

priorities. However, they told us that there is no documentation of the 

process or results of the informal approach they used to identify 

bioforensics capability needs and gaps. According to federal standards 

for internal control, documentation is necessary for the effective design, 

implementation, and operation of an entity’s internal control system.23

Lacking documentation of the processes, discussions, analyses, 

decisions, or any other activities performed to identify and prioritize 

capability gaps, it is unclear what DHS’s rationale was for the 

identification and prioritization of bioforensics capability needs and gaps. 

Identifying and prioritizing capability gaps enables the proper allocation of 

resources to the highest priority needs. Thus, without a capability gap 

analysis, DHS may not have identified and prioritized all capability needs 

and gaps, and so may not be allocating resources to address the most 

significant gaps to meet its mission needs.

DHS officials told us that no complete list of bioforensics capability gaps 

has been created since 2010. However, they told us that DHS had 

developed a document from 2013-2014—the Bioforensics Roadmap 

(Roadmap)—as a means to identify and achieve consensus from 

stakeholders on the key bioforensics capability needs on which to focus 

DHS resources.24 DHS officials said that the Roadmap lays out the 

Bioforensics R&D Program execution and also lists the key needs on 

which DHS has focused, or will focus, resources, along with the 

associated programs to address the needs. DHS developed strategic 

plans for its National Biodefense Analysis and Countermeasures Center 

(NBACC) in 2012 and Chemical and Biological Defense in 2013.25

It documented NBFAC strategic goals in 2013.26 These documents 

 23GAO, Standards for Internal Control in the Federal Government, GAO-14-704G 

(Washington, DC: September 2014).

24DHS uses roadmaps to list R&D objectives and guide investments.

25DHS National Biodefense Analysis and Countermeasures Center, Strategic Plan (Fort 

Detrick, Md.: Battelle National Defense Institute, October 2012) and DHS Science and 

Technology Directorate, Chemical and Biological Defense Research and Development 

Strategic Plan (July 19, 2013)

26While the priorities and capabilities identified in these documents are not identified as 

gaps, DHS officials told us that they considered them to be gaps.
Page 11 GAO-17-177 Bioforensics

include strategic objectives related to bioforensics that could be used to 

guide a capability gap analysis.27

A former DHS official who had participated in DHS’s process for the 

identification of bioforensics gaps told us that the process was informal 

and also that there is no documentation of the process. This official 

summarized it as generating a list of topics and issuing BAAs to address 

them. Other DHS officials confirmed that the process was informal and 

that there was no documentation of the results. They told us that they are 

unaware of the details of the processes and activities performed to 

identify capability needs and gaps. However, they did describe generally 

the informal process that DHS used. 

They indicated that the process included working with key interagency 

partners and other stakeholders—such as the FBI—and engaging in 

discussions, exchanging emails, and holding periodic meetings. DHS 

officials also stated that they met informally as needed with the FBI and 

some intelligence agencies to discuss needs and gaps. These officials 

explained that the discussions with DHS’ interagency partners were part 

of a larger process to develop and manage NBFAC and bioforensics R&D 

programs. They said that DHS coordinated with the FBI and the 

Intelligence Community to focus these programs’ activities to meet the 

needs of these end users. Further, they said that the Roadmap was 

vetted by other agencies and researchers. In addition, according to FBI 

officials, the FBI conducted assessments of its capabilities by working 

with the DHS S&T and providing direction to DHS about its capability 

needs. FBI officials stated that sometimes the FBI does not know there is 

a bioforensics capability gap until it encounters one during an 

investigation.

Independent assessments of DHS’s S&T Bioforensic R&D program have 

raised similar concerns about how DHS has identified and prioritized 

bioforensics capability gaps. For example, external assessments of the 

CBD portfolio from 2012 and 2014 found a lack of clarity about how the 

Bioforensics R&D program identified and prioritized capability gaps and 

why some projects were chosen. The reviewers recommended that the 

program manager describe the program’s basis for identifying capability 

 27According to the DHS Manual for the Operation of the Joint Requirements Integration 

and Management System, the analytical work conducted as part of a capability analysis 

provides traceability between DHS strategic guidance and the development of necessary 

capabilities.
Page 12 GAO-17-177 Bioforensics

gaps and selecting projects in future reviews. Specifically, a November 

2014 review acknowledged the need for enhancing bioforensics

capabilities but questioned the lack of information on how gaps in 

capability or knowledge guiding R&D investments were identified and 

prioritized. A 2012 review stated that it was unclear why some research 

studies were chosen over others, as well as how the selection of projects 

was linked to, or justified against, the risk assessment.

Through the Roadmap, key bioforensics efforts have been identified, 

which DHS officials characterized as gaps. The efforts listed in the 

Roadmap include (1) operational infrastructure, (2) sample collection and 

preservation, (3) sample extraction, (4) identification and characterization, 

and (5) data analysis and integration. The Roadmap also includes 

existing and future CBD and other agency programs, as well as 

commercial development, linking them to the particular capability gap 

they address.28 The Identification and Characterization effort in the 

Roadmap includes developing capabilities to characterize unique, novel, 

and engineered agents; characterize unknowns (emerging or synthetic 

organisms); identify and characterize toxins, such as ricin; and quantify 

and communicate uncertainty, which is of particular significance when 

using metagenomics and proteomics capabilities. A DHS official and FBI 

officials also told us that other items they considered to be gaps include 

the difficulties in interpreting metagenomics data, limited sequences for 

select organisms in its reference database, and the need for a greater 

ability to examine proteins. However, the Roadmap provides no additional 

details about the bioforensics capability gaps other than the projected 

timeframe of 2014 to 2020 for completion of the agency programs. Figure 

1 shows the Roadmap. According to DHS officials, to bridge the broad 

gap areas set out in the Roadmap would require resources far beyond 

those available to DHS.

 28The Bioforensics R&D Program is part of DHS S&T’s CBD Countermeasures Thrust 

Area, and the NBFAC is a component of the NBACC within CBD. 

DHS and the FBI Have 

Identified Several 

Bioforensics Capability 

Needs and Gaps
Page 13 GAO-17-177 Bioforensics

Figure 1: DHS 2014 Bioforensics Roadmap

Legend: Ribonucleic acid (RNA), complementary deoxyribonucleic acid (cDNA), single nucleotide 

polymorphisms (SNP), short tandem repeats (STR), high performance computing cluster (HPC 

cluster), and DHS Chemical and Biological defense (CBD).
Page 14 GAO-17-177 Bioforensics

In addition to DHS and the FBI, other organizations, such as the NRC of 

the NAS and the NSTC of the Office of Science and Technology Policy 

(OSTP), have been involved in identifying bioforensics capability needs. 

The NRC Committee on Science Needs for Microbial Forensics was an 

international group of experts that identified scientific challenges that 

must be met to improve the capability of bioforensics to investigate 

suspected outbreaks and to provide evidence of sufficient quality to 

support responses, legal proceedings, and the development of 

government policies. Similarly, OSTP’s National Research and 

Development Strategy for Microbial Forensics was established to guide 

and focus U.S. Government research efforts to advance the discipline of 

bioforensics. With the assistance of the NAS, we convened our own 

meeting of experts in April 2016 to review and update capability needs 

that the NRC and OSTP identified and to identify additional needs that 

might be useful for DHS and the FBI to consider when they identify their 

capability needs as part of a bioforensics capability gap analysis. Some of 

the experts provided alternative views about certain aspects of the 

identified capability needs. While some of the bioforensics capability

needs identified overlap with efforts listed in the DHS Roadmap, they 

were not formulated specifically considering DHS requirements so may 

not be relevant to DHS. However, we believe that this information could 

help inform the DHS’s and FBI’s efforts to identify capability needs and 

prioritize gaps.

Starting with the capability needs identified by the NAS and OSTP, the 

experts that participated in the GAO meeting identified and generally 

agreed upon the capability needs listed in Table 1. Some of the needs the 

experts confirmed as still relevant were similar to those identified by the 

DHS and the FBI, and some were different. For example, like DHS and 

the FBI the experts agreed that an ability to characterize genetically 

engineered agents was needed, but they also suggested that evaluating 

existing protocols, such as those for DNA sequencing, to determine 

whether they were validated, was needed. The identified needs in table 1 

can generally be grouped into three broad areas: (1) science, 

(2) technology and methods, and (3) bioinformatics and data. There are 

six needed capabilities within the science area; five within technology and 

methods; and three within the bioinformatics and data areas.

Experts Have

Identified Several 

Capability Needs for 

Bioforensics
Page 15 GAO-17-177 Bioforensics

Table 1: Capability Needs for Bioforensics Identified by Experts

Area Capability need 

Science 1. Identify, monitor, and characterize agreed on microbial species of most concern, including 

phenomena such as population dynamics and environmental effects to gene stability, gene transfer, 

and mutation rates

2. Continue research to determine mechanisms of pathogenicity, including virulence factors and host 

immune responses, focusing on problems related to bioforensics

3. Develop methods to distinguish natural, accidental, and deliberate outbreaks of infectious 

diseases, including those involving an engineered organism, rapidly and with high confidence

4. Identify forensic signatures and improved characterizations for known, emerging, enhanced, 

genetically engineered, and synthetically derived agents

5. Develop sensitive and broad detection capabilities for known, emerging, enhanced, genetically 

engineered, and synthetically derived agents

6. Continue research to realize the promise of metagenomics as it applies to microbial forensics and 

develop other technologies that can be applied to microbial forensics, including proteomics, 

metabolomics, transcriptomics, glycomics, immunogenomics, and lipodomics that can provide 

advantages over traditional methods 

Technology and methods 1. Adapt physical science applications to microbial forensics

2. Adapt more advanced, faster, and cheaper assay and sequencing technologies and standardize 

and validate them for bioforensics

3. Compile all existing protocols in use (e.g., collection, preservation, recovery, concentration, 

sampling, extraction and isolation, preservation, sequencing) to determine whether and how they 

have been validated and identify current research gaps and research efforts to avoid duplication

4. Develop and validate processes and analytical methods for microbial forensics (e.g.. sample 

collection, preservation, recovery, handling, storage, packaging, and transportation), including 

establishing standards (e.g., for components, processes, materials, data, performance), to 

demonstrate the information generated can answer key investigative and legal questions

5. Develop and validate nongenetic orthogonal methods to conduct sample characterization 

Bioinformatics and data 1. Create data repositories and reference collections for pathogens and other microorganisms and 

develop standards for metadata

2. Create reference collections for standards and other reference materials required for the 

development and validation of microbial forensics methods

3. Develop and refine bioinformatics and statistical methods for evaluating evidence in microbial 

forensics capable of incorporating diverse analytical results into forensics comparisons and building 

networks and models to help investigators draw inferences regarding sample relatedness with 

described confidence intervals. This should include new algorithms that scale to very large or 

complex databases 

Source: GAO. | GAO-17-177

While the majority of the experts agreed generally with the bioforensics 

capability needs in the three broad areas listed in table 1, some experts 

also had alternative views about some of the needs. For example, some 

experts thought that some of the needs should have a different focus or 

should be given a lower priority than others. In addition, some experts 

suggested that because it may be impossible to characterize all microbes, 
Page 16 GAO-17-177 Bioforensics

the first science capability need—the identification, monitoring, and 

characterization of microbial species—should instead focus on 

(1) developing a dynamic process and infrastructure for rapid collection 

and typing when an event occurs or (2) using a species-agnostic 

approach to identify both natural and synthetic microbes, such as 

focusing on genetic mechanisms rather than organisms.

Additionally, some experts stated that limited emphasis should be placed 

on the third science need—developing methods to distinguish among 

natural, accidental, and deliberate outbreaks. They indicated that other 

investigatory data would be available that would be better suited for 

making this determination. Instead, they said the focus should be on 

identifying introductions of additional virulence or genetic elements into an 

organism and determining other elements that suggest that somebody 

has modified the organism. There was also disagreement on which 

microbes DHS and the FBI should focus. Some experts stated that, 

because new pathogens are difficult to create, the greater concern is 

naturally-occurring or modified microbes. Further, they also said that 

distinguishing among existing organisms already presents a difficult 

enough challenge. Finally, some experts said that the focus should be on 

microbes in laboratories because they are the most relevant to 

bioforensics and are not typically studied by the larger scientific 

community.

The experts disagreed on whether the sixth science capability need—

metagenomics research—was important for bioforensics. Some stated 

that metagenomics is worth exploring as a future capability but there are 

easier problems that need to be solved. Others said that a metagenomics 

capability is not necessary for analyzing simple samples but it might be 

useful for analyzing complex samples. In addition, one expert questioned 

the fifth technology and methods need—the need for nongenetic 

orthogonal methods—indicating that it is not a requirement in court to 

have two different methods to determine a result.

For some of the bioforensics capability needs, experts indicated that other 

groups would develop the capabilities so that DHS or the FBI would not 

need to invest in them. For example, some experts said that the FBI 

should not focus its effort on the second science need—researching the 

mechanisms of pathogenicity—because it is unlikely to be closed quickly, 

and other groups are already addressing it. Regarding the second 

technology and methods need—adapting assay and sequencing 

technologies—some experts indicated that the commercial market will 

drive the development of improved sequencing technologies. Similarly, 
Page 17 GAO-17-177 Bioforensics

some experts said that agencies, such as the FBI and CDC, are working 

to address the first bioinformatics and capability need—the creation of 

data repositories and reference collections for pathogens and other 

microorganisms.

The specific bioforensics casework requirements that formed the basis for 

the DHS’s efforts were not known to the experts that participated in the 

meeting and, consequently, the list of capability needs cannot be directly 

compared to the efforts in the Roadmap. These capability needs, along 

with the alternative views presented, could help inform the DHS’ and 

FBI’s efforts to identify and prioritize bioforensics capability gaps. These 

agencies could consider this information as part of any capability gap 

analysis.

DHS and the FBI have taken actions to enhance some bioforensics 

capabilities but face numerous challenges before they can achieve the 

desired enhancements. Actions include not only the concrete steps that 

DHS has taken to enhance its capabilities, such as funding R&D 

activities, but also key strategic decisions underlying those actions. In this 

context, DHS actions include (1) developing methods-based capabilities 

to provide a broader bioforensics capability; (2) funding R&D activities to 

enhance its capabilities; (3) developing capabilities for short-term 

casework needs; (4) establishing an in-house reference database; and 

(5) developing capabilities for characterizing genetically engineered and 

unique, novel, or unknown (emerging or synthetic) agents. However, to 

achieve the capability enhancements they are pursuing, DHS and the FBI 

must overcome numerous challenges. These include (1) achieving the 

ability to interpret and communicate results from the bioforensics 

capabilities with a statistical confidence, (2) developing statistical 

frameworks, quantitative measures, and quality reference collections, 

(3) ensuring that its Bioforensic Repository Collection (BRC) contains 

quality data and appropriate agent strains, and (4) determining future 

casework needs relative to views of the evolving threat landscape. In 

addition, experts at our meeting and those we interviewed identified 

challenges regarding reference databases, the use of statistical 

frameworks, and the communication of results.

DHS and the FBI 

Have Acted to 

Enhance Bioforensics 

Capabilities but Face 

Numerous 

Challenges
Page 18 GAO-17-177 Bioforensics

DHS has taken several actions to enhance some of NBFAC’s 

bioforensics capabilities for use on FBI casework. For example, we found 

that since 2010, DHS, with FBI input, made a strategic decision to focus 

on the development of methods-based capabilities rather than agentbased capabilities for identifying and characterizing biological agents. 

This strategy is reflected in the 2012 NBACC strategic plan and its goals 

for NBFAC.29 Methods-based capabilities, according to DHS’s written 

responses to our questions, include genomics (whole genome 

sequencing and bioinformatics analysis), and analytical chemistry (mass 

spectrometry and scanning and transmission electron microscopy). In 

addition, DHS will maintain and enhance its agent-based capabilities in 

the interim—which include molecular biology, virology, bacteriology and 

toxinology—some of which will always be necessary for certain types of 

casework. Both types of capabilities will reside at NBFAC.

The FBI agrees that such enhanced capabilities are needed. In 

responding to our questions, the FBI stated that DHS’s approach will 

provide “an adaptive and agile capability to characterize unique, novel, 

engineered or emerging biological agents.” While agreeing with the need 

to develop methods-based capabilities, however, the FBI also 

acknowledged in its responses that some agent-specific capabilities will 

always be needed for its investigations.

Methods-based capabilities, according to DHS’s written responses to our 

questions, can potentially provide NBFAC with a broader bioforensics 

capability. For example, DHS stated that genomic analysis can use 

unique features as signatures to differentiate a particular isolate from 

others. DHS also responded that such features could include single 

nucleotide polymorphisms (SNP), rare variants, and epigenetic variation. 

Further, DHS stated that genomics-based characterization—including the 

ability to characterize background nucleic acids that may be derived from 

the environment in which the sample originated—represents a unique 

investigative signature that agent-based bioforensics procedures would 

miss. According to experts at our meeting, signatures range from 

anything that aids an investigation, to genetic signatures, syndromic 

signatures, metadata, and proteins, as well as other molecular signatures. 

 29Related goals included (1) identify and characterize any biological agent in any sample, 

and (2) establish a production deduction capability. See NBACC’s 2012 Strategic Plan.

DHS and the FBI are 

Focusing on MethodsBased Capabilities to 

Provide a Broader 

Bioforensics Capability
Page 19 GAO-17-177 Bioforensics

DHS officials told us that the use of methods-based approaches, such as 

genomics, have dramatically reduced investigation timeframes. They said 

that DHS can now detect and sequence not only select agents, but a 

number of other biological agents (even bioengineered ones) in a fraction 

of the time. What took years to complete in the 2001 Amerithrax case can 

now happen much more quickly with such improvements. An FBI official 

further elaborated, stating that improvements in techniques and 

technologies have led to potential increases in obtainable information and 

significant reductions in analysis times supporting bioterrorism 

investigations.

In contrast, based on our review, prior to 2010, we found that NBFAC’s 

bioforensics capabilities focused on identifying biological agents on the 

CDC and USDA select agent lists.30 In this regard, in its written responses 

to our questions, DHS stated that it has established International 

Organization for Standardisation (ISO) 17025 accredited, complementary 

assays such as culture, real-time polymerase chain reaction (PCR), and 

immunoassays for most traditional bacterial, viral, and toxin agents. 

However, unlike methods-based capabilities, DHS stated that these 

require prior knowledge of an organism and the maintenance of agent‐

specific reagents.31 Further, agent-based capabilities do not cover a wide 

array of potential threats, including genetically modified or de novo

agents, and have not been developed for known human pathogens, 

especially those that may not be cultivable. Thus, according to DHS 

responses, a methods-based approach will ultimately provide NBFAC 

with capabilities not only for analyzing challenging samples but also with 

a broader, more comprehensive bioforensics capability for characterizing 

unique, engineered, or emerging biological agents.

 30By regulation, CDC and USDA’s Animal and Plant Health Inspection Services establish 

and maintain a list of each biological agent and toxin that has the potential to pose a 

severe threat to public health and safety or to animal or plant health or products. Agentbased capabilities include molecular biology (Real-time PCR, genotyping); virology 

(culture and identification, phenotype); bacteriology (culture and identification, phenotype); 

and toxinology (toxin identification and biological activity). Methods-based capabilities 

include genomics (whole genome sequencing and bioinformatics analysis), analytical 

chemistry using mass spectrometry and scanning and electron microscopy. 

31ISO/IEC 17025:2005, Technical Corrigendum 1, Sec. 5.4.5, 2006-08-1 5. ISO 17025 is 

intended to facilitate cooperation between laboratories and others in exchanging 

information and experience and to assist in harmonizing standards and procedures. IEC is 

the International Electrotechnical Commission. The standard applies to all organizations 

performing tests or calibrations such as first-, second-, and third-party laboratories and 

laboratories where testing is part of inspection and product certification. 
Page 20 GAO-17-177 Bioforensics

Table 2 shows these two types of capabilities and the types of analyses 

they could be used to perform on evidentiary samples. For example, 

bacteriology involves culturing and deriving phenotypic information on an 

agent to identify and characterize it.32

Table 2: Agent-Based and Methods-Based Capabilities for Bioforensics Analyses

Capabilitya Type of analysis 

Basis of analysis

Agent Method

Analytical chemistry Identification, characterization of ricin, abrin and other protein toxins 

(toxinology) using mass spectrometry, which also supports proteomics 

analyses



Bacteriology Culture identification, phenotypic characterization of multiple 

organisms



Electron microscopy Elemental analysis of samples to physically characterize them for 

example, size, shape, surface texture using SEM, TEM and light 

microscopy



Genomics Whole genome genotyping, large-scale comparative analyses, 

incremental metagenomics capability, inferential analysis



Molecular biology Identification of biological agents using Real-Time PCR and 

immunoassays, genotyping



Toxinology Identification, characterization of ricin, abrin, and other protein toxins 

using, for example, immunoassays, cell-free translation assays (also 

see analytical chemistry)



Virology Culture identification, phenotypic characterization of viruses 

Source: GAO analysis of DHS documentation. | GAO-17-177

a

Capabilities also include associated quality management (ISO 17025 accredited laboratory and 

methods) and sample receipt and processing (chain of custody) for NBFAC casework.

However, toxinology, another agent-based capability, could involve 

identifying and characterizing protein toxins such as ricin using an 

immunoassay—such as ELISA, an enzyme-linked immunoabsorbant 

assay.33 In addition, analytical chemistry, a methods-based capability, 

could be used to characterize toxins by using mass spectrometry, which 

also supports proteomics analysis. That is, both types of capability could 

 32Phenotype is an organism’s expressed traits. These are determined by an organism’s 

genotype (genetic complement) and expressed genes, random genetic variation, and 

environmental influences. Examples of a phenotype would be traits such as color, size, 

shape, and behavior.

33Ricin is a poison found naturally in castor beans. ELISA (enzyme-linked immunosorbent 

assay) is a plate-based assay technique designed for detecting and quantifying 

substances such as peptides, proteins, antibodies, and hormones.
Page 21 GAO-17-177 Bioforensics

be involved in analyzing toxins. According to DHS responses to our 

questions, use of each of its mass spectrometry methods function 

independently and provide complementary information to confirm results 

derived from immunoassays and biological activity assays for protein 

toxins. Electron microscopy—a methods-based capability—involves 

nonbiological analysis of evidence samples. For example, it could provide 

elemental analysis of an agent. Transmission electron microscopy (TEM) 

and Scanning Electron Microscopy (SEM) can be used to provide images 

of nonspore forming bacteria and viruses, and castor bean products, 

among others.

DHS has solicited and funded R&D projects to enhance NBFAC’s 

bioforensics capabilities—completion of which DHS anticipates will 

extend beyond 2025. The R&D is related to areas in which DHS has 

stated there are capability gaps, or it is linked to some of the program 

responses listed in the 2014 Roadmap. It also reflects DHS’s shift toward 

methods-based approaches, such as genomics and proteomics.

Using a BAA mechanism, DHS solicited research proposals for R&D 

related to enhancing its bioforensics capabilities.34 To more clearly 

describe the type of research sought, the BAAs specified not only broad 

topic areas as well as technical topic areas—more specific, technical 

details about the type of research being solicited. Subsequently, DHS 

awarded about 36 contracts for solutions or products addressing areas 

related to bioforensics.35 According to the FBI’s response to our 

questions, it is involved in the process from start to finish, including 

assisting in drafting the BAA, the proposal evaluation and selection 

process, and meeting with DHS and the contractors throughout the 

course of the contract.

Before being used in FBI casework, SOPs and other deliverables from 

the funded research would have to make a transition to NBFAC 

operations and potentially be accredited under ISO 17025, as 

 34The Federal Acquisition Regulation (FAR) prescribes procedures for the use of the BAA 

for the acquisition of basic and applied research. BAA’s may be used by agencies to fulfill 

their requirements for scientific study and experimentation directed toward advancing the 

state-of-the-art or increasing knowledge or understanding rather than focusing on a 

specific system or hardware solution. 48 C.F.R. § 35.016. 

35We excluded from this approximate number of contracts those not related to 

bioforensics, such as chemical forensics and animal-related studies. 

DHS Has Funded R&D for 

Bioforensics
Page 22 GAO-17-177 Bioforensics

appropriate.36 In responding to our questions, DHS stated that both ISO 

17025 accreditation and deliverables such as publications provide the 

data necessary to support the general acceptance of a method within the 

scientific community and to meet the Daubert standards for admissibility 

of analysis for a federal prosecution.37 DHS officials explained that, to the 

extent possible, they publish their research results so that NBFAC’s 

bioforensics techniques can be peer-reviewed, validated, and supported 

in court. Figure 2 shows the broad topic areas and the years in which 

research was solicited through the BAAs.

Figure 2: Broad Topic Areas for DHS R&D, 2008 – 2016

Notes: 

Since 2008 and 2009, BAAs began focusing on methods-based capability development; they are 

included in the figure.

No bioforensics-related BAAs were let in 2010, 2013, and 2014.

 36Contract deliverables included knowledge products, standard operating procedures 

(SOP), technical reports, and publications to support legal admissibility goals. Not all 

deliverables require ISO 17025 accreditation. For example, knowledge products, such as 

information on biological organisms may be published in the open literature and the 

sequence data that is generated would be transitioned directly to NBFAC, according to 

DHS’s response to our questions.

37DHS stated in its responses that the research had resulted in more than 60 peerreviewed scientific publications since 2010, including papers in Genome Biology,Genome 

Research, Nature, Nature Genetics, Nature Biotechnology, PNAS, and Science.
Page 23 GAO-17-177 Bioforensics

Broad and technical topic areas: Based on our review, both the broad 

topic areas in figure 2 and the underlying technical topic areas in the 

BAAs reflect the long-term, methods-based enhancements, and also the 

enhancements to toxin analysis capabilities for the FBI’s current 

casework needs sought by DHS. For example, the following technical 

topics were included as part of the 2015 solicitation for Bioforensics 

research:38

• products to identify select agents including toxins, with high 

confidence,

• next generation and novel technologies to characterize biological 

threat agents for source attribution,

• bacterial populations of select agents with critical knowledge gaps, 

including C. botulinum and B. anthracis (North Africa, Middle East),

• high-confidence methods for metagenomics analysis of complex 

biologicals in complex samples to support whole genome sequencing, 

and

• informatics and statistical tools.

DHS-funded R&D: In line with the broad topic areas indicated in the 

figure—bacterial population genetics, sequence-based approach to 

bioforensics, and bioforensics research—we found that DHS-funded R&D 

contracts include the following areas:39

• population genetics for forensics,

• biological toxin identification,

• metagenomics sequence data,

• statistical confidence in evidentiary materials based on bacterial 

population genetics,

• forensic proteomics of virus production,

• Bayesian taxonomic assignment for next-generation sequencing, and

• sequencing-based bioforensics analyses.

 38OBAA -14-003, Whole Genome Approach to Microbial Forensics (WGAMF), 2014.

39This list does not include all DHS-funded forensic research. For example, we excluded 

chemical forensics R&D projects.
Page 24 GAO-17-177 Bioforensics

R&D supports DHS’s efforts to develop methods-based capabilities, 

including sequencing methods to enable genomic analysis of any 

organism in any sample, as well as bioinformatics methods for de novo

assembly, metagenomic classification, comparative analysis, 

identification of genetic engineering signatures, and the inference of 

biological function. For example, according to DHS’s responses to our 

questions,

• Population genetics: research into population genetics, a 5-year 

timeline project, is published in the open scientific literature, in the 

sequence data in GeneBank; NBFAC used the information to better 

understand the genetic diversity of the organisms studied in that 

project.40 According to documentation we reviewed, results from such 

studies will refine understanding of the population genetics of certain 

select agents to better calculate match statistics in a forensic 

setting.41

• Biological toxin characterization: Regarding toxins, DHS has 

funded contracts to develop SOPs for protein toxin characterization 

using mass spectrometry, among other projects.

• Metagenomics sequencing: DHS research into genetic issues is 

ongoing. DHS is seeking a means for future use of metagenomics 

analyses on complex samples. Regarding metagenomics, DHS plans 

research into high-confidence metagenomics analysis of complex 

biological samples, as well as developing statistical models and 

software to identify the organisms in a complex sample and estimate 

their relative abundance, including developing an existing system for 

probabilistic reconstruction of the taxonomic structure present in a 

metagenomic sample.

• Bioforensic proteomics: DHS has also funded research on 

proteomics—including proteomics of virus production—and analysis 

of proteins and metabolites of unknown samples to complement 

genetic characterization.

 40Bacterial population genetics is the study of the genetic diversity of bacterial 

populations. It attempts to define such diversity in terms of mutation, for example, and 

other factors.

41Velsko, S.P., Bacterial Population Genetics in a Forensic Context: Developing More 

Rigorous Methods for Source Attribution, LLNL-TR-420003 (Livermore, Calif: Lawrence 

Livermore National Laboratory, Oct. 30, 2009).
Page 25 GAO-17-177 Bioforensics

DHS and the FBI are Enhancing NBFAC’s Biological Toxin Analysis 

Capabilities for Current Bioforensics Casework: Based on our review 

and DHS’s responses, DHS’s primary focus is on bioforensics capabilities 

in the short term to address the FBI’s current casework needs. Such 

casework has involved the FBI’s investigation of multiple biocrimes 

involving the use of ricin, including a case in 2013 in which ricin was sent 

to the U.S. President. NBFAC analyzed some of the samples in that case, 

according to the FBI’s responses to our questions.

FBI casework carried out by NBFAC involves the FBI’s transporting 

evidentiary samples to NBFAC, which (1) develops a sample analysis 

plan (which could involve traditional as well as bioforensics analyses) for 

FBI approval, (2) conducts analyses, and (3) reports the results to law 

enforcement, which uses them to inform the bioforensics investigation. 

Based on our review, for prosecution in a case involving ricin, the 

scientific evidence may need to establish that the toxin is present in an 

evidentiary sample.42 We found that a combination of analytical 

capabilities may be used to confirm this, with each detecting a specific 

target. For example, to confirm the presence of ricin in a sample, antibody 

tests, such as ELISA, and mass spectrometry can be used for detecting 

the presence of ricin, and examining the protein, respectively.43 Added to 

them can be cell-free translation assays, another type of antibody test, 

which also detects ricin. We also found that NBFAC’s capabilities for 

analyzing ricin toxins initially included all the independent capabilities 

above, with the exception of an accredited mass spectrometry capability 

 42Possessing a biological toxin to use as a weapon is a crime punishable by monetary 

fines, life imprisonment, or both, 18 U.S.C. § 175(a).Section 178 of Title 18 defines toxin 

as the toxic material or product of plants, animals, microorganisms (including, but not 

limited to, bacteria, viruses, fungi, rickettsiae or protozoa), or infectious substances, or a 

recombinant or synthesized molecule, whatever their origin and method of production, and 

includes (1) any poisonous substance or biological product that may be engineered as a 

result of biotechnology produced by a living organism; or (2) any poisonous isomer or 

biological product, homolog, or derivative of such a substance. 

43Mass spectrometry can identify protein-based toxins by their molecular mass, amino 

acid sequence, and enzymatic activity.

Ricin

Ricin, or Ricinus communis, is one of the most 

poisonous naturally occurring substances. 

Ricin is derived from the beans of the castor 

plant. Ricin is toxic to cells and damages all 

human organs. It is considered a select agent 

(toxin). No antidote is available.

Source: GAO analysis of scientific literature and CDC 

information. GAO rendering of castor beans. | GAO-17-177
Page 26 GAO-17-177 Bioforensics

for characterizing ricin and other toxins.44 Based on our review, we found 

that NBFAC had contracted with a laboratory to examine protein toxins by 

mass spectrometry when it did not yet have that capability. Doing so, 

according to their responses, resulted in a 2—3 day delay, and the 

laboratory was also not accredited under ISO 17025. As a result, the FBI 

further responded that it requested that DHS develop an in-house ISO 

17025 accredited toxin analysis capability at NBFAC. The FBI provided 

the equipment and funding for this transition to NBFAC.45

Enhancing Genomics and Proteomics Is a Long-Term Effort: 

According to a DHS official, DHS is continuing to enhance both genomics 

and proteomics capabilities, which is expected to provide a 

complementary capability that will link proteomic analysis to 

metagenomics analysis of complex samples, thereby providing additional 

information about an agent.46 Further, according to this official, genomics 

and mass spectrometry will support developing metagenomics and 

proteomics. Based on our review, some of the ways in which 

metagenomics capabilities may be used are as follows:

 44An example of such a case is U.S. v. Levenderis, in which the defendant was indicted by 

the federal government in 2011 and convicted in 2014 of possessing a toxin (ricin) for use 

as a weapon under subsection 175(a) of Title 18. NBFAC analyzed the evidence in this 

case with two methods, while an outside contractor laboratory analyzed evidentiary 

samples using a third test, a mass spectrometry method it had developed. NBFAC did not 

then have an accredited mass spectrometry capability. The three assays NBFAC used to 

confirm the presence of ricin toxin were an ELISA test, the Cell Free Translational Assay, 

or CFT, and the matrix-assisted laser desorption and Ionization (MALDI) time-of flight 

(TOF) mass spectrometry analysis. A fourth test using another type of mass 

spectrometry–tandem mass spectrometry analysis–along with that of the ELISA and CFT 

tests, ultimately confirmed the presence of ricin.

45DHS, in responding to our questions, stated that to do this it worked with a national 

laboratory to develop methods and establish mass spectrometry capabilities to support 

NBFAC casework. This effort involved identifying equipment, training subject matter 

experts at NBFAC, and developing SOPs for mass spectrometry for identifying and 

characterizing protein toxins, including ricin.

46DHS responded that, currently genomics can identify the nearest common ancestor of 

an agent. To do this, NBFAC uses procedures involving high-throughput DNA sequencing 

and bioinformatics to identify and characterize a biological sample’s contents by analyzing 

its nucleic acids and comparing them to a reference database. Genomics-based analyses 

may include isolate-level genotyping of bacteria and viruses, metagenomic analysis of a 

wide range of complex samples, de novo sequencing of genomes ranging from viruses to 

mammals, and statistical/bioinformatic analysis of DNA and protein sequences to support 

inference of biological function. 
Page 27 GAO-17-177 Bioforensics

• Metagenomics: It allows sampling of the genomes of microbes 

without culturing them; rather, the DNA is directly isolated from the 

sample before genome sequencing. A DHS official stated that DHS 

plans to provide comprehensive metagenomics analysis of complex 

evidentiary samples. These types of samples may contain both 

microbial and human DNA as well as mixtures that derive from 

possible processing steps (growth media, etc.), which could provide 

links to a possible source.47

• In the context of metagenomics, according to a DHS official, “complex 

samples may be from any environment and can be mixtures of many 

types of organisms. The simplest of metagenomics samples may be 

viruses in the tissue culture which contains the genomes of two 

organisms, the cell line, and the virus. The most complex 

metagenomic samples are from soil samples. Soil samples may 

contain an organism of interest, at low concentration, but also will 

likely have DNA and other biological materials from things such as 

plants, animals, fungi, bacteria and viruses. The ability for the forensic 

laboratory to collect metagenomic data and analyze it relies on the 

development of tools for metagenomics.” In this regard, according to 

an expert who we contacted, metagenomics or the evaluation of 

environmental samples for genetic information is a task that may not 

give DHS the returns from its investment. It is a very time consuming 

technique and should probably be left to academia and/or industry. 

Once these methods are developed, DHS would be able to apply the 

most applicable techniques, according to this expert.

• Figure 3 illustrates the possible composition of a complex sample. 

Metagenomics analysis of a complex sample could reveal the 

presence of DNA and other types of material at different percentages, 

including eukaryotic nucleic acids.48 However, because evaluating 

metagenomic sequence data is based on relative abundances, large 

 47According to a scientist in academia who we interviewed, complex samples include 

those that are generally mixtures that derive from a matrix of possible processing steps 

that include growth (agar, cell culture, carbon and nitrogen sources, complex media), 

separation (heat shock etc.); washing (detergents, water/buffers); drying (air, acetone, 

spray dry, lyophilize); grinding (mortar and pestle, ball mill); and additives (flow enhancers, 

resins, encapsulates, irritants), and the biothreat agent itself.

48Living organisms are included in one of two groups—Eukaryotes or Prokaryotes—based 

on their cell structures. Eukaryotic organisms consist of cells that have a membranedbound nucleus and organelles. All animals are eukaryotes. Others include plants and 

fungi. In contrast, prokaryotic organisms consist of cells that lack a cell nucleus or 

organelle that is encased in a membrane, and they include bacteria.
Page 28 GAO-17-177 Bioforensics

amounts of data may be generated. Interpreting these data and their 

meaning in terms of an agent’s source will be necessary.
Page 29 GAO-17-177 Bioforensics

Figure 3: The Possible Composition of a Metagenomics Sample
Page 30 GAO-17-177 Bioforensics

Based on our review, some of the ways in which proteomics capabilities 

may be used are as follows:

• Proteomics: Proteomics is the study of proteomes. A proteome is a 

set of proteins produced in an organism, system, or biological 

context.49 In response to our questions, DHS advised us that it plans 

to establish a proteomics capability for NBFAC sometime in the future. 

Mass spectrometry is being used for proteomics analysis and is able 

to provide information indicative of a particular protein.

• While proteomics does not replace genomic analysis, it may provide 

additional information if the microbial DNA is too damaged for 

analysis, according to an expert who attended our April meeting. In 

addition, according to this expert, there are differences in microbes 

between those naturally occurring and those grown in laboratories, 

including differences in growth patterns. Proteins express themselves 

based on different food sources. Consequently, according to this 

expert, analyzing the microbes to determine the growth medium used 

could be useful for bioforensics.50 Further, protein profiles have the 

potential to provide information on the environment that the 

microorganism has experienced. For example, cultivation might 

provide information about the skills of the people who grew the 

organisms. Thus, proteomics provides a different level of 

discrimination from that of genomics. Finally, according to another 

expert who we contacted, proteomics analysis should become a 

valuable tool for bioforensics and may rival genetic information when 

methods have matured.

Based on DHS’s responses to our questions, achieving a genomics and 

proteomics capability will also require (1) a bioinformatics and a statistical 

framework for inference and analysis of unknowns in microbial isolates, 

and (2) significantly expanded genome databases and an understanding 

of the underlying determinants of various pathogenic traits. DHS 

responded that both of these are currently funding priorities. In this 

regard, DHS stated that NBFAC continues to expand a major genomics 

capability that includes multiple, complementary sequencing platforms 

 49Proteins are large organic compounds; they are made of a linear chain of amino acids. 

The goal of proteomics is to decipher the structure and function of all the proteins in a cell 

under specific conditions. 

50According to this expert, mass spectrometry is used on the peptides to determine mass 

and sequence, which is then searched against a genomic database to determine where 

that peptide and sequence is found. Which proteins are expressed can show the 

environment where the microbes grew. However, not all proteins are expressed.
Page 31 GAO-17-177 Bioforensics

and advanced bioinformatics within high-performance computing 

environment. In its responses, DHS termed this approach as “agentagnostic” as the analytical procedures require no knowledge of which 

agent might be present in a sample. However, while DHS also stated that 

it provides confidence estimates for aspects of its genome sequencing 

and continues its incremental development of its genomics capability, it 

also acknowledges the need for statistical frameworks.51 For example, it 

stated that “the issues regarding statistical uncertainty require the 

development of statistical frameworks to ensure that attribution signatures 

are clearly defined and understood; that there is standardization, 

validation and verification of the signatures; that relevant source 

populations are fully characterized and understood; the limitations of 

measurement tools are known, and the statistical methods being used are 

appropriate for the signatures data.” Finally, DHS stated in its response to 

our questions, that understanding and communicating of the uncertainty 

is of particular significance when using metagenomics and proteomics on 

complex sample types.

Other experts we interviewed also agreed that there is a need for more 

flexible bioforensics capabilities. For example, an expert from our meeting 

stated that currently, characterizing an agent is achieved by using 

sequence data. Learning what can be exploited for this purpose is in its 

early stages. In addition, a U.K. official we interviewed said that while a 

priority list of organisms will still be needed for responding to emerging 

pathogens and diseases or synthetic biological agents, now a more 

agnostic or “horizon spanning approach” will be used. Nevertheless, not 

all experts were in agreement that DHS should pursue metagenomics for 

bioforensics purposes, at least not in the short-term. For example, in a 

2016 independent assessment of DHS Bioforensics R&D program, 

reviewers recommended a “more measured investment” in metagenomics

and expressed doubt that an operational metagenomics capability was 

likely to be available at NBFAC in 5 years. Instead, they suggested that 

DHS take a more proactive investment stance by following developments 

in the field that were occurring elsewhere.

 51Regarding its current genomic capability, DHS states that it provides confidence 

intervals for aspects of its sequencing and continues its incremental development of the 

capability—for example, assembly quality, alignment, repeat structure, recombination 

history, sequence quality, and phylogenetic tree statistics. DHS’s ongoing incremental 

development is focusing on the following (1) Defining standards for comparative genomics 

in bioforensics (for example, “forensic-grade” SNPs are defined); (2) Increasing sensitivity 

and resolution (for example, ultra-rare variants), and (3) faster and more efficient 

comparative tools for larger datasets.
Page 32 GAO-17-177 Bioforensics

Completion of Capability Enhancements: Based on our review of the 

roadmaps that DHS provided to us regarding the bioforensics 

enhancements, DHS estimates that most of the R&D tasks associated 

with capability enhancements will be completed by 2025 or later, with 

some exceptions. For example, in July 2016, a DHS official indicated that 

DHS’s new mass spectrometry casework capability may be available after 

it has been accredited under ISO 17025 over the next 12 months (in 

2017). In addition, DHS has a 3-5 year focus for developing 

metagenomics so that it can be used on casework. Completion of more 

advanced enhancements will likely extend beyond 2025, according to 

DHS responses to our questions, such as for genomics and proteomics, 

areas that are evolving. Activities include establishing (1) integrated 

processes within metagenomics analyses to facilitate high resolution 

characterization of all agents and nucleic acids in complex samples; (2) a 

bioinformatic and statistical framework for phenotypic inference and 

analysis of “unknown unknown” microbial isolates; and (3) increased 

capabilities to support large-scale proteomic analysis integrated with 

inferential analyses. See appendix II for more details on the BAAs.

DHS is also taking actions to establish an in-house reference collection of 

biological materials—the NBFAC BRC—which will provide materials for 

comparative forensic analyses, assay development and evaluation, and 

proficiency testing. According to DHS responses to our questions, the 

BRC is a long-term storage site for materials acquired from other 

institutions (government, academia, commercial and international 

sources) and NBACC projects. Housed at NBACC, it includes select and 

nonselect agent bacteria and viruses, toxins, and their near neighbors. 

The BRC supports characterization of bacterial and viral agents by 

determining phylogenetic relatedness of different bacterial and viral 

isolates and enabling isolate-level characterizations, which according to 

DHS, is important for isolates that have never been fully characterized or 

sequenced.52 DHS began obtaining a variety of biomaterials such as 

select agents and toxins through subcontracts with government agencies, 

which were stored in external laboratories. In fiscal year 2010, the new 

 52Phylogeny is the evolutionary history of a taxonomic group of organisms. It is essential 

in understanding biodiversity, genetics, evolutions, and ecology among groups of 

organisms. Phylogeny shows the relationships between groups of organisms (taxa) such 

as differences and similarities among them. Phylogeny is represented by a tree diagram 

called a phylogenetic tree. Phylogenetics uses phylogenetic tree diagrams to study 

evolutionary histories and relatedness among various groups of organisms. The 

relatedness between taxa is usually demonstrated through molecular sequencing data 

and morphological data matrices. 

DHS is establishing an InHouse Reference Collection
Page 33 GAO-17-177 Bioforensics

NBACC laboratory opened, after which the collection was consolidated 

within the biocontainment facilities at NBACC and became available for 

use as a reference material. DHS states that it is working with the FBI to 

expand the number of strains of interest in the BRC. In addition, DHS 

projects that develop information on biological organisms are published in 

the open, peer-reviewed literature; sequence data are published in 

GeneBank and are available to the larger community and NBFAC.

DHS’s actions also include developing incrementally a capability for 

identifying and characterizing genetically engineered novel, and unknown 

(emerging or synthetic) agents that uses methods-based capabilities. In 

this regard, NBFAC has developed a genomics capability that DHS 

asserts can be used to infer genetic engineering from DNA sequencing 

and protein sequences. Genetic engineering involves inserting a foreign 

sequence of genetic codes into an existing sequence of genetic codes in 

a target organism with a view to altering some of its functions. DHS states 

it can identify genetic modifications by screening against genes of interest 

(for example, virulence factors or antibiotic resistance genes), comparing 

genome alignments, and comparing regions with unusual sequence 

composition to those typically found in nature.

In the past, restriction enzymes have been used to cut DNA and insert 

specific genes from a different organism to produce a desired effect (for 

example, producing human insulin using bacterial cells), which results in 

“scarring” at the restriction sites. Genome characterization and analysis 

according to DHS’s and the FBI’s responses to our questions, 

respectively, would be able to detect such scarring. However, gene 

editing techniques are evolving and may be harder to detect. Clustered, 

regularly interspaced, short palindromic repeats (CRISPR) Cas-9—is now 

available to researchers. It engineers microbes by inserting genes, 

although unlike previous methods the restriction sites may not be evident, 

and the enzymes used will not cause scarring.53 Figure 4 is a simple 

illustration of genetic engineering using restriction enzymes.

 53Using CRISPR involves a piece of RNA (a chemical messenger, which can be used to 

recognize a target section of DNA) and an enzyme called a nuclease that can snip 

unwanted genes out and paste in new ones.

DHS is Developing a Capability 

to Identify and Characterize 

Engineered, Novel, and 

Unknown Agents
Page 34 GAO-17-177 Bioforensics

Figure 4: Genetic Engineering with Restriction Enzymes

Note: Figure for illustrative purposes only.

Experts from our meeting, as well as other experts we interviewed, 

indicated that identifying genetic engineering could be approached by 

determining an agents’ virulence and then using capabilities such as 

mass spectrometry to identify if elements exist that suggest modification. 

Nevertheless, it was thought that a genetically engineered agent would 

have some parts that remain unchanged, which would help to determine 

its characteristics. For example, according to an expert at our meeting, 

the focus should be on (1) identifying those introductions of additional 

virulence or genetic elements into an organism, which can be done fairly 

quickly, and then determining if there are other elements that suggest 

somebody has modified the organism and (2) using methods like mass 
Page 35 GAO-17-177 Bioforensics

spectrometry, microscopy, and other methods that can identify the means 

of production or culture and dissemination or delivery of the organism. In 

addition, according to a U.K. expert we interviewed, the core part of the 

genome in an engineered agent would not be changed, and it also must 

reproduce and metabolize. If it is an engineered virus, it would have some 

similarities to other viruses, such as in how it attaches itself to a cell to 

propagate its genome. So within its genome some signatures would be 

available for comparison. Further, this expert stated that even if the 

organism was a synthetic one and CRISPR Cas-9 had been used, he 

would still look to see whether any scarring was present (see figure 4).

Regarding synthetic agents, DHS asserts that they can be analyzed 

similarly to those that are genetically engineered, with the addition of 

NBFAC’s inferential analysis capability, whose analysis will provide clues 

to the functionality of a synthetic agent.54 DHS is developing a capability 

that will allow NBFAC to characterize unique, novel agents, “unknowns” 

(emerging or synthetic organisms) and “unknown, unknowns” (de novo

synthetic organisms). However, achieving this capability will also require 

a bioinformatics and a statistical framework for inference and analysis of 

unknowns in microbial isolates and expanded genome databases, 

according to DHS.55 DHS indicates that it is developing a “multi-layered 

inferential analysis capability” that would include establishing comparative 

methods for the analysis of any DNA or protein sequence to identify such 

things as peptides, restriction sites, and a statistical model that allows 

confidence estimates to be placed on these analyses.

DHS faces numerous challenges as it attempts to enhance its 

bioforensics capabilities. Our review of agency documentation and related 

literature, and interviews with agency officials, scientists, and subject 

matter experts at our meeting and elsewhere, as well as our prior work, 

indicate that challenges must be overcome if DHS is to develop enhanced 

 54In this type of analysis (1) sequence data would be compared to all known sequences in 

the database, (2) potential protein-coding regions in the sequence would be compared to 

all known proteins in the database, and (3) codon adaptation could be inferred by 

comparing the predicted protein matches to the nucleic acid matches for a sequence. 

(Codon is a sequence of three adjacent nucleotides forming a unit of genetic code that 

determines the insertion of a specific amino acid in a polypeptide chain during protein 

synthesis or the signal to stop protein synthesis).

55A statistical framework allows for statistically meaningful comparative analyses; it is a 

set of concepts and organizing principles that support the compilation and presentation of 

a set of statistics. 

DHS Faces Numerous 

Challenges in Enhancing 

Its Bioforensics 

Capabilities
Page 36 GAO-17-177 Bioforensics

capabilities suitable for bioforensics. These include capabilities that not 

only can be relied on for identifying and characterizing known agents but 

also those that have been genetically engineered, or are unique, novel or 

unknown (emerging or synthetic). Challenges that DHS faces include 

(1) the ability to interpret and communicate results with defined statistical 

confidence, (2) obtaining access to quality references and databases for 

bioforensics analysis, and (3) the effect of the evolving threat landscape 

on future casework needs. Further, their results must be also able to 

stand up to court scrutiny.

DHS plans to develop advanced metagenomics and proteomics 

capabilities. However, it is not clear to what extent or when DHS will be 

able address key challenges related to enhancing its bioforensics 

capabilities that include interpreting results from metagenomics and 

proteomic analyses, with a defined statistical confidence, according to 

both DHS and the FBI officials. Further, communicating the uncertainty 

associated with the results will be particularly important when using these 

capabilities on complex sample types.

Without a defined level of statistical confidence, the probative value of 

inferences made from the results of such analyses may not be known.56

In general, as the level of statistical confidence increases in these 

inferences—signifying a higher degree of scientific certainty—the 

probative value of the inference also increases. In figure 5, we have 

extracted and expanded on one dimension of “the forensic continuum” 

that has been used to represent the evaluation and analysis of a 

bioforensics sample and its probative value. As indicated in the figure, 

probative value depends on confidence in analysis and the interpretation 

and meaning of evidence.

 56Probative value refers to evidence that helps prove a fact or an issue.

DHS Faces Challenges in Its 

Ability to Interpret and 

Communicate Analyses 

Results with Statistical 

Confidence
Page 37 GAO-17-177 Bioforensics

Figure 5: The Dependence of Probative Value on Confidence in Analysis, Interpretation, and the Meaning of the Evidence

Note: Figure for illustrative purposes only.

Investigative leads or bioforensics data may rely on the use of 

bioinformatics and data and inferences using those data. In this regard, 

according to a DHS official, an issue DHS continues to struggle with is 

how to interpret metagenomics analysis—whether it is possible to define 

with certainty whether a piece of the genome of an agent is present—

versus defining the error rates for each sequencing base call, which DHS 

can do.

DHS solicitations for R&D reflect some of these challenges, including the 

following extract from a related 2012 BAA solicitation regarding 

interpreting the results of metagenomics analyses:

“Currently, it is difficult to assign confidence to the results of 

metagenomic analyses. For example, in metagenomic 

sequencing, what do a small number of reads that match a 

particular organism say about the probability that the organism is 

actually present in the sample? New methods are needed to 

assess the likelihood that an organism is present in a 

metagenomic sample and to provide confidence intervals on 

abundance estimates.
Page 38 GAO-17-177 Bioforensics

Bioforensics R&D is looking to invest in the development and 

application of mathematical models for (1) estimating the 

likelihood of a genome being present in a metagenomic sample, 

and (2) the most likely composition of a metagenomic sample 

including a list of genomes and their relative abundance. The 

system should go beyond metagenomic classification to provide a 

statistically supported estimate of sample composition that could 

be used in a biothreat agent detection context.” 57

Based on our review, we found that analysis of metagenomics data sets 

will rely on advanced bioinformatics analyses that involve a large 

statistical component. However, forensic casework may involve mixtures, 

and separating these into individual components may be difficult—a 

problem that may also apply to metagenomics. In this regard, challenges 

involve the development and applications of appropriate bioinformatics 

and data that will provide the ability to not only to describe the relative 

abundance of sequence data but also to make inferences using those 

data to provide either an investigative lead or to support attribution. 

Efforts to achieve this are complex and will be conducted over multiple 

years. For example, according to NBACC’s 2015 annual plan, “a 

sequence-based, bioinformatics-driven genomics approach is a complex 

endeavor that requires incremental implementation of critical technologies 

over multiple years.”

Regarding proteomics, challenges remain in interpreting data. For 

example, according to experts at our meeting, a quantitative measure for 

proteomics needs to be available so that an informed decision can be 

made. However, this is complicated by the lack of a framework for 

expressing confidence in a result. Further, related to data analysis and 

interpretation, for example, the potential for rapidly expanding protein 

databases to result in false matches exists and the lack of standardized 

approaches to proteomic data analysis is problematic.58

DHS must address several challenges related to its reference materials 

that could affect NBFAC’s comparative analysis of evidentiary samples. 

According to DHS’s responses to our questions, these include access to 

 57Broad Agency Announcement BAA 12-11, Whole Genome Approach to Microbial 

Forensics (WGAMF), Department of Homeland Security, S&T Directorate. 

58Bruce Budowle and others, Microbial Forensics, 2nd ed. (Burlington, Mass.: Academic 

Press, 2011), p 457.

DHS is Challenged by Data 

Quality and Accessing Agent 

Strains for the BRC
Page 39 GAO-17-177 Bioforensics

reference strains of interest, international agents, and ensuring the quality 

of the data in the BRC. During our review we found that in contrast to 

human DNA—a single species—the challenges for bioforensics involve a 

multitude of species. Further, the quality of the data entered into a 

particular database, including the metadata, and whether the database is 

kept up to date could affect analysis if NBFAC uses that database. In 

addition, ensuring that agents of interest are available for comparative 

analyses is necessary.

Regarding the BRC specifically, not all strains are readily available and 

obtaining agents internationally raises issues, according to DHS 

responses. Further, not all researchers are willing to share their strains. 

As a result, DHS is working with the FBI to develop an acquisition and 

curation plan to expand the number of strains of interest in the BRC.59

A DHS official stated that replacing agent-specific assays with DNA 

sequencing methods will require DHS to have a comprehensive, 

sophisticated database, which it currently does not have. Therefore, 

ensuring the usefulness and quality of its reference collection and its 

ability to obtain the strains of interest will continue to be a challenge for 

DHS.

Experts at our meeting and others we interviewed identified two key 

challenges associated with enhancing bioforensics capabilities: 

(1) accessing and maintaining quality data on global microbial species in 

databases and (2) implementing statistical frameworks and acceptable 

communications of statistical analyses in court.

Reference databases and quality data: Experts and officials in both the 

United States and the United Kingdom whom we interviewed had differing 

opinions about the challenges associated with obtaining access to global 

microbial species and maintaining quality data for comparative analyses 

of samples. For example, some indicated the need to establish and 

maintain and a central database, whereas others considered it necessary 

only to have the ability to access one that is relevant when an incident 

occurs. Experts at our meeting also expressed reservations about 

whether a centralized system is the best solution. They stated that it is 

possible that a hybrid system in which each organization would own its 

own dataset but would allow it to be searched by others might be a 

 59Digital curation is the selection, preservation, maintenance, collection, and archiving of 

digital assets. Digital curation establishes, maintains, and adds value to repositories of 

digital data for present and future use. 

Experts Identified Two Key 

Challenges to Enhancing 

Bioforensics Capabilities
Page 40 GAO-17-177 Bioforensics

possible solution. Alternatively, organizations could be required to send 

their data to a central database in addition to storing it locally.

Questions have been raised about how reference data can be used 

effectively for bioforensics. For example, regarding the use of population 

genetics, it has been observed in the literature that a more useful 

database for each pathogen would consist of a detailed record of human 

and enzootic outbreaks noted through international outbreak surveillance 

systems, and “representative” genetic sequences from each outbreak.60

Another expert suggested that if necessary one could individually forage 

and collect organisms of interest in relevant areas or countries. Further, 

according to these experts, much is known about Bacillus anthracis, but 

other organisms like Burkholderia are much more challenging. To solve 

this problem would involve first going back to close the gaps in the 

reference databases and the population genetics. 

Another challenge involved concerns about managing the quality of the 

data entered into the database to ensure it meets quality standards. 

Regarding the quality of such references for bioforensics, standards are 

needed for data repositories and reference collections of pathogens and 

other microorganisms, according to the experts at our meeting. Also, 

because of the uncertainty about the reference data, the meeting’s 

experts stated that the raw data should be maintained for further 

analyses. In addition, according to these experts, questions about the 

meaning of the data for these applications and the confidence value of 

the data need to be resolved before focusing on them for bioforensics 

purposes. According to a U.K. official we interviewed, the level of 

uncertainty in matching microbes cannot be quantified, and attribution 

depends on a reference set, which is incomplete for microbes. It could be 

concluded that the microbe in question has the same DNA as that of a 

microbe in the reference database, but not with certainty that it would not 

match another microbe that is not in the reference database. Because of 

the limits of using one approach, it is important to also use traditional 

forensics to build an evidence base. In court, traditional forensics, in 

addition to expert testimony on bioforensics, would therefore be used for 

attribution.

Statistical frameworks and communicating results: As we reported in 

2014, a statistical framework allows for statistically meaningful 

 60Velsko, Bacterial Population Genetics, p. 3.
Page 41 GAO-17-177 Bioforensics

comparative analyses; it is a set of concepts and organizing principles 

that support the compilation and presentation of a set of statistics.61

Experts at our meeting expressed the view that a gap that permeates

science, capabilities, and bioinformatics is the lack of a formulation or 

framework for expressing confidence in genomics results as well as 

similar challenges with non-genetic results. This is especially true with 

mixed, metagenomics samples. Further, how to combine or communicate 

uncertainties and error rates associated with the analytical and collection 

processes needs greater clarity, according to the experts at our meeting. 

They stated that to have a more robust statistical foundation, it is critical 

to do enough experiments to assess the various contributions of these 

sources of variability. NBFAC is moving into the realm of metagenomics, 

which has problems with the statistical unknowns associated with it. 

Metagenomic samples may contain mixtures as we stated previously. In 

this regard, these experts stated that the problem of mixtures is an 

opportunity for statistical methods to improve the results for different kinds 

of evidence. For example, is the evidence confirmatory; is it consistent 

with what’s in the database, and what are some possible alternatives that 

could have given rise to the evidence? In addition, these experts stated 

that challenges in a bioforensics context include the need for a 

quantitative measure for genetics, proteomics, or other methods so that 

an informed decision can be made. 

Communicating results using statistical probabilities may not always be 

acceptable by courts, despite the need for statistical frameworks to assist 

in interpreting bioforensics analyses. Even if accepted, such statistical 

information may not be understood. This issue is important because 

statistics could play a large part in some types of analyses. For example, 

using statistics when communicating the results of human DNA analysis 

are generally accepted by courts in the United States. For interpretation 

of bioforensic results, according to the experts at our meeting, the 

question should be: What is the confidence you have achieved with the 

data or information that you have? How that confidence is communicated 

is important. It is forensic evidence, a piece of the puzzle. It adds value, 

 61See GAO-15-80, p. 33. As we reported, the significance of such statistical inference 

relies on the analyst’s ability to quantify both the confidence in test results and the 

frequency with which results match. Confidence, in this context, refers to the level of 

reliability and accuracy investigators assign to the test results obtained from the 

measurement tools used to identify the properties of interest in the samples. The 

frequency of the sample properties’ presence, or generation in a relevant population of 

possible sources, is a measure of how common or rare the properties are and provides 

context for the probative value of the evidence. 
Page 42 GAO-17-177 Bioforensics

but the confidence for it may be low, whereas the confidence for some 

other evidence may be high. The level of uncertainty for that result should 

also be indicated. Thus, bioforensic analyses of microbial DNA, and its 

associated statistical elements, may have to overcome many obstacles 

before they reach a similar level of acceptance in a U.S. federal court.

Other experts have acknowledged such challenges. For example, 

regarding the U.S. legal system, issues may arise when new methods are 

applied for the first time in bioforensics that have not undergone the depth 

of scrutiny undergone by traditional forensic techniques. Also, reference 

databases used for comparisons take time to develop.62 Our interviews 

with U.K. scientists and government officials provided some insights into 

issues associated with human DNA analysis results, which are generally 

accepted by courts and that could have implications for bioforensics in 

both the United Kingdom and the United States. For example, regarding 

the communication of probability data, according to a U.K. official we 

interviewed, while academics say there is some set level of probability to 

achieve “beyond a reasonable doubt,” the court is concerned with the 

baseline probability. However, this official stated that human DNA is the 

only science in which the baseline probability data is considered 

incontrovertible. Further, he said that in almost every other science, the 

legal system would want assurances from expert witnesses regarding the 

analysis results—not the numerical scientific results themselves.

In light of the discussion above, it is not clear how long it will take for the 

results of metagenomics and proteomics analyses to be acceptable to 

courts. Nevertheless, what is clear is that the ability to quantify statistical 

uncertainty will require the use of comprehensive databases that contain 

characteristics of signatures and information on the variations in the 

population of the agent in question. 

A long-term challenge facing NBFAC, according to DHS’s responses to 

our questions, is the increasingly complex biological threat landscape: 

New infectious disease agents emerge every year, and advances in 

genetic engineering and “do it yourself” biology methods make the 

nefarious use of enhanced and biological agents a possibility. DHS 

further responded that as a result NBFAC must regularly establish new 

methods and assays to support bioforensics casework that may involve 

 62John M. Butler, Advanced Topics in Forensic DNA Typing: Methodology, (Waltham, 

Mass, Academic Press, Feb. 2012). 

The Evolving Threat 

Landscape Raises Questions 

about the Bioforensics 

Capabilities That Will Be 

Needed
Page 43 GAO-17-177 Bioforensics

future threats. Further, we found it is still challenging to distinguish 

between a natural and a deliberately released organism. However, 

according to the experts at our meeting, when using “natural,” “accidental” 

and “deliberate,” the issue could be more to do with the means by which 

an agent is used, or about the characteristics of the agent itself. 

Determining intent is likely to rely on information beyond the science 

alone. In addition, while epidemiologic tools determine whether something 

is unusual, what is also needed is a defined and validated tool that will 

determine whether a microbe is unusual, made by humans, cultured, or 

engineered.

Although DHS is developing capabilities to detect manipulated agents, it 

faces several challenges related to the perceived potential for the creation 

of agents that could cause harm accidentally or intentionally. To identify 

and characterize novel synthetic agents, these challenges go beyond 

identifying changes in an agent’s genomics (such as its antibiotic 

resistance). DHS, in its responses to our questions, stated that identifying 

more complex traits is more difficult because of the current scientific 

understanding of how these processes work at the molecular level. DHS 

also responded that both genetic engineering and NBFAC’s ability to infer 

its intended effects require a deeper understanding of the physiology of 

the biological agent as well as its interaction with a human host. Concerns 

have also been raised about the potential for gain of function research to 

result in manipulation of microbial agents with the potential for causing 

harm.63 Such manipulations could involve, for example, agents or toxins 

in which harmful consequences have been enhanced, such as making 

them antibiotic resistant, more virulent, or more transmissible to 

humans.64 The use of CRISPR Cas-9 also raises other issues because it 

may be more difficult to detect than were previous gene editing 

 63Gain of function generally refers to changes that result in either enhancement of 

acquisition of new biological phenotypes or functions. National Academies of Sciences, 

Engineering, and Medicine, 2016. Gain-of-Function Research: Summary of the Second 

Symposium, (Washington D.C.: Mar. 10-11, 2016), p. 14.

64Dual use research of concern guidelines state that research that is intended to do the 

following would be of concern: (a) enhances the harmful consequences of the agent or 

toxin; (b) disrupts immunity or the effectiveness of an immunization against the agent or 

toxin without clinical or agricultural justification; (c) confers to the agent or toxin resistance 

to clinically or agriculturally useful prophylactic or therapeutic interventions against that 

agent or toxin or facilitates their ability to evade detection methodologies; (d) Increases 

the stability, transmissibility, or the ability to disseminate the agent or toxin; (e) alters the 

host range or tropism of the agent or toxin; (f) enhances the susceptibility of a host 

population to the agent or toxin; or (g) generates or reconstitutes a listed eradicated or 

extinct agent or toxin. 
Page 44 GAO-17-177 Bioforensics

approaches. However, not all agree that the risk of possible misuses of 

biology is significant.

Regarding the use of genetically engineered agents to cause harm and 

the likelihood of this becoming a problem, we found differences in the 

views of the experts we spoke to both here and in the United Kingdom. 

Some said that it is difficult to create new pathogens so the use of 

naturally-occurring microbes is of the greatest concern. While 

acknowledging there are many technically possible misuses of biology, 

they concluded that it is far more likely that minor modifications would be 

made to existing organisms rather than the creation of new ones.65

According to an expert we contacted, developing a new microbe with 

novel pathogenic characteristics or antibiotic resistance significantly more 

difficult than introducing these characteristics by gene manipulation. 

Thus, one of the challenges DHS faces is to consider the risks in relation 

to not only the bioforensics capabilities it needs but also its strategy for 

addressing current and potential threats.66

DHS has identified some bioforensics capability gaps since 2010 using an 

informal, undocumented process but has not systematically identified the 

gaps or performed a bioforensics capability gap analysis. In the absence 

of a bioforensics gap analysis demonstrating the existence of gaps, it is 

difficult to determine whether DHS has identified all its capability needs 

and gaps. Identifying gaps and prioritizing bioforensics capability needs 

and gaps can help guide the proper allocation of resources to the highest 

priority needs. Therefore, without a capability gap analysis and 

documentation of the results of its process for identifying gaps, the 

rationale for DHS’s resource allocations and its plans for future 

enhancements to its existing capabilities are not clear.

 65For example, a UPMC paper stated that, “though there are many technically possible 

misuses of biology, most of the scientists we interviewed thought it unlikely that either 

individuals or small groups would adopt such approaches. ‘Old bugs,’ such as anthrax, 

tularemia, and foot-and-mouth disease (FMD), were the biggest worries among those 

interviewed. In general, they think there are enough relatively simple paths to making a 

biological weapon to render more technically difficult approaches unattractive and, 

therefore, less likely to be pursued.” Center for Biosecurity of UPMC, The Industrialization 

of Biology and Its Impact on National Security (Baltimore, MD: June 8, 2012).

66GAO has ongoing work on DHS’s efforts to characterize biological threats.

Conclusions
Page 45 GAO-17-177 Bioforensics

We recommend that the Secretary of Homeland Security—in consultation 

with the Federal Bureau of Investigation—conduct a formal bioforensics 

capability gap analysis to identify scientific and technical gaps and needs 

in bioforensics capabilities to help guide current and future bioforensics 

investments and update its analysis periodically.

We provided a draft of this report for review and comment to DHS and the 

FBI. DHS provided written comments, which are reproduced in appendix 

IV. DHS concurred with our recommendation. The FBI did not provide

comments. Neither DHS nor the FBI provided technical comments. In its 

response, DHS described actions it plans to take to address the 

recommendation. Specifically, according to DHS, S&T's Homeland 

Security Advanced Research and Projects Agency's Chemical and 

Biological Defense (CBD) Division has initiated a formal, welldocumented capability analysis of its Bioforensics R&D program. Further, 

DHS stated that CBD will leverage this analysis to conduct a parallel 

capability analysis of the Chemical Forensics and Attribution program that 

addresses similar analytical and attribution needs for chemical threat 

agents. DHS states that the CBD Division staff has prepared newly 

updated Operational Requirements Documents and Strategic Plans 

(Fiscal Years 2017-2021) for both programs, although we have not 

reviewed these documents. According to DHS, the CBD Division initially 

identified and compiled a number of bioforensics capability needs from a 

review of external programs and meetings with end-users, such as the 

FBI, and it is identifying and grouping additional needs under three areas 

(science, technology and methods, and bioinformatics and data) through 

reviews of documents, such as the National Research Council, Science 

Needs for Microbial Forensics, 2014, and GAO’s report, Anthrax: Agency 

Approaches to Validation and Statistical Analyses Could be Improved

(GAO-15-80), among others. According to DHS, the CBD Division is 

conducting the formal capabilities analysis using methods and best 

practices identified in the documents that include the DHS Instruction 

Manual 107-01-001-01, DHS Manual for the Operation of Joint 

Requirements Integration and Management System, April 21, 2016; 

DHS S&T "Requirements Development Guide" April 2008; and GAO’s

reports, Program Evaluation: Experienced Agencies Follow a Similar 

Model for Prioritizing Research (GAO-11-176) and Chemical, Biological, 

Radiological, and Nuclear Risk Assessments: DHS Should Establish 

More Specific Guidance for Their Use (GAO-12-272). Finally, according 

to DHS, the CBD Division is consolidating and prioritizing these needs to 

ensure that they are in alignment and harmonized with current research 

goals and strategic plans within DHS, S&T, Homeland Security Advanced 

Recommendation for 

Executive Action

Agency Comments 

and Our Evaluation
Page 46 GAO-17-177 Bioforensics

Research and Projects Agency, and the CBD Division. DHS plans to 

complete these efforts by June 30, 2017, and states that the CBD 

Division will ensure that the formal analysis is updated on an annual basis 

and is used to guide current and future bioforensics investments.

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies of this report to the 

Secretary of Homeland Security and the Director of the FBI, appropriate 

congressional committees, and other interested parties. The report is also 

available at no charge on GAO’s website at http://www.gao.gov.

If you and your staff have any questions about this report, please contact 

Timothy M. Persons, Ph.D. at (202) 512-6412 or personst@gao.gov. 

Contact points for our Office of Congressional Relations and Office of 

Public Affairs may be found on the last page of this report. Key 

contributors the report are listed in appendix V.

Timothy M. Persons, Ph.D. 

Chief Scientist
Appendix I: Objectives, Scope, and 

Methodology

Page 47 GAO-17-177 Bioforensics

For this report, we evaluated (1) the extent to which DHS and the FBI 

have identified gaps in their bioforensics capabilities since 2010, 

(2) bioforensics needs experts have identified, and (3) any actions DHS 

and the FBI have taken to enhance their bioforensics capabilities, 

including those for characterizing a novel synthetic biological weapon, 

and any challenges they have experienced in enhancing bioforensics

capabilities.

To determine the extent to which DHS and the FBI have identified gaps in 

their bioforensics capabilities, we reviewed agency documents and 

interviewed relevant agency officials about their efforts to identify such 

gaps since 2010, which is when the Department of Justice closed the 

FBI’s investigation into the 2010 anthrax case. We examined agency 

planning documents, such as DHS’s Strategic Plan 2015-2019 and 

NBFAC’s Bioforensics Roadmap for research, among others. We 

reviewed DHS policy and guidance, such as DHS’s Joint Requirements 

Integration and Management System, which formed the basis for the 

criteria we used to compare and assess the extent to which DHS had 

identified capability gaps or conducted a capability gap analysis of its 

bioforensics capabilities. We also interviewed agency officials, including 

those with DOD, to determine whether any gaps had been identified that 

related to bioforensics and their interactions with DHS in this regard.

We developed a list of bioforensics needs that experts had identified. To 

do this, we identified capabilities that might be needed for bioforensics 

purposes from a 2014 NRC publication entitled Science Needs for 

Microbial Forensics: Developing Initial International Research Priorities

and the 2009 National Research and Development Strategy for Microbial 

Forensics from the National Science and Technology Council (NSTC). 

We excluded capability needs in the literature that were not related to 

science and technology development as these would have been beyond 

our scope. We grouped the remaining capability needs into three broad 

areas: (1) science, (2) technologies and methods, and (3) bioinformatics 

and data. We then convened, with the assistance of the National 

Academy of Sciences (NAS), a 2-day meeting of 16 experts to discuss 

and update the capability needs we identified, including identifying issues 

related to these needs. To identify the experts appropriate for the 

meeting, we worked iteratively with NAS staff to identify and review 

biographical information and relevant qualifications of experts, as well as 

factors such as representation from academia, industry, and expertise in 

a range of areas. The Board on Life Sciences of NAS solicited 

nominations for the expert panel from its extensive contacts in the 

microbial forensics area. From this initial list, NAS selected experts based 

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 48 GAO-17-177 Bioforensics

on their knowledge and expertise in forensics, microbiology, molecular 

genetics, non-genetic methods, genetic engineering, bioinformatics, 

statistics, and legal issues related to bioforensics. Once we came to 

agreement with NAS on the final list of 16 experts for the meeting, these 

experts were evaluated for any conflicts of interest. A conflict of interest 

was considered to be any current financial or other interest that might 

conflict with the service of an individual because it (1) could impair 

objectivity and (2) could create an unfair competitive advantage for any 

person or organization. We discussed internally all potential conflicts. The 

experts were determined to be free of conflicts of interest, and the group 

as a whole was judged to have no inappropriate biases. See appendix III 

for a list of the experts. The meeting was recorded and transcribed to 

ensure that we accurately captured the experts’ statements, and we 

reviewed and analyzed the transcripts as a source of evidence. We 

developed the session topics based on our researchable objectives and 

issues that were identified in our audit work. The session topics were 

gaps in the science underpinning bioforensics capabilities, gaps in 

capabilities (technologies) and methods for attribution, and gaps in 

bioinformatics, data and statistical Interpretation of bioforensics. We 

subsequently obtained their comments on the list of capability needs 

identified during the April 2016 meeting to update and amend it based on 

their input.

To determine the actions DHS and the FBI had taken to enhance their 

bioforensics capabilities since 2010 and any challenges they 

encountered, we reviewed agency documents, including planning 

documents and research and development (R&D) efforts. We also 

examined DHS’s actions to enhance NBFAC’s capabilities for the long 

term as well as for the FBI’s casework. We reviewed DHS’s Broad Area 

Announcements (BAA) and Open Broad Area Announcements (OBAA) 

from 2008 to 2016. These are the mechanisms by which DHS solicits 

research to develop its bioforensics capabilities. We obtained details on 

contracted external R&D efforts. Deliverables included statistical models,

SOPs, and genetic sequences from external researchers. To determine 

any challenges to enhancing bioforensics capabilities, we reviewed 

agency documentation, including planning and contract documentation, 

related literature, and our prior work on bioforensics. We interviewed 

agency officials and scientists, including those at DHS, DOD, and the FBI 

and obtained the opinions of experts in the United Kingdom, which 

collaborates with DHS and the FBI on bioforensics-related issues, as well 

as those in the United States regarding bioforensics-related challenges. 

We also discussed potential challenges with experts present at our expert 

meeting. We conducted site visits to national laboratories and academic 
Appendix I: Objectives, Scope, and 

Methodology

Page 49 GAO-17-177 Bioforensics

institutions conducting research on bioforensics-related issues, including 

issues related to synthetic biology. These included discussions with DHS 

contractors, scientists in academia, officials from the U.K. Home Office, 

Public Health England at Porton Down, and scientists in academia 

regarding challenges related to bioforensics capabilities. We also 

interviewed some of the scientists involved in conducting research for 

DHS.

We conducted this performance audit from July 2015 to January 2016 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.
Appendix II: DHS R&D to Enhance 

Bioforensics Capabilities, 2008 – 2016

Page 50 GAO-17-177 Bioforensics

Table 3: Broad and Technical Topic Areas in Broad Area Announcements (BAA), 2008 – 2016 

Yeara Subject area Technical focus areas for R&D

2008 Bacterial population 

genetics in a forensics 

context

• Goal of program is to develop algorithms and analytical tools that will provide precision 

and statistical power to inferences concerning the degree of relatedness (based upon 

matching comparisons) of agents

• Research on genome stability, host preferences and interactions, genetic mobility of 

virulence factors, identification of polymorphic sites and mutational hot spots, 

geographical distribution, and effects of host–pathogen interaction

• Methods for determining rates of mutation and recombination of the pathogen genomes 

and the identification of adaptive mutations that can have forensic utility

• Establishing match criteria for discriminating “difference” or “sameness” in sample 

comparisons

• Developing statistically rigorous sampling strategies to acquire spatially referenced 

genetic information on reservoirs of these pathogens

• Developing bioinformatics-based analytical tools for supporting hypotheses testing 

regarding pathogen origin that go beyond current phylogeny-based inferential methods 

and can meet forensic (legal) admissibility

2009 Bacterial and viral 

bioforensics research 

and development

• Develop novel techniques to culture threat agents from complex environmental 

samples

• Improve dry collection and extraction strategies for forensic samples

• Develop detection methods for rare variant detection in a bacterial sample using ultrahigh throughput next generation sequencing technology

• Understand dynamics of mobile elements in select agent bacteria

• Develop forensic genotyping methods for select agent viruses

• Develop novel applications of orthogonal methods to genetic characterization of

biological threat agent signatures and their sample matrices

2011 Sequence-based 

approach to 

bioforensics analysis

• Develop biased primer set design to amplify biological threat agents from complex 

backgrounds

• Production methods for ultraclean reagents

• Sequence data error model for next-generation and single molecule sequencing 

platforms

• Taxonomic classification of metagenomic sequences

2012 Whole genome 

approach to microbial 

forensics

• Develop and apply mathematical models for statistical confidence measurements in 

metagenomic analysis

• Develop a procedure to transport agents from BSL-3 to BSL-2 laboratories

• Produce whole-genome sequencing to capture the global biodiversity of human, plant, 

and animal pathogens (bacterial, viral, and fungal) in support of microbial forensics 

analysis

• Development and population of comparative genomic database with pathogen 

sequence data at the National Center for Biotechnology Information Center

2015 Bioforensics research 

R&D: Whole genome 

approach to microbial 

forensics

• Products to identify select agents including C. botulinum toxins, with high confidence

• Next generation and novel technologies to characterize biological threat agents (the 

organism, the agent, or the sample matrix) for source attribution

• Research on the bacterial populations of select agents with critical knowledge gaps, 

including C. botulinum and B. anthracis (North Africa, Middle East)

Appendix II: DHS R&D to Enhance 

Bioforensics Capabilities, 2008 – 2016
Appendix II: DHS R&D to Enhance 

Bioforensics Capabilities, 2008 – 2016

Page 51 GAO-17-177 Bioforensics

Yeara Subject area Technical focus areas for R&D

2016 Whole genome 

approach to microbial 

forensics

• Identify and sequence near neighbors of Francisella tularensis

• Metagenomics analysis of complex biological samples

• High confidence metagenomics analysis of complex samples

Source: GAO analysis of DHS BAA documentation. | GAO-17-177

a

Year is w hen the BAA was published.
Appendix III: List of Experts

Page 52 GAO-17-177 Bioforensics

The names and affiliations of the experts who participated in the group 

meeting held April 20-21, 2016, in Washington, D.C. are as follows:

• Christopher Bidwell, J.D., Senior Fellow for Nonproliferation Law and 

Policy at the Federation of American Scientists.

• Bruce Budowle, Ph.D., Professor, Executive Director of Institute of 

Applied Genetics Molecular and Medical Genetics, University of North 

Texas Health Science Center.

• Rockne Harmon, J.D. Consultant, Instructor at U.C. Davis in the 

Masters in Forensic Science program.

• Dag Harmsen, MD, Ph.D., Professor, Head of Research, Center for 

Oral and Maxillofacial Surgery, Department of Periodontology, 

University of Munster.

• Molly Isbell, Ph.D., Director of Quality Assurance and Statistical 

Sciences. Signature Science, LLC.

• Dana Kadavy, Ph.D., Director of Biological Services, Signature 

Science, LLC.

• Karen Kafadar, Ph.D., Commonwealth Professor and Chair of 

Statistics, University of Virginia.

• Paul Keim, Ph.D., Regents’ Professor in Biology Cowden Endowed 

Chair in Microbiology Northern Arizona University’s Microbial, 

Genetics and Genomics Center. Northern Arizona University.

• Jack Melling, Ph.D. (via phone), Consultant.

• Stephen S. Morse, Ph.D., Professor, Epidemiology, Founding Director 

and Senior Resident Scientist, Center for Public Health Preparedness, 

Columbia University,

• Karen Nelson, Ph.D., President, The J. Craig Venter Institute.

• David Relman, MD, Thomas C. and Joan M. Merigan Professor in 

Medicine, and Microbiology and Immunology, Co-Director of the 

Center for International Security and Cooperation, Stanford University, 

and Chief of Infectious Diseases, the Veterans Affairs Palo Alto 

Health Care System.

• Tom Slezak, Ph.D., Associate Program Leader for Informatics for the 

Global Security Program Efforts, Lawrence Livermore National 

Laboratory.

• Stephen Turner, Ph.D., Assistant Professor of Public Health Sciences, 

University of Virginia School of Medicine.

Appendix III: List of Experts 
Appendix III: List of Experts

Page 53 GAO-17-177 Bioforensics

• Stephan Velsko, Ph.D., Senior Scientist and Associate Program 

Leader Lawrence Livermore National Laboratory.

• Karen Wahl, Ph.D., Chemist, Pacific Northwest National Laboratory.

The comments of most of these experts represented the views of the 

experts themselves and not the agency, university, or company with 

which they are affiliated.
Appendix IV: Comments from the Department 

of Homeland Security

Page 54 GAO-17-177 Bioforensics

Appendix IV: Comments from the 

Department of Homeland Security
Appendix IV: Comments from the Department 

of Homeland Security

Page 55 GAO-17-177 Bioforensics
Appendix IV: Comments from the Department 

of Homeland Security

Page 56 GAO-17-177 Bioforensics
Appendix IV: Comments from the Department 

of Homeland Security

Page 57 GAO-17-177 Bioforensics
Appendix V: GAO Contacts and Staff 

Acknowledgments

Page 58 GAO-17-177 Bioforensics

Timothy M. Persons, (202) 512-6412 or personst@gao.gov.

In addition to the individuals named above, Sushil Sharma (Assistant 

Director), Pille Anvelt, James Ashley, Hazel Bailey, Amy Bowser, Caitlin 

Dardenne, Jack Melling, Jeff Mohr, Penny Pickett, Amber Sinclair, Maria 

Stattel, Elaine Vaurio, and Elizabeth Wood made key contributions to this 

report.

Appendix V: GAO Contacts and Staff 

Acknowledgments

GAO Contacts

Staff 

Acknowledgments

(460639)
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to http://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.