Scientific & Technical Review

Panel Final Report for

2021-S-0003

Standards for Determining

Analytical and Stochastic

Thresholds for Application to

Forensic DNA Casework Using

Electrophoresis Platforms

Organization of Scientific Area Committees (OSAC) for Forensic Science
1

STRP Final Report

2021-S-0003

Standards for Determining Analytical

and Stochastic Thresholds for

Application to Forensic DNA Casework

Using Electrophoresis Platforms

Organization of Scientific Area Committees (OSAC) for Forensics Science

July 01, 2021

Disclaimer:

This report was produced by an independent Scientific and Technical Review Panel (STRP). The

views expressed in the report do not necessarily reflect the views or policies of the U.S.

Government. Visit the OSAC website for more information on OSAC’s STRP process.

Scientific & Technical Review Panel Members

• Susan Greenspoon, Virginia Department of Forensic Science
• Brian Hoey, Missouri State Highway Patrol

• Catherine Knutson, Minnesota Bureau of Criminal Apprehension

• Dan Krane, Wright State University

• Ron Reinstein, Arizona Supreme Court

• Bill Thompson, University of California, Irvine

• Kristen Thoms, Bode Technology Group

2

Report Summary:

The Scientific and Technical Review Panel (STRP) for “Standards for Determining Analytical

and Stochastic Thresholds for Application to Forensic DNA Casework Using Electrophoresis

Platforms” is an independent panel appointed by the National Institute of Standards and

Technology (NIST). A STRP is established with a range of experts to consider how well a

standard meets the needs of the forensic science, law enforcement, and legal communities, and to

recommend improvements to the standards under review. The STRP appreciates the efforts of

Christian Westring, Human Forensic Biology Subcommittee member, while serving as the

subcommittee liaison to this STRP during the review process.

The STRP began its review process with a kickoff meeting on December 15, 2020 and concluded

with this STRP final report. The panel reviewed the draft standard and prepared comments for

the OSAC Human Forensic Biology Subcommittee.

Report Components:

The STRP reviewed this draft standard against OSAC’s STRP Instructions for Review which

include the following content areas: scientific and technical merit, human factors, quality

assurance, scope and purpose, terminology, method description and reporting results. The details

below contain a brief description of each reviewed content area and the STRP’s assessment of

how that content was addressed in the draft OSAC Proposed Standard.

1. Scientific and Technical Merit: OSAC-approved standards must have strong scientific

foundations so that the methods practitioners employ are scientifically valid, and the

resulting claims are trustworthy. In addition, standards for methods or interpretation of
results must include the expression and communication of the uncertainties in measurements

or other results.

1.1. Consensus View – The STRP believes that this standard is sufficient, well

grounded in theory, complete, and capable of producing accurate, repeatable and

reproducible results.

It was the consensus view of the STRP that the Human Forensic Biology

Subcommittee sufficiently addressed their comments, so the standard is not

overly prescriptive or overly generalized.

With the addition of suggested statistical parameters such as probability,

standard deviation, and one-way analysis of variance (ANOVA) included in the

notes of 4.1.1 and 4.1.6, the standard is not only strengthened, but provides

more substantive guidance to the laboratory.

1.2. Minority View – None

3

2. Human Factors: All forensic science methods rely on human performance in acquiring,

examining, reporting, and testifying to the results. In the examination phase, some standards

rely heavily on human judgment, whereas others rely more on properly maintained and

calibrated instruments and statistical analysis of data.

2.1. Consensus View – The STRP believes this standard sufficiently articulates the

basis for determining analytical and stochastic thresholds. By standardizing the

nature and extent of the data that labs should collect to provide a foundation for

their thresholds, and the way in which labs should use the empirical data to

determine and the nature of the statistical analyses required to assess the

appropriateness of the lab's thresholds, the standard mitigates contextual bias

and improves human objectivity. Furthermore, by prescribing predetermined

specifications and quality attributes in a validation summary, a reviewer can

independently conclude whether the validation met its predetermined

specification.

2.2. Minority View – None

3. Quality Assurance: Quality assurance covers a broad range of topics. For example, a

method must include quality assurance procedures to ensure that sufficiently similar results

will be obtained when the methodology is properly followed by different users in different

facilities.

3.1. Consensus View – There was a split view that method modification should

require re-evaluation of both analytical and stochastic thresholds, and that the

parenthetical language from the respective sections (4.1.4-analytical, 4.2.4-

stochastic) may be combined for both thresholds. While the STRP appreciates
the distinction for assessment of expected baseline noise from an instrument

independently of the method applied to a sample prior to electrophoresis, the

panel also recognizes enhanced sensitivity methods may reduce the ability to

distinguish resulting allelic data from baseline noise, thus potentially impacting

the analytical threshold.

3.2. Minority View – The minority view of the STRP was that 4.1.4 states

specifically that the analytical threshold shall be reevaluated when modifications

to the methods are made and including the language from 4.2.4 would only

serve in making the standard more prescriptive.

A second suggestion was to change the word “when” to “whenever”.

4. Scope and Purpose: Standards should have a short statement of their scope and purpose.

They should list the topics that they address and the related topics that they do not address.

Requirements, recommendations, or statements of what is permitted or prohibited do not

belong in this section.

4

4.1. Consensus View – The STRP finds the standard contains an appropriate scope

and purpose. Additionally, the forward sufficiently introduces the reader to the

subject of the standard and the questions it is intended to address.

4.2. Minority View – None

5. Terminology: Standards should define terms that have specialized meanings. Only rarely

should they give a highly restricted or specialized meaning to a term in common use among

the general public.

5.1. Consensus View – The STRP believes most definitions and terms used

throughout this standard are well defined, and use common language and

ordinary meaning. Upon final review, a few exceptions were noted.

There is no formal definition for the use of the statistical term K or a contextual

reference.

Definition 3.8 Locus (loci) should state “Unique physical location(s)...” for

consistency.

The STRP recommends including a statement in the terminology report section

to detail that variations from the FBI Quality Assurance Standards (QAS)

definitions are to ensure consistency of the OSAC Lexicon across OSAC

disciplines, but do not contradict or otherwise negate the QAS definitions.

Furthermore, the STRP recommends including references to QAS and
SWGDAM to not only underscore the aforementioned point, but also emphasize

that labs are accountable for compliance of their validations to those standards.

5.2. Minority View – None

6. Method Description: There is no rule as to the necessary level of detail in the description of

the method. Some parts of the method may be performed in alternative ways without

affecting the quality and consistency of the results. Standards should focus on standardizing

steps that must be performed consistently across organizations to ensure equivalent results.

Alternatively, standards can define specific performance criteria that are required to be

demonstrated and met rather than specifying the exact way a task must be done. For example,

it may be enough to specify the lower limit for detecting a substance without specifying the

equipment or method for achieving this limit of detection.

6.1. Consensus View – The STRP believes this document standardizes a portion of a

method, but more specifically is demonstrative of how to more appropriately

conduct method validation. It sufficiently standardizes how validations must be

performed consistently across laboratories such that results are repeatable and

reproduceable.

6.2. Minority View – None

5

7. Reporting Results: Methods must not only be well described, scientifically sound, and

comprehensive but also lead to reported results that are within the scope of the standard,

appropriately caveated, and not overreaching.

7.1. Consensus View – The STRP believes this standard makes clear what should be

included in a validation summary report, that the results must be approved by

the appropriate official, and that the results shall be trained and available for

review.

7.2. Minority View – The minority view of the STRP was that validation reports and

data be made readily available to the public (e.g., accessible through a website

and not overtly subject to Sunshine or Freedom of Information Act requests).

STRP members from public laboratories understand and tacitly support the spirit

of this minority view; however, data practice decisions fall outside the scope of

this standard. Moreover, proprietary information and intellectual property may

be a barrier to commercial entities making documents public, and if legislated to

do so, such documents may require significant redaction.
6