Brooklyn Defender Services 177 Livingston Street 7th Floor T (718) 254-0700 www.bds.org 

Brooklyn New York 11201 F (718) 254-0897 

December 13, 2019 

Via OSAC Open Comment Portal 

Forensic Science Standards Board 

Organization of Scientific Area Committees 

 For Forensic Science 

National Institute of Standards and Technology 

U.S. Department of Commerce 

https://www.surveymonkey.com/r/B9K8QM6 

Re: Request for Comment on Standard 040—Standard for Forensic 

DNA Interpretation and Comparison Protocols. 

Dear Forensic Science Standards Board: 

Brooklyn Defender Services (“BDS”) submits these comments in opposition to 

placing the Organization of Scientific Area Committees for Forensic Science’s (OSAC) 

Biological Data Interpretation & Reporting Subcommittee’s Proposed Standard for 

Forensic DNA Interpretation and Comparison Protocols, ASB approved February 2019, 

ANSI approved September 2019 (hereinafter, “Standard 040”), ANSI/ASB Standard 040, 

1

st Edition 2019, on the OSAC Registry. 

 While BDS applauds the OSAC’s commitment to developing uniform standards 

across forensic science fields, as with companion Standard 020, the proposed Standard 

040 falls woefully short in several critical respects. Before this standard is included in the 

OSAC Registry, these shortcomings must be addressed.1

Defining validation. Standard 040 contains only a single reference to 

developmental validation, and that reference appears in passing in Annex B.2 Like 

Standard 020, Standard 040 never distinguishes the baseline requirement that methods 

be developmentally validated before being internally validated and used in the 

interpretation of DNA data. Similarly, Standard 040 includes no requirement that the 

 

1

 If, despite these serious shortcomings, Standard 040 is admitted to the Registry, these 

comments are offered for consideration in the drafting of future versions of this standard. 

2

 “First, DNA data interpretation and comparison protocols are derived from developmental and 

internal validation data (Section 4.1), after which the interpretation protocols will be assessed in 

accordance with the limitations defined in the protocol to determine whether the data (either in 

part or as a whole) are suitable or unsuitable for interpretation and comparison (Section 4.2).” 

Annex B at 5. 
Brooklyn Defender Services 177 Livingston Street 5th Floor T (718) 254-0700 www.bds.org 

Brooklyn New York 11201 F (718) 254-0897 

 

underlying scientific principles of a technique be peer-reviewed, developmentally 

validated, or scientifically sound.3

Standard 040 at 4.1 should read “The laboratory interpretation protocols and 

comparison protocols, including criteria for drawing conclusions from comparisons 

between evidentiary data and reference (or other evidentiary data), shall be based on, 

developed from, and supported by developmental validation studies and internal 

validation studies.” 

Defining qualifications. Standard 040 does not mention or address the 

qualifications needed for the personnel responsible for the interpretation and comparison 

protocols and does not refer to any other standard that might define those qualifications. 

Standard 040 should either define the appropriate qualifications for the involved 

personnel or specifically reference the standard that controls those qualifications. 

Defining the effective date. Standard 040 is not clearly retroactive and does 

not prescriptively define when new, updated, or expanded protocols are required. Annex 

B, which is only informative, begins: “It is the intent of this document that any DNA data: 

1) that fall outside the acceptable range of the interpretation and/or comparison method 

employed; 2) for which no suitable/appropriate documented protocol exists; or 3) for 

which no suitable internal validation studies exist to support the method, will not be 

interpreted or compared by the laboratory until the standards are sufficiently met and 

approved by the appropriate authority(ies) within the laboratory.” This compound 

sentence gestures at an effective date and triggering events for the creation of protocols. 

But the sentence is muddled and is buried in an informative annex. 

Standard 040 should be clear, specifically define a retroactive effective date, and 

include required triggering events for the creation, updating, or editing of protocols. 

Defining a scientifically appropriate scope. Standard 040’s Annex A 

Foundational Principles state: “This document applies to any type of DNA testing 

technology and methodology used, including . . . rapid protocols, etc., where mixtures of 

DNA may be encountered, analyzed, interpreted and compared.” Id. at 1.2. By including 

“rapid protocols,” Standard 040 clearly suggests that the OSAC is approving laboratory 

use of rapid systems on mixture analysis. This is contrary to the position of numerous 

oversight bodies, including SWGDAM, the FBI’s Quality Assurance Standards, and the 

Texas Forensic Science Commission.4 It is also scientifically unsupported. Scientific 

 

3

 In contrast, the Federal Bureau of Investigation’s Quality Assurance Standards for Forensic 

DNA Testing Laboratories requires “STANDARD 8.2 Developmental validation shall precede 

the use of a novel methodology for forensic DNA analysis.”; defines developmental validation 

under 8.2.1; and states “8.2.2 Peer-reviewed publication of the underlying scientific principle(s) 

of a technology shall be required.” 

4 See, e.g., Maura Dolan, ‘Rapid DNA’ promises breakthroughs in solving crimes. So why does it 

face a backlash?, Los Angeles Times (September 25, 2019) at 

https://www.latimes.com/california/story/2019-09-24/rapid-dna-forensics-crime-police; 

Rapid DNA, Federal Bureau of Investigation at 

https://www.fbi.gov/services/laboratory/biometric-analysis/codis/rapid-dna. 
Brooklyn Defender Services 177 Livingston Street 5th Floor T (718) 254-0700 www.bds.org 

Brooklyn New York 11201 F (718) 254-0897 

 

Working Group on DNA Analysis Methods, Position Statement on Rapid DNA Analysis

at 1 (“Rapid DNA technology is not currently suitable for crime scene samples . . . .”). 

Standard 040’s Annex A should not include “rapid protocols.” 

Addressing contamination. Standard 040 does not contain a single 

requirement that interpretation and comparison protocols include a review of 

appropriate data from either standards or controls. Tellingly, the only discussion of 

contamination comes in Requirement 4.2.4, where Standard 040 opaquely states that 

interpretation protocols need to address the “limitations of the interpretation methods 

such as characterizing and defining . . . issues associated with . . . potential contamination 

events.” Given that the threat of contamination is an ever-present specter, particularly as 

techniques continue to become ever more sensitive, this omission is both startling and 

inexcusable. 

 Standard 040 should directly define contamination and include clear 

requirements for documented protocols governing both the use of standards and controls 

and the detection of contamination. 

Addressing non-conformity events. Standard 040 does not address the need 

for a protocol covering instances of non-conformity uncovered by the technical or 

administrative review. Additionally, the Standard does not address the need for a protocol 

regulating analyst requests to deviate from the established protocols. 

 Standard 040 should directly address the need for documented protocols 

regulating instances of non-conformity with the interpretation and comparison protocols 

uncovered during the review process and/or instances of analyst requests to deviate from 

established interpretation and comparison protocols. Standard 040 should, at a 

minimum, include a requirement that non-conformity events be documented and that 

any ensuing corrective action, or similar review, be documented and included in the case 

file. 

Defining “data that cannot be interpreted” and “data that are 

unsuitable for comparison”. Standard 040 requires that interpretation protocols 

address both “criteria for defining what are interpretable data versus data that cannot be 

interpreted” and “suitable for comparison versus data that are unsuitable for 

comparison.” Standard 040.4.2.5 and 4.2.6. Furthermore, Standard 040 defines 

“unsuitable for comparison” as “data that cannot be used for comparisons for reasons 

including, but not limited to, poor or limited data quality, mixture complexity, or a failure 

to meet quality assurance requirements.” Id. at 3.7. However, Standard 040 never defines 

“cannot be interpreted.” The Standard fails to connect either data that cannot be 

interpreted and/or data that are unsuitable for comparison to “the limitations of the 

interpretation methods used,” and instead separates the limitations discussed in 4.2.4 

from the criteria required in 4.2.5 and 4.2.6. 

 Standard 040 should include a definition of both “cannot be interpreted” and 

“unsuitable for comparison.” Standard 040 should also connect the limitations identified 
Brooklyn Defender Services 177 Livingston Street 5th Floor T (718) 254-0700 www.bds.org 

Brooklyn New York 11201 F (718) 254-0897 

 

by internal validation to the establishment and definition of both data that cannot be 

interpreted and data that is unsuitable for comparison. 

Addressing statistical calculations. Standard 040 does not include any 

requirement for comparison protocols to address the appropriate calculation of statistical 

significance for any inclusionary result. This is such a fundamental step in the comparison 

process that the Standard’s failure to address it is glaring. 

 Standard 040 should include a requirement that comparison protocols address 

statistical calculations. 

Addressing the availability of interpretation and comparison protocols 

for review. Both Standard 020 gestures to “documented conformance” being made 

“readily available for review” by “stakeholders who use reports generated by the DNA 

mixture test protocols and procedures” and the updated QAS requires “all validation 

documentation be retained and available for review.” See the Federal Bureau of 

Investigation’s Quality Assurance Standards for Forensic DNA Testing Laboratories

(effective July 1, 2020) at 8.9. However, Standard 040 does not include any requirement 

that the laboratory’s documented interpretation and comparison protocols be made 

readily available for review. 

 Standard 040 should explicitly require that the laboratory’s documented 

interpretation and comparison protocols be retained and electronically available for 

review by stakeholders (including criminal defense attorneys) who use reports generated 

by the DNA mixture test protocols and procedures. See National Commission on Forensic 

Science, Recommendation to the Attorney General Transparency of Quality 

Management System Documents (Recommending that all quality management system 

documents be immediately made accessible to the public in an electronic format upon 

request and posted on the laboratory’s website within one year of the recommendations 

adoption), https://www.justice.gov/archives/ncfs/page/file/839706/download. 

 Because Standard 040 fails to adequately define validation, required 

qualifications, its own effective date, a scientifically appropriate scope, contamination, 

complexity thresholds, statistical calculations, and documentation requirements, this 

standard should not be included in the OSAC Registry. Instead, these critical 

shortcomings should be addressed, and the standard should be improved prior to 

inclusion. 

Sincerely, 

/s/ Elizabeth Daniel Vasquez

Elizabeth Daniel Vasquez 

Special Forensic Science Counsel 

Brooklyn Defender Services 

177 Livingston Street, 7th Floor 

Brooklyn, New York 11201