Brooklyn Defender Services 177 Livingston Street 7th Floor T (718) 254-0700 www.bds.org 

Brooklyn New York 11201 F (718) 254-0897 

December 13, 2019 

Via OSAC Open Comment Portal 

Forensic Science Standards Board 

Organization of Scientific Area Committees 

 For Forensic Science 

National Institute of Standards and Technology 

U.S. Department of Commerce 

https://www.surveymonkey.com/r/2XKW9WR 

Re: Request for Comment on Standard 020—Standard for Validation Studies of DNA 

Mixtures, and Development and Verification of a Laboratory’s Mixture 

Interpretation Protocol. 

Dear Forensic Science Standards Board: 

Brooklyn Defender Services (“BDS”) submits these comments in opposition to 

placing the Organization of Scientific Area Committees for Forensic Science’s (OSAC) 

Biological Data Interpretation & Reporting Subcommittee’s Proposed Standard for 

Validation Studies of DNA Mixtures, and Development and Verification of a Laboratory’s 

Mixture Interpretation Protocol, ASB approved February 2018, ANSI approved 

September 2018 (hereinafter, “Standard 020”), ANSI/ASB Standard 020, 1st Edition 

2018, on the OSAC Registry. 

 While BDS applauds the OSAC’s commitment to developing uniform standards 

across forensic science fields, the proposed standard falls woefully short in a number of 

critical respects. Before this standard is included in the OSAC Registry, these 

shortcomings must be addressed.1

Defining validation. While the title of Standard 020 and the “terms and 

definitions section” both refer to “validation” generally, Standard 020 only substantively 

addresses internal validation. Standard 020 never references developmental validation, 

and never distinguishes the baseline requirement that methods be developmentally

validated before being internally validated and used in the interpretation of DNA data. 

Similarly, Standard 020 includes no requirement that the underlying scientific principles 

of a technique be peer-reviewed, developmentally validated, or scientifically sound.2

 

1

 If, despite these serious shortcomings, Standard 020 is admitted to the Registry, these 

comments are offered for consideration in the drafting of future versions of this standard. 

2

 In contrast, multiple standards currently passing through the standards development process 

appropriately define validation, and discuss the foundational importance of developmental 
Brooklyn Defender Services 177 Livingston Street 5th Floor T (718) 254-0700 www.bds.org 

Brooklyn New York 11201 F (718) 254-0897 

 

Either (A) the standard should be re-titled to include the restrictive adjective 

“Internal” and the definition of “validation” in the “terms and definitions section” should 

be removed or (B) the standard should specifically define “validation” as a process 

inclusive of both developmental and internal validation, state that this standard only

addresses internal validation, and refer the reader to the additional standard(s) that cover 

developmental validation.3

Defining qualifications. Standard 020 does not mention or address the 

qualifications needed for the personnel conducting the validation and does not refer to 

any other standard that might define those qualifications. The only reference to the 

specific laboratory personnel who should be involved in the internal validation process 

comes in the Conformance section. There, Standard 020 states “Documented 

conformance to these requirements need to be: (1) approved by the laboratory’s DNA 

Technical Leader or other appropriate personnel . . . .” In addition to being a poorly 

constructed sentence, this requirement alone cannot ensure that internal validation—the 

most critical phase of laboratory technique adoption—is conducted by qualified personnel 

or that its appropriate completion is actually approved by qualified personnel. 

Standard 020 should either define the appropriate qualifications for the involved 

personnel or specifically reference the standard that controls those qualifications. 

Defining the effective date. Standard 020 is not clearly retroactive and does 

not prescriptively define when internal validation is required. The standard only 

specifically requires the verification of existing protocols, see Requirements 4.4.3 

(“Verification shall be performed on new, existing, and modified mixture interpretation 

protocols.”), but merely advises that previous validation be reviewed without requiring 

retroactive review. See Scope 1.2 (“Laboratories are advised to review their previous 

 

validation. See, e.g., ASB Standard 038, “Standard for Internal Validation of Forensic DNA 

Analysis Methods,” (First Edition, 2019); ASB Standard 077, “Standard for Developmental and 

Internal Validation of Forensic Serological Methods,” (First Edition, 2019); and ASB Standard 

018, “Standard for Validation of Probabilistic Genotyping Systems,” (First Edition, 2019). 

Additionally, the Federal Bureau of Investigation’s Quality Assurance Standards for Forensic 

DNA Testing Laboratories requires “STANDARD 8.2 Developmental validation shall precede 

the use of a novel methodology for forensic DNA analysis.”; defines developmental validation 

under 8.2.1; and states “8.2.2 Peer-reviewed publication of the underlying scientific principle(s) 

of a technology shall be required.” 

3 Compare, e.g., Standard 020 at 3.5 (“Validation” is defined as “The process of performing a set 

of experiments that establish the efficacy, reliability, and limitations of a method, procedure or 

modification thereof; establishing recorded documentation that provides a high degree of 

assurance that a specific process will consistently produce an outcome meeting its 

predetermined specifications and quality attributes.”) with the Federal Bureau of Investigation’s 

Quality Assurance Standards for Forensic DNA Testing Laboratories at 2 (“Validation” is 

defined as “[A] process by which a procedure is evaluated to determine its efficacy and reliability 

for forensic casework analysis and includes the following: (1) Developmental validation is the 

acquisition of test data and determination of conditions and limitations of a new or novel DNA 

methodology for use on forensic samples. (2) Internal validation is an accumulation of test data 

within the laboratory to demonstrate that established methods and procedures perform as 

expected in the laboratory.”). 
Brooklyn Defender Services 177 Livingston Street 5th Floor T (718) 254-0700 www.bds.org 

Brooklyn New York 11201 F (718) 254-0897 

 

validation for compliance with this standard, supplement validation where necessary, and 

modify existing protocols accordingly.”). 

Standard 020 should be clear, and specifically prescribe retroactive review for 

conformity with its requirements.4

Defining a scientifically appropriate scope. Standard 020’s Scope states: 

“This standard applies to any type of DNA testing technology and methodology used, 

including . . . rapid protocols.” Id. at 1.2. By including “rapid protocols,” Standard 020 

clearly suggests that the OSAC is approving laboratory use of rapid systems on mixture 

analysis. This is contrary to the position of numerous oversight bodies, including 

SWGDAM, the FBI’s Quality Assurance Standards, and the Texas Forensic Science 

Commission.5 It is also scientifically unsupported. Scientific Working Group on DNA 

Analysis Methods, Position Statement on Rapid DNA Analysis at 1 (“Rapid DNA 

technology is not currently suitable for crime scene samples . . . .”). 

Standard 020’s Scope should not include “rapid protocols.” 

Defining “unsuitable for comparison” and a complexity threshold.

While Standard 020 states that “the data from the validation studies . . . shall provide 

guidance for the types of mixed DNA profiles that will be interpreted by the laboratory” 

and requires the studies to “aid in assessing and defining the [methodologies’] 

limitations,” Standard 020 does not address the role of the internal validation in 

developing, as required by Standard 040, “criteria for defining what are interpretable data 

versus data that cannot be interpreted” and “suitable for comparison versus data that are 

unsuitable for comparison.” Standard 040.4.2.5 and 4.2.6. Similarly, Standard 020 does 

not specifically address mixture complexity at all. Instead, Standard 020 gestures toward 

“defining the limitations,” but includes no substantive discussion of methodological 

limitations or any requirement that validation actually incorporate a sufficient quantity 

of data to identify and define those methodological limitations. 

 Similarly, by limiting the mixture study requirements in Requirement 4.2, 

Standard 020 actively avoids an internal validation protocol that would “defin[e] the 

limitations” of the methodology. Standard 020 reads: “The mixture studies shall include, 

at a minimum, mixed DNA samples that: . . . Are representative of those typically

encountered and interpreted by the testing laboratory.” Id. at 4.2 and 4.2.1 (emphasis 

added). The limiting adjective “typically” should be removed, as the standard should 

require that mixture studies include all types of samples encountered by the laboratory 

which the laboratory intends to interpret and compare. 

 

4

 Annex A “Foundational Principles” states: “It is the intent that this standard be applied to any 

existing interpretation and comparison protocols and that the protocol be revised as needed.” 

But Annex A is marked as “informative,” not “normative.” 

5 See, e.g., Maura Dolan, ‘Rapid DNA’ promises breakthroughs in solving crimes. So why does it 

face a backlash?, Los Angeles Times (September 25, 2019) at 

https://www.latimes.com/california/story/2019-09-24/rapid-dna-forensics-crime-police; 

Rapid DNA, Federal Bureau of Investigation at 

https://www.fbi.gov/services/laboratory/biometric-analysis/codis/rapid-dna. 
Brooklyn Defender Services 177 Livingston Street 5th Floor T (718) 254-0700 www.bds.org 

Brooklyn New York 11201 F (718) 254-0897 

 

 Standard 020 should include a definition of “cannot be interpreted” and 

“unsuitable for comparison” that is consistent with Standard 040. Standard 020 should 

also address the role of internal validation in establishing and defining a complexity 

threshold for interpretation. Standard 020 should not limit internal validation to “typical” 

samples, but instead should require all sample types that will be tested, interpreted, and 

compared by the laboratory. 

Defining “documented conformance” and “be[ing] made readily 

available for review.” Standard 020’s commitment that “documented conformance” 

be made “readily available for review” by “stakeholders who use reports generated by the 

DNA mixture test protocols and procedures” gestures toward an essential requirement 

for validation testing more broadly. Specifically, it signals the underlying necessity that 

validation testing be comprehensively documented, and that “all validation 

documentation be retained and available for review.” See the Federal Bureau of 

Investigation’s Quality Assurance Standards for Forensic DNA Testing Laboratories

(effective July 1, 2020) at 8.9. However, Standard 020 is not explicit in the requirement 

that all portions of internal validation testing be comprehensively documented and does 

not specifically require that all validation documentation be retained and available for 

review. 

 Standard 020 should explicitly require that all validation data be documented, and 

that all validation documentation be retained and electronically available for review by 

stakeholders (including criminal defense attorneys) who use reports generated by the 

DNA mixture test protocols and procedures. See National Commission on Forensic 

Science, Recommendation to the Attorney General Transparency of Quality 

Management System Documents (Recommending that all quality management system 

documents be immediately made accessible to the public in an electronic format upon 

request and posted on the laboratory’s website within one year of the recommendations 

adoption), https://www.justice.gov/archives/ncfs/page/file/839706/download. 

 Because Standard 020 fails to adequately define validation, required 

qualifications, its own effective date, a scientifically appropriate scope, a complexity 

threshold and documentation requirements, this standard should not be included in the 

OSAC Registry. Instead, these critical shortcomings should be addressed, and the 

standard should be improved prior to inclusion. 

Sincerely, 

/s/ Elizabeth Daniel Vasquez

Elizabeth Daniel Vasquez 

Special Forensic Science Counsel 

Brooklyn Defender Services 

177 Livingston Street, 7th Floor 

Brooklyn, New York 11201