Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

1

NIST Inter-Laboratory Studies 

for DNA Mixture Interpretation

Michael D. Coble, PhD

Carolyn R. Hill, MS; David L. Duewer, PhD; and 

Margaret C. Kline, MS

National Institute of Standards and Technology

66th Annual AAFS Meeting

February 20, 2014

Seattle, WA

NIST and NIJ Disclaimer

Past and Present Funding: Interagency Agreement 

between the National Institute of Justice and NIST 

Office of Law Enforcement Standards

Points of view are mine and do not necessarily represent 

the official position or policies of the US Department of Justice or the 

National Institute of Standards and Technology. 

Certain commercial equipment, instruments, software and materials are 

identified in order to specify experimental procedures as completely 

as possible. In no case does such identification imply a 

recommendation or endorsement by the National Institute of 

Standards and Technology nor does it imply that any of the 

materials, instruments or equipment identified are necessarily the 

best available for the purpose.

Interlaboratory Studies

• The method by which multiple laboratories 

compare results and demonstrate that the 

methods used in one’s own laboratory are 

reproducible in another laboratory.

• These tests are essential to demonstrate 

consistency in results from multiple laboratories.

(J.M. Butler, Forensic DNA Typing, p. 216)
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

2

NIST DNA Mixture Interlab Studies

• Mixed Stain Study 1 (MSS1)

• April 1997 – November 1997

• 22/28 labs participated

• 6 reference samples, 4 two-person mixtures, 

1 three-person mixture (stains on paper)

• Focus = donor types given a complete set of 

reference sources.

Margaret Kline Dave Duewer

NIST DNA Mixture Interlab Studies

• Mixed Stain Study 2 (MSS2)

• January 1999 – May 1999

• 45/52 labs participated

• Part A = 4 reference samples, 1 two-person 

mixture, 1 three-person mixture (stains on 

paper)

• Part B = 1 mixture at 5 different concentrations

• Focus = donor types given an incomplete set of 

references; DNA Quantification Study.

MSS1 and MSS2

J Forensic Sci 2001;46(5):1199–1210.
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

3

Technology in motion

Technology in motion

NIST DNA Mixture Interlab Studies

• Mixed Stain Study 3 (MSS3)

• December 2000 – October 2001

• 74/83 labs participated

• 1 single-source, 5 two-person mixtures, 1 threeperson mixture (all extracts)

• Focus = effect of DNA quantitation on STR 

typing performance.
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

4

NIST DNA Mixture Interlab Studies

• Mixture Study 2005 (MIX05)

• January 2005 – August 2005

• 69/94 labs participated

• 4 two-person mixtures (only electronic data)

• Focus = evaluate the “lay of the land” and 

determine future needs for training and tools for 

interpretation. 

MIX05 Poster Presentation at ISHI

http://www.cstl.nist.gov/biotech/strbase/interlab/MIX05.htm

Conclusions: Wide range of variation 

within and between laboratories
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

5

How MIX13 differs from MIX05 study

MIX13 (2013) MIX05 (2005)

Response 106 labs 69 labs

Number of 

cases provided

5 cases 4 cases

Case types 

being mimicked

Sexual assault & 

touch evidence

Sexual assault 

evidence

Mixture 

complexity

2, 3, >3-person

(potentially related, 

low-template, 

inclusion/exclusion)

all 2-person

(all unrelated, 

male/female; various 

major/minor ratios)

Scenarios 

provided

Yes No

MIX 13 – NIST Interlaboratory Study on 

Mixture Interpretation - Purpose

• MIX05 – conducted in 2005. Since then a great 

deal of effort has been focused on 

improvements in DNA mixture interpretation.

• 2010 SWGDAM Guidelines approved in January 

2010 – many labs have changed their protocols 

recently. 

• MIX13 – Interpretation challenge – no samples 

to run. 

MIX 13 – NIST Interlaboratory Study on 

Mixture Interpretation - Goals

• (1) To evaluate the current “lay of the land” 

regarding STR mixture interpretation across the 

community. 

• (2) To measure consistency in mixture interpretation 

across the U.S. after the publication of the 2010 

SWGDAM guidelines.

• (3) To learn where future training and research could 

help improve mixture interpretation and reporting.
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

6

Alaska

Hawaii

MIX13 Participants from 106 Laboratories

45 states had at least one lab participate

Green = participants

Gray = no data returned

Federal Labs

FBI (DOJ)

ATF (DOJ)

USACIL (DOD)

Canada

RCMP

CFS

Montréal

52 state labs 

(40 states)

48 local labs

3 federal

3 non-U.S.

Due to the number of laboratories 

responding and the federal, state, 

and local coverage obtained, this 

MIX13 interlaboratory study can be 

assumed to provide a reasonable 

representation of current U.S. 

forensic DNA lab procedures 

across the community

MIX13 was also used an intra-lab study

Comments from TL of a MIX13 Lab

• Thank you for the opportunity to participate in this 

exercise! Some of these were very challenging and 

provoked a lot of conversation.

• I had a majority of the analysts in our Forensic 

Biology Unit interpret these profiles independently in 

an effort to identify inconsistencies and areas where 

we need to improve. It was very interesting how 

much the results varied! I’ve included two 

spreadsheets that demonstrate this – “MIX13 

summary of allele calls” and “MIX13 summary of 

stats and conclusions.” 

16 different analysts examined the data in this particular lab
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

7

Purpose of MIX13 Cases

According to German Stain Commission (2009) mixture types: 1 = A, 2 = C, 3 = ?, 4 = B, 5 = ?

Challenge provided to study responses

Case 1 ~1:1 mixture (2-person)

Case 2 Low template profile with potential 

dropout (2-person)

Case 3 Potential relative involved (3-person)

Case 4 Minor component (2-person)

Case 5 Complex mixture (>3-person) with # of 

contributors; inclusion/exclusion 

issues

Case 04 – Bite Mark Evidence

Minor component (2-person)

“German Type B”

Scenario

• Evidence: saliva swab from a bite mark on the victim.

• A female waiting at a bus stop in the late evening is 

attacked from behind and pushed to the ground. A 

motorist driving by witnesses the attack, pulls his car 

over, and runs to her aid. As the Good Samaritan comes 

upon the scene, the perpetrator bites the victim on the 

back of her neck before running away.

• The motorist is able to give a good description of the 

perpetrator and a few days later, the police arrest a 

suspect. He is positively identified in a police lineup by 

the witness. 
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

8

Case 04 – PP16HS

Case 04 – IDPlus

Case 04 – One Suspect

Individual Inclusion? Ratio

Victim Included ~3.5

Suspect 4A Included 1
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

9

Primary Goals

• Primary purpose – will labs choose to 

deconvolve this mixture since the mixture ratio is 

close to the limit of deconvolution for many labs?

• All labs have included the suspect in the mixture.

At least 2

32%

more than 1

2%

assume 2

40%

2 or more

4%

cons w/ 2

16%

cons w/ 2 or 

more

2%

cons w/ at least 2

2%

mixture of DNA

2%

# of Contributors

CPI

30%

LR

14%

mRMP

56%

Statistical Interpretation
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

10

CPI Analysis

• Stats ranged from 1 in 3,070 to 1 in 862,000 with 

a median of 1 in 14,380.

Alleles below ST (ID+)

7 Loci

D8

D21

D7

CSF

D2

TPOX

FGA

1 of 12 ID+ labs 

included all but D7

Alleles below ST (PP16HS)

7 Loci

D21

Penta E

D13

vWA

D8

TPOX

FGA

All PP16 labs 

excluded these

markers
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

11

One PP16HS Lab (CPI)

Excluded Penta D from the 

CPI stats because the 11 

allele may fall below ST due 

to stutter from the 12 allele

150 RFU

(ST)

A similar situation with ID+

If 10% stutter from the 12 

allele (163 RFU) is part of the 

11 allele, then the remaining 

peak (70 RFU) is below the ST 

No CPI labs excluded D16 

from the stat

mRMP/LR Analysis

• Stats ranged from 1 in 358,000 to 1 in 412 

Quintillion with a median of 1 in 2.58 Quadrillion
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

12

Uncertainty with D16

12

1633

11

233

70 RFU

(POI)

163 RFU

(Stutter)

Uncertainty with D16 

Observed Approaches

Drop the locus

(Below ST)

12

1633

11

233

70 RFU

(POI)

163 RFU

(Stutter)

Uncertainty with D16 

Observed Approaches

Use 2p

(Below ST)

12

1633

11

233

70 RFU

(POI)

163 RFU

(Stutter)
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

13

Uncertainty with D16 

Observed Approaches

Use 2pq or p2

(to infer genotypes)

11,12 or 11,11

12

1633

11

233

150 RFU

(ST)

Uncertainty with D16 

Observed Approaches

Use 2pq

(to infer genotype)

11,12

(matches POI)

“Certainty”

12

1633

11

233

150 RFU

(ST)

dropped

19%

2p

38%

2pq

35%

p2 or 

2pq

8%

Statistics with D16 (mRMP/LR)

Probabilistic approaches to interpretation 

can be useful to reduce uncertainty and subjectivity

4.2. For calculating the CPE or 

RMP, any DNA typing results 

used for statistical analysis must

be derived from evidentiary 

items and not known samples.
Coble – NIST Mixture Interlaboratory

Studies

02/20/2014

14

Summary

• Most labs have validated and implemented AT and STs 

since MIX05. However, there is still a great deal of 

variation in interpretation across the US.

• An Idea – if everyone uses the same AT/ST, then one 

would expect to see similar results.

• Reality – the results were all over the place, Some of this 

was to be expected since each lab’s protocol is different 

(e.g. dropping a locus vs, 2p). 

• Probabilistic approaches will also deconvolve the mixture 

(without dropping loci), and can do so without bias.

NIST/NRC Postdoc Program

Working in the Applied Genetics Group at NIST

• Current stipend (2014) is $66,256 per year

– Currently a limit of 120 slots per year

– Congressionally-mandated program for NIST

– Maximum 2-year appointments

• Awardees must be U.S. citizens

• Awardees are chosen through a national competition 

administered by the National Research Council of the National 

Academy of Sciences.

• Two competitions per year 

– deadlines of February 1 and August 1

• Contact either Dr. Peter Vallone (peter.vallone@nist.gov) or 

or Dr. Michael Coble (michael.coble@nist.gov)

http://www.nist.gov/iaao/postdoc.cfm

http://nrc58.nas.edu/RAPLab10/Opportunity/Program.aspx?LabCode=50

Selected Topics

Rapid DNA Typing

DNA Mixture Analysis

Forensic Applications of Next-Gen 

Sequencing

DNA Extraction efficiency

Forensic SNPs

Y-STRs 

Open to suggested topics/projects

Thank You!

John Butler Margaret Kline

Robin Cotton Becky Hill

Charlotte Word Dave Duewer

John Paul Jones

mcoble@nist.gov

301-975-4330

Outside funding agencies:

NIJ – Interagency Agreement with the Office of Law Enforcement Standards