FORENSIC EVIDENCE: 

THOUGHTS OF AN 

ACCIDENTAL TOURIST 

PROFESSOR JULES EPSTEIN 

NIST, NOVEMBER 2016 
DISCLAIMER
“20 YEARS AGO TODAY...” 

We hope that these 

commentaries and the NIJ 

report spur a broader 

debate about the value of 

DNA technology and the 

role of science in the 

criminal justice system’s 

search for truth.

Janet Reno
What Am I Doing Here?
A Personal History in Forensics
WHAT CAN I OFFER TO SCIENTISTS?

IF I AM CORRECT IF I AM MISTAKEN
First, Accolades
Forensics – Some Data:

Cleveland Homicides 2008-2011

214 CASES WITH PROBATIVE 

FORENSICS

Disposition:

◦214 cases

◦199 arrests

◦187 convictions/pleas

Sentence Length:

◦13.6 years average

71 CASES WITH NO PROBATIVE 

FORENSICS

Disposition:

◦71 cases

◦49 arrests

◦39 convictions/pleas

Sentence Length:

◦9.4 years average
Forensics and ‘Ordinary’ Crime

when DNA evidence was collected at 

property crime scenes: 

◦ Suspect identifications and arrests 

doubled.

◦ Cases accepted for prosecutions 

doubled.

◦ DNA was twice as effective in 

identifying suspects as fingerprints.
Forensics and “Error Correction”
A “NEEDS ASSESSMENT”

POLICING THE LAWYERS POLICING THE DISCIPLINES
Let’s Start With Lawyers
The Dimension of the 

Problem
Forensics and the Courts
JUDGES ARE NOT SCIENTISTS (2001)

Judges had the most difficulty 

operationalizing falsifiability and error 

rate, 

with only 5% of the respondents 

demonstrating a clear understanding 

of falsifiability and 

only 4% demonstrating a clear 

understanding of error rate. 
Courts continue to rely on forensic 

evidence without fully understanding 

and addressing the limitations

Forensics and The Courts (2009)
In a number of...disciplines, forensic 

science professionals have yet to 

establish either the validity of their 

approach or the accuracy of their 

conclusions, and the courts have 

been utterly ineffective in addressing 

this problem.

Forensics and the Courts (2009)
Since 2009, 

fewer than 140 

cases even 

mention the NAS 

Report

It’s Not Unusual
Attorneys and Forensics

Misreading lab reports

The trial closing:

“Since his DNA was the 

major contributor, he 

wore it more recently.” 

WRONG
Attorneys and Forensics 2 –

An Exemplar

In his opinion, the marks at the wig shop 

were made by the pry bar in question. 

Bergeron did not use phrasing 

overemphasizing his certainty.
ATTORNEYS and WAIVER

I believe there are serious questions about whether 

[TOOLMARK MATCHING] evidence has an adequate 

scientific foundation to allow its admission under 

MRE 702. 
ATTORNEYS and WAIVER - AWOL

I concur in this Court's order denying leave to 

appeal, however, because this issue is 

unpreserved 

People v. McAdoo, (Mich. Mar. 6, 2015)
Science and Law 

in the Rear View Mirror

What happens when 

science admits its 

limits years after a 

conviction?
Trial in 1995:

Peele testified that the 

composition of elements in the 

molten lead of a bullet fragment 

found in Kulbicki’s truck 

matched the composition of 

lead in a bullet fragment 

removed from the victim’s 

brain;
The Conclusions

a similarity of the sort one would “‘expect’” if “‘examining 

two pieces of the same bullet.’

a bullet taken from Kulbicki’s gun was not an “exac[t]” 

match to the bullet fragments, but 

was similar enough that the two bullets likely 

came from the same package.
The Legal Problem 

11 years after his conviction, CBLA had fallen out of 

favor.

Translated: The NRC expressed concerns, however, 

relating to the interpretation of the results of bullet 

lead examinations.
Back to 1995 – The Clue

a report coauthored by [this expert] in 

1991 

◦the composition of lead in some bullets was 

◦the same as 

◦lead in other bullets packaged many months 

later in a separate box
The Lawyer’s Duty: 2 Views

the failure to properly investigate 

forensic methodology and 

adequately cross-examine a 

forensic expert may be a basis for 

deficient performance. 

Kulbicki v. State, 440 Md. 33, 47, 

99 A.3d 730, 738 (2014) 

Counsel did not perform 

deficiently by dedicating their 

time and focus to elements of 

the defense that did not involve 

poking methodological holes in a 

then-uncontroversial mode of 

ballistics analysis.

Maryland v. Kulbicki, 2015 U.S. 

LEXIS 4693, *5 (U.S. Oct. 5, 2015)
The Unmentioned Issues

Was there a curriculum vitae?

No mention of a duty of a lawyer to read up on the 

discipline (at least as long as it’s currently accepted)

Here’s what Maryland wrote:

"[f]ailure to investigate the forensic evidence is not what a 

competent lawyer would do.”
ARE FRYE AND DAUBERT FIXES?
Do Frye and Daubert Hearings 

Make for More Accurate Science?

United States v. Llera-Plaza I: expert witnesses 

will not be permitted to 

present...testimony...that a particular latent 

print is in fact the print of a particular person.

United States v. Llera-Plaza II: Let it in. 

I am now persuaded that the standards which 

control the opining of a competent fingerprint 

examiner satisfy Daubert. 
Why Are Daubert and Frye Inadequate?

Daubert’s “evidentiary” reliability test does 

not mean necessarily mean “accurate”

Frye’s “general acceptance” test varies with 

the community
Why Are Daubert and Frye Inadequate?

Daubert is applied far more stringently in civil cases than in criminal 

ones, and

even in criminal ones, Daubert serves a more effective screening role 

for filtering out flawed defense expert testimony than weak 

prosecution and police-generated forensic science. 
Why Are Daubert and Frye Inadequate?

The cost of litigation

The overworked indigent defense counsel

No resources/modeling

AND

Perhaps adversary hearings are not the best model
WHAT ABOUT FORENSIC DISICPLINES?
PROBLEM 1 – WE STILL PERMIT

RISKY EVIDENCE

Bite Mark Matching

“Perfect match...”

“match up...”

“matching...”

“the biter...”

“reasonable dental certainty...”

“one in a million...”
FORENSIC ODONTOLOGISTS’ 2014 

RESEARCH - OOPS

100 cases, each with 3 questions: 

• Was there sufficient evidence to render an opinion on 

whether it was a human bitemark? 

• Using newly proposed (draft) ABFO decision tree, was the 

injury a human bitemark? 

• If a human bitemark, did it have distinct, identifiable 

arches and individual tooth marks? 
Results – decision spread!

Look at the spread of decisions for individual cases. 

"2!

3!

8!

13!

18!

23!

28!

78!68!81!

86!

65!

70!52!

85!99!15!

100!

69!18!35!

4!

43!

98!

60!94!56!73!44!66!36!83!22!

55!

62!

89!

3!

41!77!

30!

51!

76!10!

11!21!27!47!48!

92!96!72!

8!

74!

1!

50!

19!25!

This!is!a!human!bite!mark! This!is!sugges; ve!of!a!bite!mark! This!is!not!a!bite!mark!
Results – decision spread!

Look at the spread of decisions for individual cases. 

0!

5!

10!

15!

20!

25!

30!

35!

40!

34!49!71!90!95!29!97! 2! 40! 5! 75!24!39!53!84!91!23!31!37!87!12!17!32!67!93!13!38!46! 6! 79!82!28!58!59!64! 9! 14!20!42!45!57!88!16!54!61!80!26!33!63! 7!

This!is!a!human!bite!mark! This!is!sugges; ve!of!a!bite!mark! This!is!not!a!bite!mark!
PROBLEM 2 - ETHICS
ETHICS – A MIXED MESSAGE?

FORENSIC SCIENCE COMMISSION

Once a report is issued and the 

adjudicative process has 

commenced,

Communicate fully with the 

parties

Unless a privilege, protective 

order or law prevents disclosure

ATTORNEY GENERAL RESPONSE

◦ Honestly communicate with all 

parties

◦ When communications are 

permitted by law or agency 

practice
PROBLEM 3 –

COMMUNICATING RESULTS
Q: Do you have an opinion as to the 

probability that the defendant's left 

ear is the source of the latent 

impression which was left at the 

scene of the crime in this case?

A: I do have an opinion, yes.

Earprint Testimony
A: I think it's probable that it's the 

defendant's ear is the one that was 

found on the scene...I'm 100 percent 

confident with that opinion. 

Earprint Testimony
What does that 

mean?

Is that science?

Certain that it is Probable
HAVE WE LEARNED? 

SHOE/TIRE IS THE SOURCE OF THE 

IMPRESSION

IF EVIDENCE AND SAMPLE SHARE 

CLASS CHARACTERISTICS 

AND ONE OR MORE RANDOMLY 

ACQUIRED CHARACTERISTICS
THE NON-SCIENTIST’S QUESTIONS
SHOE/TIRE IS THE SOURCE OF 

THE IMPRESSION

IF EVIDENCE AND SAMPLE 

SHARE CLASS CHARACTERISTICS 

AND ONE OR MORE RANDOMLY 

ACQUIRED CHARACTERISTICS

SHOE/TIRE PROBABLY MADE 

THE IMPRESSION

BASED ON THE CORRESPONDENCE 

OF CLASS CHARACTERISTICS

IN COMBINATION WITH SPECIFIC 

WEAR AND/OR RANDOMLY 

ACQUIRED CHARACTERISTICS
PROBLEM 4 - RELIABILITY
DO WE YET KNOW HOW RELIABLE 

EXPERTS ARE?
PROBLEM 5

When is Enough Enough?
PROBLEM 5

When is Enough Enough?

an examiner may conclude that two items 

have a common origin 

if their marks are in “sufficient 

agreement,” 
WHAT’S THE TAKE-AWAY?
IF I AM MIS-INFORMED?
AND IF I AM CORRECT?

MEASURE, MEASURE, MEASURE