Strategic	Research	Direc.ons	in	

Forensic		Science	&	Informa.on	

Technology	

at	NIST

Dr.	Mar'n	Herman	

Informa'on	Technology	Laboratory	(ITL)	

Na'onal	Ins'tute	of	Standards	and	Technology	

mar'n.herman@nist.gov
Outline	

• Overview	and	mission	of	NIST	Informa6on	Technology	

Laboratory	(ITL)	

• Examples	of	past	ITL	accomplishments	in	forensics	

• Strategic	research	challenges	

• Addressing	the	research	challenges	
NIST	Laboratories	

©	R.	Rathe

©	R.	Rathe

J.	Burrus
Informa6on	Technology	Laboratory	(ITL)	

ITL	Mission:		

To	promote	US	innova6on	

and	industrial	

compe66veness	by	

advancing	

measurement	science,	

standards,	and	

technology	

through	research	and	

development	in	

informa6on	technology,	

mathema6cs,	and	

sta6s6cs.	

• Computer	Security	

• Mathema6cs		(Applied	&	Computa6onal)	

• Networking	

• Sta6s6cs	

• SoOware		

• Informa6on	Access	

• Usability	

• Biometrics	

• Video	&	audio	analy6cs	

• Cloud	Compu6ng	

• Complex	Systems	

• Forensic	Science	

• Health	Informa6on	Technology	

• Trusted	Iden66es	in	Cyberspace	

• Pervasive	Informa6on	Technology	

• Quantum	Informa6on	

• Smart	Grid	

• Vo6ng	Standards	

• Virtual	Measurement	Systems	
Examples	of	Past	ITL	Accomplishments	in	

Forensics	

Achieving	greater	reliability,	accuracy,	

validity	and	throughput	of	forensic	

analyses	
ANSI/NIST-ITL	Standard	

• Basis	for	biometric	data	exchange	

• Enhances	reliability	and	throughput	of	

biometric	iden6fica6on	

• Required	to	exchange	fingerprints	with	FBI,	

RCMP,	NATO,	DoD,	DHS,	INTERPOL,	other	

organiza6ons	in	100	na6ons	

* Fingerprints, Palmprints & Footprints

(Exemplars & latent prints)

* Forensic Markups of Images (2D & 3D feature 

points, contour mapping, extended feature 

set markups for latent prints)

* Iris * DNA * Associated Images

* Body Images (Face, Scars, Marks, Tattoos, 

Pattern Injuries, Identifying Characteristics) 

* Dental forensics * Voice Contact:	Brad	Wing,	NIST	
Contacts:	Jonathon	Phillips,	NIST	

				Patrick	Grother,	NIST		

ITL	Face	Recogni.on	Test	&	

Evalua.on		

• Independent	government	evalua6ons	of	

commercial	and	prototype	Face	Recogni6on	

technologies	

• Tests	recogni6on	performance	of	systems	

• Results	help	iden6fy	future	research	direc6ons	

• Has	resulted	in	driving	down	error	rates	in	stateof-the-art	systems	by	two	orders	of	magnitude	in	

last	20	years	
Contacts:	Barbara	Gu?man,	NIST		

				Doug	White,	NIST	

Na.onal	SoEware	Reference	Library		

• Used	to	iden6fy	soOware	resident	on	digital	media	

• Enhances	reliability,	accuracy,	repeatability	and	

throughput	in	computer	forensics	

• Used	daily	by	virtually	all	computer	forensics	

opera6ons	in	the	U.S.	(and	many	interna6onally)	

• Both	criminal	and	civil	inves6ga6ons	
Contacts:	Barbara	Gu?man,	NIST		

					James	Lyle,	NIST	

					Rick	Ayers,	NIST	

Computer	Forensic	Tool	Tes.ng	

• Scien6fic	test	methodology	and	tes6ng	for	

computer	and	mobile-device	forensic	tools	

• Determines	reliability	and	validity	of	tools	

• Helps	users	select	appropriate	forensic	tools;	

helps	toolmakers	improve	tools	

• Helps	determine	Daubert	admissibility	of	tools	

					in	legal	proceedings	
Current	Forensics	Projects	in	ITL	

• Forensic	Human	Iden6ty	

– Fingerprints	(latent	&	ten-prints)	

– Face	recogni6on	

– Speaker	recogni6on	

– Palm	prints,	plantars	(footprints)	

– Iris	recogni6on,	DNA,	ta`oos		

– Dental	records,	bite	marks	

• Digital	Forensics	

– Computer	forensics	

– Mobile	device	forensics	

– Cloud	compu6ng	forensics	

– Cybersecurity	incident	handling	

• Mul6media	Forensics

– Video/image/audio	analy6cs	

and	content	extrac6on	

• Surveillance	&	mul6media	event	

detec6on	and	recogni6on	

• Search	for	persons,	objects	or	

places	

• Person	tracking	

• Sta6s6cs	for	Forensics	

– Sta6s6cal	analysis	for		

• Illicit	drug	measurements	

• Markings	on	bullets	&	casings	

• Ballis6c	markings	database	and	

matching	

Measurements,	Standards,	&	Technology	
Goals	of	ITL	Forensic	Science	

Advance	measurements	and	standards	infrastructure	for	forensic	

science	through	computer	science,	math,	sta6s6cs	and	human	

factors.

Goals:	 – Improve	accuracy	and	reliability	

– Determine	sources	of	human	error	and	reduce	human	observer	bias	

– Develop	scien6fic	underpinnings	

– Determine	scien6fic	and	sta6s6cal	validity	of	forensic	methods	and	

standards	

– Establish	sta6s6cal	founda6ons,	including	measures	of	uncertainty	

– Develop	compu6ng	technologies	and	reference	data	

– Enhance	usability	and	interoperability	

NAS		

Report	
ITL	Cri.cal	Themes	in	Forensic	Science

• Image	and	PaIern	Analysis	

• Measurement	and	Uncertainty	

• Interoperability	of	Forensic	Data	

• Automated	Forensic	Compu.ng	Technologies
Strategic	ITL	Research	Challenges	in	Forensics	

• Scien6fic		underpinnings	

• Sta6s6cal	founda6ons	

• Valida6on	studies	

• Human	bias	and	error	

• Compu6ng	technologies	and	reference	data	

• Usability	

• Interoperability	
Scien6fic	Underpinnings	Challenges	

• Wide	variability	in	scien6fic	founda6ons	

across	forensic	science	disciplines	

– Par6cular	issues	in	pa`ern-based	and	digital	

forensic	disciplines	

• Need	to	solidify	underpinnings	in	

measurement	science	and	technology	for	

paIern-based	forensics	and	digital	forensics	

• Research	needed	to	address	accuracy,	

reliability,	and	validity	
• Measures	for	quality	of	pa`ern	data	and	digital	data	

• Measures	for	reliability,	repeatability,	reproducibility,	

accuracy,	validity	(in	pa`ern	and	digital	disciplines)	

• Measurement	science	to	advance	pa`ern-based	

forensics	

– Variability	in	features	of	pa`erns	

– Informa6on	content	of	pa`erns	

– Characteriza6on	of	search	and	matching	

– Characteriza6on	of	signal	and	image	processing	techniques	

• Measurement	science	to	advance	digital	forensics	

– Cloud	compu6ng	

– Mobile	devices	

– Mul6ple	devices	and	opera6ng	systems	

– Pervasive	encryp6on	

– Explosion	in	size	of	storage,	file	formats,	data	

• Metrology	for	real-6me,	in-the	field	analysis	in	pa`ern	&	

digital	forensics	

• Theore6cal	limits	of	reliability	and	accuracy	

Specific	Examples	 • AFIS	scien6fic	underpinnings	

• Quality	measures	of	latent	prints	

• Superimposed	fric6on	ridge	marks	

• Characterize		latent	image	

enhancements	

• Data	quality	for	forensic	speaker	

recogni6on	

• Impression	evidence:	measures	of	

informa6on	content,	probability	

models,	sta6s6cs	of	comparisons		

• Network	forensics	underpinnings	

• Cloud	compu6ng	forensics	

• Quality	of	video,	audio,	and	images	

• Bloodstain	Image	enhancement/

analysis	

• Forensic	shape	metrology:		

– matching	of	hair	and	fibers;	teeth	and	their	

marks,	skulls	and	bones;	par6cles	from	

explosives,	cements,	sand,	etc.;	ta`oos	and	

6re/shoe	impressions	

Poten6al	ITL	Research	to	Address

Scien6fic	Underpinnings	
Strategic	ITL	Research	Challenges	in	Forensics	

• Scien6fic		underpinnings	

• Sta6s6cal	founda6ons	

• Valida6on	studies	

• Human	bias	and	error	

• Compu6ng	technologies	and	reference	data	

• Usability	

• Interoperability	
Sta6s6cal	Founda6ons	for	Forensic	Science	

NIST	currently	par6cipates	with	BIPM	in	GUM	and	VIM	to	

provide	basis	for	measuring	uncertainty	

GUM:	Joint	Commi`ee	for	Guides	in	Metrology.	Evalua'on	

of	measurement	data	–	Guide	to	the	expression	of	

uncertainty	in	measurement.	Interna6onal	Bureau	of	

Weights	and	Measures	(BIPM),	Sèvres,	France,	September	

2008.	

VIM:	Joint	Commi`ee	for	Guides	in	Metrology.	Interna'onal	

vocabulary	of	metrology	–	Basic	and	general	concepts	and	

associated	terms	(VIM).	Interna6onal	Bureau	of	Weights	and	

Measures	(BIPM),	Sèvres,	France,	2008.	
Sta6s6cal	Founda6ons	Challenges	

• In	many	forensic	disciplines,	there	is	no	

well-defined	system	for	determining	error	

rates	

• Establish	uniqueness	of	marks	or	features	

in	paIerns	

• Develop	improved	measures	of	uncertainty	

for	forensic	measurements	and	for	

conclusions	of	forensic	tests	and	analyses	

• Develop	addi.onal	sta.s.cal	tools	and	

prac.ces	and	integrate	them	into	forensic	

tests	and	analyses	
Poten6al	ITL	Research	to	Address

Sta6s6cal	Founda6ons

• Metrology	and	methodology	for	determining	

error	rates	

• Improved	measures	for	uncertainty	

• Sta6s6cal	tools	for	extrac6ng/analyzing	

digital	evidence	

• Sta6s6cal	tools	to	establish	uniqueness	of	

features	in	large	popula6ons	

• Sta6s6cal	tools/techniques	for	use	by	

prac66oners	to	ensure	

– use	of	proper	experimental	design	in	tes6ng	

evidence	

– proper	sta6s6cal	analysis	of	test	results	

– formula6on	of	valid	forensic	inferences	

• Sta6s6cal	tools	for	use	in	valida6on	studies	

Specific	Examples	

• Latent	fingerprints:	

– Measures	of	informa6on	

content,	probability	models,	

and	sta6s6cs	of	comparisons		

• Handwri6ng:		

– Quan6fy	conclusions	

– Uncertainty	determina6on		

– Uniqueness	of	handwri6ng	

features	

• Uncertainty	measures	in	

digital	forensics	

• Uncertainty	measures	in	3D	

crime	scene	models	

• Sta6s6cal	tools	to	interpret	

trace	evidence	or	

toxicology	levels	
Strategic	ITL	Research	Challenges	in	Forensics	

• Scien6fic		underpinnings	

• Sta6s6cal	founda6ons	

• Valida6on	studies	

• Human	bias	and	error	

• Compu6ng	technologies	and	reference	data	

• Usability	

• Interoperability	
Valida6on	Challenges	

• Many	forensic	methods,	prac.ces,	and	

technologies	need	further	valida.on,	

including	

– PaIern-based	forensics	

– Digital	forensics	

– Interoperability	standards	and	technologies	

– Automated	compu.ng	technologies	

• Valida.on	studies	are	needed	that	

determine	how	well	these	approaches	

perform	under	a	variety	of	condi.ons	of	

use	
Poten6al	ITL	Research	to	Address

Valida6on	Studies

• Rigorous	methodology	for	valida6ng	

forensic	methods,	standards,	

prac6ces,	and	tools	

– Accuracy	and	reliability	required	to	be	

scien6fically	and	sta6s6cally	determined	

under	a	broad	range	of	condi6ons	

– Es6mate	uncertainty	or	errors	

• Metrology	and	methodology	for	

determining	strengths	and	limita6ons	

of	forensic	procedures	

• Develop	testbeds	for	performing	

valida6on	tests	and	experiments	

Specific	Examples	 • NIST	Computer	Forensic	Tool	

Tes6ng	(CFTT)	

– E.g.,	vola6le	memory	and	live	

forensics,	triage,	6melining

• Forensic	speaker	recogni6on	

evalua6on	series	

– Test	and	evalua6on	of	automated	

systems			

– T	&	E	of	combined	human	experts	

and	automated	speaker	

recogni6on	

• Automated	latent	AFIS:		

– Large-scale	benchmark	

evalua6ons:	automa6c	feature	

extrac6on	and	matching	as	well	

as	standardized	features	handmarked	by	human	experts	

• Quan6ta6ve	computer	models	

and	metrics	for	bloodstain	pa`ern	

forma6on		
Strategic	ITL	Research	Challenges	in	Forensics	

• Scien6fic		underpinnings	

• Sta6s6cal	founda6ons	

• Valida6on	studies	

• Human	bias	and	error	

• Compu6ng	technologies	and	reference	data	

• Usability	

• Interoperability	
Human	Bias	and	Error	Challenges	

• Need	to	understand	sources	of	unconscious	bias	

• Need	to	develop	mi.ga.on	strategies	

• Need	to	develop	test	methodology	to	determine	and	

evaluate	human	error	and	the	sources	of	error	

• Need	to	develop	metrology	to	characterize	and	quan.fy	

human	error	
Poten6al	ITL	Research	to	Address

Human	Bias	and	Error

• Work-flow	studies	of	prac66oners	to	

develop	sources	of	human	bias	and	

error	

• Measure	repeatability	and	

reproducibility	of	prac66oner	

prac6ces	

• Metrology	for	human	error	

• Develop,	test,	and	evaluate	

mi6ga6on	strategies		

Specific	Examples	 • Latent	examiner	bias:	metrology,	

test	methodology	

• Handwri6ng	examiner	bias,	errors,	

and	decision-making	

• Work-flow	studies	on	bloodstain	

pa`ern	examiners	to	determine	

human	bias,	errors,	decision	making	

• NSF	Workshop*	recommended	

studies,	e.g.,	Study	#3:	

What	are	examiners	doing	during	

forensic	comparisons?		

– What	details	do	they	look	for	and	

what	is	the	degree	of	variability	

among	qualified	examiners?		

– Do	examiners	use	implicit	

decision	rules	for	drawing	

conclusions?	

*	Workshop	on	Cogni6ve	Bias	and	Forensic	

Science,	Northwestern	University	School	of	

Law,	September	23-24,	2010		
Strategic	ITL	Research	Challenges	in	Forensics	

• Scien6fic		underpinnings	

• Sta6s6cal	founda6ons	

• Valida6on	studies	

• Human	bias	and	error	

• Compu6ng	technologies	and	reference	data	

• Usability	

• Interoperability	
Compu6ng	Technologies	and	Reference	Data	Challenges	

• Compu6ng	technologies	and	reference	data	sets	can	lead	to	

greater	accuracy,	reliability,	func6onality,	and	throughput	

• New	technologies	and	data	sets	need	to	be	developed,	

tested	and	validated	
Poten6al	ITL	Research	to	Address

Compu6ng	Technologies	and	Reference	Data	

• Test/evaluate	effec6veness	of	

compu6ng	technologies	for	forensic	

analysis	

– Exis6ng	technologies	

– New	technologies	

• Test	proposed	standards,	guidelines,	

valida6on	methods,	and	user	interfaces	

related	to	automated	compu6ng	

technology	

• Develop	testbeds

• Foster	innova6on	through	challenge	

problems	

• Develop	and	validate	reference	data	sets	

Specific	Examples	 • Accelerate	AFIS	latent	matching	

through	large-scale	open	evalua6ons		

• Automated	face	recogni6on	

systems:	evalua6ons	to	accelerate	

technologies	for	robust	recogni6on	

• Content-based	video	search	

challenge	problems	and	evalua6ons	

• Evalua6ons	of	automated	

handwri6ng	search	and	matching	

• Forensic	text	analysis:	Accelerate	

technologies	through	evalua6ons	

• Content-based	event	detec6on	in	

video:	Accelerate	technologies	

through	evalua6ons	

• DNA	soOware	tools	

– Expert	system	tools	to	help	the	DNA	

prac66oner	become	more	

produc6ve	and	improve	DNA	data	

quality	
Strategic	ITL	Research	Challenges	in	Forensics	

• Scien6fic		underpinnings	

• Sta6s6cal	founda6ons	

• Valida6on	studies	

• Human	bias	and	error	

• Compu6ng	technologies	and	reference	data	

• Usability	

• Interoperability