Version	1.1	/	Date	of	Issue:	2.09.16 /	Issuing	Authority:	Forensic	Science	Standards	Board	(FSSB)

OSAC	RESEARCH	NEEDS	ASSESSMENT	FORM

Title	of	research	need: Examiner	Reliability	Study:	Black/White	Box	Study	on	Footwear	and	Tire	

Examiners

Keyword(s): Footwear,	Tires,	Conclusions,	Variability,	Accuracy,	Error

Submitting	subcommittee(s): Footwear	&	Tire Date	Approved: 07-Mar-2016

(If	SAC	review	identifies	additional	subcommittees,	add	them	to	the	box	above.)

Background	Information:

1. Description	of	research	need:

Evaluate	the	diversity	of	the	conclusions	provided	by	trained	footwear/tire	examiners	when	conducting	

comparisons	of	footwear/tire	impression	evidence	test	samples.	Additionally,	evaluate	the	diversity	of	the	

conclusions	provided	by	trained	footwear/tire	examiners	versus	those	provided	by	laypersons	when	

conducting	comparisons	of	footwear/tire	impression	test	samples.	The	research	outcomes	should	do	some	

or	all	of	the	following:		(1.)	Quantify	intra-examiner,	inter-examiner	and	examiner-layperson	variability	as	a	

function	of	the	quality/quantity	of	the	evidence	provided,	(2.)	Quantify	intra-examiner,	inter-examiner	and	

examiner-layperson	variability	as	a	function	of	the	test	taker’s	education	and	discipline-specific	training	and	

experience,	(3.)	Identify	aspects	of	the	exam	process	and	evidence	that	are	sources	of	consistency	in	

reporting	conclusions,	(4.)	Identify	aspects	of	the	exam	process	and	evidence	that	are	sources	of	variability	

in	reporting	conclusions,	(5.)	Elucidate	the	process	by	which	examiners	assess	and	interpret	footwear/tire	

impression	evidence	(e.g.,	quality,	sufficiency,	etc.).		Note:		Practitioner	involvement	in	providing	subject	

matter	expertise	during	the	planning	phase	of	this	research	is	highly	encouraged	in	order	to	ensure	that	the	

research	outcomes have	applicability	to	casework,	and	the	test	samples	are	as	realistic	as	possible	under	the	

research	constraints.

2. Key	bibliographic	references	relating	to	this	research	need:

L.	Hammer,	K.	Duffy,	J.	Fraser,	N.	Nic	Daeid.	A	study	on	the	variability	in	footwear	impression	comparison	

conclusions.	Journal	of	Forensic	Identification.	Vol	63,	No.	2	(2013).	pp.	205-218.

B.	T.	Ulery,	R.	A.	Hicklin,	J.	Buscaglia,	M.	A.	Roberts,	Accuracy	and	reliability	of	forensic	latent	fingerprint	

decisions.	Proceedings	of	the	National	Academy	of	Sciences	of the	United	States	of	America.	Vol.	108,	No.	

19	(2011).	pp.7733–7738.

B.	T.	Ulery,	R.	A.	Hicklin,	M.	A.	Roberts,	J.	Buscaglia.	Measuring	what	latent	fingerprint	examiners	consider	

sufficient	information	for	individualization	determinations.	PLOS	ONE.	Vol.	9,	No.	11 (2014).	e110179	pp.	1-
Version	1.1	/	Date	of	Issue:	2.09.16 /	Issuing	Authority:	Forensic	Science	Standards	Board	(FSSB)

16.

R.	A.	Hicklin, J.	Buscaglia,	M.A.	Roberts,	S.	B.	Meagher,	W.	Fellner,	M.	J.	Burge,	M.	Monaco,	D.	Vera,	L.R.	

Pantzer,	C.C.	Yeung, T.	N.	Unnikumaran.	 Latent	Fingerprint	Quality:	A	Survey	of	Examiners.	Journal	of	

Forensic	Identification. Vol.	61,	No.	4 (2011).	 pp.	385-418.

G.	M.	Langenburg.	 A	Performance	Study	of	the	ACE-V	Process:	A	Pilot	Study	to	Measure	the	Accuracy,	

Precision,	Reproducibility,	Repeatability,	and	Biasability	of	Conclusions	Resulting	from	the	ACE-V	Process.	

Journal	of	Forensic	Identification. Vol.	59,	No.	2 (2009).		pp.	219-256.

H.	Majamaa,	&	A.	Ytti.	 A	Survey	of	the	Conclusions	Drawn	of	Similar	Footwear	Cases	in	Various	Crime	

Laboratories.	Forensic	Science	International.		Vol.	82,	No.	1	(1996).		pp. 109-120.

National	Research	Council.	Strengthening	Forensic	Science	in	the	United	States:	A	Path	Forward;	The	

National	Academies	Press:	Washington,	D.C.	(2009).	Chapter	5,	p. 148.

3a. In	what	ways	would	the	research	results	improve	current	laboratory	capabilities?

The	results	of	this	research	would	be	considered	by	the	footwear/tire	examiner	community,	laboratories	

and	accrediting	bodies	in	order	to	implement	necessary	changes	to	the	methodology,	standard	operation	

procedures,	training	programs	and	other	quality	assurance	practices	to	reduce	examiner	error	and	minimize	

intra-/inter-examiner	variation	in	reporting	conclusions.

3b. In	what	ways	would	the	research	results	improve	understanding	of	the	scientific	basis	for	the	

subcommittee(s)?

Examinations	of	footwear/tire	impression	can	be	extremely	complex,	and	the	factors	influencing	a	

comparison	are	not	static	from	case	to	case.	The	variables	at	play	are	the	value	of	the	evidence	(i.e.,	quality,	

quantity,	clarity	and	limitations)	and	the	examiner’s	education,	training	and	experience.	As	such,	a	degree	of	

variation	in	opinion	is	expected.	The	results	of	this	research	would	provide	a	better	understanding	of	factors	

that	influence	the	comparative	and	decision-making	process.	Those	factors	include:	the	quality	of	the	

evidence;	class	characteristics	and	randomly	acquired	characteristics	(quality,	quantity,	clarity,	complexity,	

number	and	size);	the	examiner’s	education,	training	and	experience;	examiner	certification	and	laboratory	

accreditation;	and	peer	review.

3c.		In	what	ways	would	the	research	results	improve	services	to	the	criminal	justice	system?

The	research	results	would	provide	the	ability	to	understand	(and	if	possible,	quantify)	the	

conditions/factors	that	influence	the	examiner’s	ability	to	analyze	footwear/tire	impression	evidence	and	

accurately	interpret	their	findings.	Further,	the	results	would	be	extremely	valuable	in	promoting	

transparency,	objectivity,	and	the	communication	between	experts	and	laypersons,	particularly	within	the	

criminal	justice	system.
Version	1.1	/	Date	of	Issue:	2.09.16 /	Issuing	Authority:	Forensic	Science	Standards	Board	(FSSB)

4.		Status	assessment	(I,	II,	III,	or	IV): I Major gap	in	

current	

knowledge

Minor	gap	in	

current	

knowledge

No	or	limited

current	research	

is	being	conducted

I III

Existing current	

research	is	being	

conducted

II IV

This	research	need	has	been	identified	by	one	or	more	subcommittees	of	OSAC	and	is	being	provided	as	an	

informational	resource	to	the	community.
Version	1.1	/	Date	of	Issue:	2.09.16 /	Issuing	Authority:	Forensic	Science	Standards	Board	(FSSB)

Approvals:

Subcommittee Approval	date: 07-Mar-2016

(Approval	is	by	majority	vote	of	subcommittee.	Once	approved,	forward	to	SAC.)

SAC

1.		Does	the	SAC	agree	with	the	research	need? Yes x No

2.		Does	the	SAC	agree	with	the	status	assessment? Yes x No

If	no,	what	is	the	status	assessment	of	the	SAC:

Approval	date: 17-Mar-2016

(Approval	is	by	majority	vote	of	SAC.		Once	approved,	forward	to	NIST	for	posting.)