Version	1	/	Date	of	Issue:	10.29.15	/	Issuing	Authority:	Forensic	Science	Standards	Board	(FSSB)

OSAC	Research	Needs Assessment Form

Title	of	research	need:

Keywords:

Submitting subcommittee(s): Date	Approved:

(If	SAC	review	identifies	additional	subcommittees,	add	them	to	the	box	above.)

Background	information:

1.	 Description	of	research	need:

2.	 Key	bibliographic	references	relating	to	this	research	need:

Proficiency	tests	are	an	essential	part	of	a	forensic	DNA	laboratory’s	quality	assurance	program	and	

the	current	FBI	QAS	document	specifies	that	analysts,	technical	reviewers	and	technicians	shall	

undergo	semi-annual	external	proficiency	testing	in	each	technology	performed.	This	is	a	valuable	

system	for	monitoring in-house	competency	and	performance and	identifying	intra- and	interlaboratory	differences.	Current	proficiency	tests	target	the	complete	forensic	biology	workflow	

starting	with	body	fluid	determination.	Because	there	has	to	be	a	known	consistent	outcome	for	the	

analysis	these	tests	cannot	be	based	on	compromised	or	complex	samples	that	could	potentially	fail	

to	yield	a	result	even	if	handled	properly.	There	is	no	current	commercially	available	proficiency	

test	addressing	complex	data	interpretation,	except	tests	for	kinship	laboratories	that	may	include	

a	paper	challenge	for	complex	pedigrees.	Several	collaborative	studies	dealing	with	mixtures	or	

population	database	searches	have	shown	the	benefits	of	comparing intra- and inter-laboratory	

performance.				Several US	laboratories	have	implemented	internal	competency	monitoring	for	

mixture	interpretation	and	statistical	evaluations,	e.g.	by	assigning	complex	data	sets	to	specific	

analysts.	Research	should	explore	the	appropriate	design	for	these	types of	data	sets,	and	how	to	

set	ground	truth	expectations	for	statistical	evaluations,	especially	if	probabilistic	tools,	with	data	

simulation	steps	are	involved.	

National	Commission	on	Forensic	Science	(2016)	Proficiency	testing	in	forensic	science.	Views	

document	on	https://www.justice.gov/ncfs/work-products

Federal	Bureau	of	Investigation	(2000)	Quality	Assurance	Standards	for	Forensic	DNA	Testing	

Laboratories.	Forensic	Science	Communications,	July	2000,	Volume	2,	Number	3.

JWDe	Keijser,	M Malsch,	ET	Luining, et	al,	(2016)	Differential reporting	of	mixed	DNA	profiles	and	

its	impact	on	jurists’ evaluation	of evidence. An	international	analysis. Forensic	Science	

International:	Genetics http://dx.doi.org/10.1016/j.fsigen.2016.03.006

L	Prieto,	C	Alves,	B	Zimmermann	et	al	(2013)	GHEP-ISFG	proficiency	test	2011:	paper	challenge	on	

evaluation	of	mitochondrial	DNA	results. Forensic	Science	International:	Genetics 7:10-15	

M	Crespillo,	PA	Barrio,	JA	Luque	et	al	(2014)	GHEP-ISFG	collaborative	exercise	on	mixture	profiles	

of	autosomal	STRs	(GHEP-MIX01,	GHEP-MIX02	and	GHEP-MIX03):	Results	and	evaluation.	Forensic	

Science	International:	Genetics 10:64-72.

Proficiency	testing	for	complex	data	interpretation	and	biostatistical	

evaluations	

DNA	mixtures,	proficiency	testing,	statistics, probabilistic	genotyping	,	reliability,	

consistency

BDRIC 8/25/16
Version	1	/	Date	of	Issue:	10.29.15	/	Issuing	Authority:	Forensic	Science	Standards	Board	(FSSB)

3a.		In	what	ways	would	the	research	results	improve	current	laboratory	capabilities?

3b.		In	what	ways	would	the	research	results	improve	understanding	of	the	scientific	basis	for	the	

subcommittee(s)?

3c.		In	what	ways	would	the	research	results	improve	services	to	the	criminal	justice	system?

4.		Status assessment	(I,	II,	III,	or	IV):		

This	research	need	has	been	identified	by	one	or	more	subcommittees	of	OSAC	and	is	being	provided	as	an	

informational	resource	to	the	community.

Major gap	

in	current	

knowledge

Minor gap	

in	current	

knowledge

No	or	limited

current	research	

is	being	

conducted

I III

Existing current	

research	is	being	

conducted

II IV

Complex	data	interpretation	and	biostatistical	evaluations	are	amongst	the	most	challenging	

aspects	of	forensic	DNA	testing.	External	proficiency	tests	to	be	used	by	a	variety	of	laboratories	

would allow	technical	leaders	to	identify	performance	issues	not	only	on	an	individual	analyst’s	

level	but	also	for	the	system	as	a	whole.	Internal	competency	monitoring	may	not	catch	

problematic	polices	and	threshold	decisions	that	affect	conclusions	and	weight	of	the	evidence.	

Laboratories	have	been	compiling	internal	data	sets	and	designing	competency	tests	for	mixture	

interpretation	and	statistics.	The	BDRIC	is	poised	to	issue	a	best	practice	recommendation	

encouraging	this	quality	measure.	It	would	be	helpful	to	get	professional	proficiency	test	designers	

involved and be	able	to	recommend	this	for	inter-laboratory	monitoring	as	well.			

External	proficiency	tests	for	complex	data	interpretation	and	biostatistical	evaluations	would	

result	in	improved	quality	monitoring	across	different	laboratories.	Legal	community	stakeholders	

would	have	a	mechanism	to	view	the	results	for	their	local	forensic	science	provider	service	and	

can	identify	their	adherence	to	generally	accepted	interpretation	and	calculation	principles.	This	

will	improve	the	overall	reliability	of	conclusions	and	statistical	weight	presented	in	court.	

I
Version	1	/	Date	of	Issue:	10.29.15	/	Issuing	Authority:	Forensic	Science	Standards	Board	(FSSB)

// Approval	date:

(Approval	is	by	majority	vote	of	subcommittee.		Once	approved,	forward	to	SAC.)

1.		Does	the	SAC	agree	with	the	research	need?				Yes		X⃝							No							⃝		

2.		Does	the	SAC	agree	with	the	status	assessment?				Yes				X	⃝							No							⃝	

If	no,	what	is	the	status assessment	of	the	SAC:

Approval	date:

(Approval	is	by	majority	vote	of	SAC.		Once	approved,	forward	to	NIST	for	posting.)

Subcommittee 04/29/2016

SAC

08/24/2016