FINAL by: OSAC Program Office; Version 2, January 13, 2020 1

OSAC RESEARCH NEEDS ASSESSMENT FORM

Title of research need: R1: Examiner Reliability Study: White Box Study on Footwear and Tire 

Examiners

Describe

the need:

This is a revision (R1) to the previously listed need “Examiner Reliability Study: Black/White 

Box Study on Footwear and Tire Examiners.” The revision emphasizes and focuses 

exclusively on the immediate need of white box studies.

Understand the evidence evaluation and comparison process conducted by trained 

footwear/tire examiners. The research outcomes should determine one or more of the 

following: (1.) Quantify intra-examiner, inter-examiner and examiner-layperson variability 

in evaluating the quality/quantity of information present in impressions, (2.) Quantify intraexaminer, inter-examiner and examiner-layperson variability in opinions and evaluations as 

a function of the test taker’s education and discipline-specific training and experience, (3.) 

Identify aspects of the exam process and evidence that are sources of consistency in 

reporting conclusions, (4.) Identify aspects of the exam process and evidence that are 

sources of variability in reporting conclusions, (5.) Elucidate the process by which 

examiners assess and interpret footwear/tire impression evidence (e.g., quality, 

sufficiency, etc.), (6.) Evaluate adherence to a specific conclusion/opinion scale.

Note: Practitioner involvement in providing subject matter expertise during the planning 

phase of this research is highly encouraged in order to ensure that the research outcomes 

have applicability to casework, and the test samples are as realistic as possible under the 

research constraints.

Keyword(s): Footwear, Tires, Variability, Interpretation, Consistency, Adherence

Submitting subcommittee(s): Footwear & Tire Date Approved: 10/24/2022

Background Information:

1. Does this research need address a gap(s) in a current or planned standard? (ex.: Field identification system 

for on scene opioid detection and confirmation)

N/A

2. Are you aware of any ongoing research that may address this research need that has not yet been published 

(e.g., research presented in conference proceedings, studies that you or a colleague have participated in but 

have yet to be published)?

FBI Footwear Black Box Study
FINAL by: OSAC Program Office; Version 2, January 13, 2020 2

3. Key bibliographic references relating to this research need: (ex.: Toll, L., Standifer, K. M., Massotte, D., eds. 

(2019). Current Topics in Opioid Research. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-180-3)

Jacqueline A. Speir, Nicole Richetelli, Lesley Hammer. Forensic Footwear Reliability: Part I—Participant 

Demographics and Examiner Agreement. J Forensic Sci, Vol. 65, No. 6 (2020). doi: 10.1111/1556-4029.14553

Nicole Richetelli, Lesley Hammer, and Jacqueline A. Speir. Forensic Footwear Reliability: Part II—Range of 

Conclusions, Accuracy, and Consensus. J Forensic Sci, Vol. 65, No. 6 (2020). doi: 10.1111/1556-4029.14551 

Nicole Richetelli, Lesley Hammer, and Jacqueline A. Speir. Forensic Footwear Reliability: Part III—Positive 

Predictive Value, Error Rates, and Inter-Rater Reliability. J Forensic Sci, Vol. 65, No. 6 (2020). doi: 10.1111/1556-

4029.14552 

L. Hammer, K. Duffy, J. Fraser, N. Nic Daeid. A study on the variability in footwear impression comparison 

conclusions. Journal of Forensic Identification. Vol 63, No. 2 (2013). pp. 205-218. 

B. T. Ulery, R. A. Hicklin, J. Buscaglia, M. A. Roberts, Accuracy and reliability of forensic latent fingerprint 

decisions. Proceedings of the National Academy of Sciences of the United States of America. Vol. 108, No. 19 

(2011). pp.7733–7738. 

B. T. Ulery, R. A. Hicklin, M. A. Roberts, J. Buscaglia. Measuring what latent fingerprint examiners consider 

sufficient information for individualization determinations. PLOS ONE. Vol. 9, No. 11(2014). e110179 pp. 1- 16.

R. A. Hicklin, J. Buscaglia, M.A. Roberts, S. B. Meagher, W. Fellner, M. J. Burge, M. Monaco, D. Vera, L.R. Pantzer, 

C.C. Yeung,T. N. Unnikumaran. Latent Fingerprint Quality: A Survey of Examiners. Journal of Forensic 

Identification.Vol. 61, No. 4(2011). pp. 385-418. 

G. M. Langenburg. A Performance Study of the ACE-V Process: A Pilot Study to Measure the Accuracy, Precision, 

Reproducibility, Repeatability, and Biasability of Conclusions Resulting from the ACE--V Process. Journal of 

Forensic Identification.Vol. 59, No. 2(2009). pp. 219-256. 

H. Majamaa, & A. Ytti. A Survey of the Conclusions Drawn of Similar Footwear Cases in Various Crime 

Laboratories. Forensic Science International. Vol. 82, No. 1 (1996). pp.109-120. 

National Research Council. Strengthening Forensic Science in the United States: A Path Forward; The National 

Academies Press: Washington, D.C. (2009). Chapter 5, p.148.

4. Review the annual operational/research needs published by the National Institute of Justice (NIJ) at 

https://nij.ojp.gov/topics/articles/forensic-science-research-and-development-technology-working-groupoperational#latest? Is your research need identified by NIJ?

Yes

5. In what ways would the research results improve current laboratory capabilities?

The results of this research would be considered by the footwear/tire examiner community, laboratories and 

accrediting bodies in order to implement necessary changes to the methodology, standard operating procedures, 

training programs and other quality assurance practices to reduce examiner error, minimize intra- and interexaminer variation in evidence evaluation.
FINAL by: OSAC Program Office; Version 2, January 13, 2020 3

6. In what ways would the research results improve understanding of the scientific basis for the 

subcommittee(s)?

Examinations of footwear/tire impression can be extremely complex, and the factors influencing a comparison 

are not static from case to case. The variables at play are the value of the evidence (i.e., quality, quantity, clarity 

and limitations) and the examiner’s education, training and experience. As a consequence, a degree of variation 

in evaluation is expected. The results of this research would provide a better understanding of factors that 

influence the identification of comparative features, and the value attributed to those features during the 

decision-making process. Factors include: the quality of the evidence; class characteristics and randomly 

acquired characteristics (quality, quantity, clarity, complexity, number and size); the examiner’s education, 

training and experience; examiner certification and laboratory accreditation; and peer review.

7. In what ways would the research results improve services to the criminal justice system?

The research results would provide the ability to understand (and if possible, quantify) the conditions/factors 

that influence the examiner’s attention, and his or her ability to analyze footwear/tire impression evidence and 

accurately interpret findings. Further, the results would be extremely valuable in promoting transparency, 

objectivity, and the communication between experts and laypersons, particularly within the criminal justice 

system.

8. Status assessment (I, II, III, or IV): II

Major gap in 

current 

knowledge

Minor gap in 

current 

knowledge

No or limited

current research is 

being conducted I III

Existing current 

research is being 

conducted II IV

This research need has been identified by one or more subcommittees of OSAC and is being provided as an 

informational resource to the community.