FINAL by: OSAC Program Office; Version 2, December 1, 2020 1

OSAC RESEARCH NEEDS ASSESSMENT FORM

Title of research need: Study to Assess the Accuracy and Reliability of Firearm and Toolmark 

Examinations

Keyword(s): black box, toolmarks, validation, firearms, forensic science, error rate

Submitting subcommittee(s): Firearms & Toolmarks Date Approved: March 5, 2021

(If SAC review identifies additional subcommittees, add them to the box above.)

Background Information:

1. Does this research need address a gap(s) in a current or planned standard? (ex.: Field identification system 

for on scene opioid detection and confirmation)

Studies designed to assess examiner accuracy and reliability in source attribution-type conclusions (i.e. error rate 

studies).

2. Are you aware of any ongoing research that may address this research need that has not yet been published 

(e.g., research presented in conference proceedings, studies that you or a colleague have participated in but 

have yet to be published)?

Unknown

3. Key bibliographic references relating to this research need: (ex.: Toll, L., Standifer, K. M., Massotte, D., eds. 

(2019). Current Topics in Opioid Research. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-180-3)

• Baldwin, D.P., Bajic, S.J., Morris, M., and Zamrow, D., "A Study of False-Positive and False-Negative Error Rates in 

Cartridge Case Comparisons", Ames Laboratory, USDOE Technical Report #IS-5207, April 7, 2014.

• Ulery BT, et al. “Accuracy and Reliability of Forensic Latent Print Decisions” 2011 108(19) pp. 7733-7738.

4. Review the annual operational/research needs published by the National Institute of Justice (NIJ) at 

https://nij.ojp.gov/topics/articles/forensic-science-research-and-development-technology-working-groupoperational#latest? Is your research need identified by NIJ?

Yes, “Determination of accuracy and reliability of forensic analyses and conclusions, including potential sources of 

error”.

5. In what ways would the research results improve current laboratory capabilities?

Error rate studies have the potential to provide laboratories with information on what types of errors occur, the 

frequency of errors and why they may occur. This can help shape laboratory quality programs in order to help 

mitigate errors.
FINAL by: OSAC Program Office; Version 2, December 1, 2020 2

6. In what ways would the research results improve understanding of the scientific basis for the 

subcommittee(s)?

Numerous studies have been performed, however additional studies with different designs will continue to provide 

the subcommittee with knowledge about errors. This data will help guide our work product regarding analytical, 

documentation and case review standards.

7. In what ways would the research results improve services to the criminal justice system?

This research would provide additional data to help the justice system assess the overall error rate of the firearm and 

toolmark discipline and fulfill one the Daubert criteria (error rate).

8. Status assessment (I, II, III, or IV): IV

Major gap in 

current 

knowledge

Minor gap in 

current 

knowledge

No or limited

current research is 

being conducted I III

Existing current 

research is being 

conducted II IV

This research need has been identified by one or more subcommittees of OSAC and is being provided as an 

informational resource to the community.