Firearms and Toolmarks

Physics and Pattern SAC

Todd Weller

AFTE 2018
Subcommittee Leadership

Position Name Organization Email

Chair Todd Weller Weller Forensics toddweller@wellerforensics.com

Vice Chair Andy Smith San Francisco Police Dept 

Crime Lab 

andy.smith@sfgov.org 

Executive 

Secretary

Wendy Gibson VA Dept of Forensic 

Science

Wendy.gibson@dfs.virginia.gov

2

In October 2017, Andy Smith stepped down as Chair and Todd Weller was elected Chair
Subcommittee Voting Members

# Name Organization

1 Steve Scott FBI

2 Mike Neel BATF

3 Xiaoyu Alan Zheng NIST

4 Ted Vorburger, PhD NIST

5 Eric Collins Contra Costa Co Sheriff Forensic Services

6 Doug Murphy FBI

7 Max Morris, PhD Iowa State University

8 Chris Monturo Miami Valley Regional Crime Lab

9 Erich Smith FBI

10 James Carroll Los Angeles County Sheriff’s Office

11 Cassey Allen TX Department of Public Safety - Austin

12 Ryan Lilien, MD, PhD Cadre Research Labs 

13 Ron Nichols Nichols Forensic Science Consulting

14 Bill Demuth Illinois State Police

15 Melissa Oberg* Indiana State Police Crime Lab

16 Erica Lawton* Alabama Department of Forensic Science

17 Nicholas Petraco* John Jay College

3

*Melissa Oberg, Erica Lawton, and Nicholas Petraco are new members starting late 2017
Subcommittee Affiliate Members

# Name Organization

1 Jodi Marsanopoli BATF

2 John Murdock Contra Costa County Crime Laboratory

3 Nancy McCombs California Department of Justice, Fresno

4 John Marshall RCMP Forensic Lab, Ottawa

5 Gerhard Wevers Institute of Environmental Science and Research, 

New Zealand

6 Carey Alvarez Bacha Aurora Police Department Crime Laboratory

4
5

Employer Classification Mix of Voting 

Members

20%

25%

20%

10%

20%

5%

Employer Classification

Federal Government

State Government

Local Government

Private Forensic Pracitioner

Academic/Researcher

Industry Technology Partner
OSAC Firearms & Toolmarks Website

• Google “OSAC Firearms”

• Top Hit

7

https://www.nist.gov/topics/forensic-science/organization-scientific-areacommittees-osac/firearms-and-toolmarks

• SWGGUN legacy documents, research needs, draft standards, etc.
OSAC Firearms & Toolmarks Website

8
Overview & Roadmap of FATM 

Subcommittee

9

Firearm Examination TG

Standards for day to day 

laboratory examinations

Novel Technology TG

Standards for emerging 

technology; e.g. 3D 

imaging and comparison 

algorithms 

Uncertainty of 

Measurement TG

Standards for 

measurement uncertainty 

and quantifying decisions

Training TG

Standards new examiners 

and training program

Executive Committee

Manage task groups (TG) and performs 

administrative tasks of subcommittee

Terminology

Standards for FATM 

terminology 

Criteria for Identification TG

Standards regarding 

comparison conclusions

Winter 2015: FATM subcommittee formed six task groups to focus on 

the most pressing/important topic areas in the discipline. The same 

task groups still exist and continue to work on standards and guidelines 

in each topic area.
Task Group 1: Firearm and 

Toolmark Examination Standards

• Develop standard methods for the examination of firearm and 

toolmark evidence

• E.g. Day-to-day examination protocols and test methods

Task Group Name: Firearm and Toolmark Examination Standards 

Task Group Chair Name: Jim Carroll*

Members: Steve Scott, Cassey Allen, Erica Lawton

Task Group Chair Contact Information: JPCarrol@lasd.org

10

*Jim Carroll accepted leadership of this task group in late 2017
Firearm & Toolmark Examination Task Group 

Progress

Document Title Status

Standard Guideline/Best Practice for the Safe Handling 

of Firearms and Ammunition 

Sent to ASB

Standard Test Method for the Examination and Testing 

of Firearms

Draft in Progress

11
Task Group 2: Novel Technology

• Provide standards for new measurement and comparison 

methods to enhance the process of toolmark examinations.

• Specify hardware quality assurance protocols and validation 

requirements for software and statistical conclusions.

• Specify processes for utilizing new technologies in forensic 

laboratories. This includes hit verification, virtual microscopy, 

inter-lab data exchange, dispute resolution, and training.

Task Group Name: Technology Development, Validation, and Implementation 

Task Group Chair Name: Dr. Ryan Lilien

Members: Alan Zheng, Erich Smith, Dr. Nick Petraco

Affiliate Members: John Marshall

Task Group Chair Contact Information: ryan@cadreresearch.com 

12
Technology Development, Validation, and 

Implementation Task Group Progress

Document Title Document Status

Standard for Topography Comparison Software

for Firearm and Toolmark Analysis

Sent to ASB

Standard for Implementation of 3D Technologies in 

Forensic Laboratories for Firearm and Toolmark 

Analysis

Sent to ASB

Standard for 3D Measurement Systems and 

Measurement Quality Control for Firearm and 

Toolmark Analysis

Sent to ASB

Virtual Microscopy Initial Draft in Progress

13
Task Group 3: Uncertainty of 

Measurement

• Standards and procedures to express the uncertainty for 

measurements of physical quantities that matter for firearm and 

toolmark analysis; 

• Includes: issues associated with quantifying uncertainty in decision 

making.

Task Group Name: Uncertainty of Measurement

Task Group Chair Name: Dr. Ted Vorburger

Members: Chris Monturo, Dr. Max Morris, Dr. Nick Petraco

Affiliate Members: Carey Alvarez Bacha

Task Group Chair Contact Information: tvtv@nist.gov 

14
Uncertainty of Measurement Task Group 

Progress

Document Title Status

Standard for Barrel and Overall Length Measurements for 

Firearms, includes guidelines and example spreadsheet for 

calculating uncertainty

Sent to ASB

Uncertainty for Trigger Pull Measurement Draft in progress

Statistical Standard/Guideline for Source Identification and 

Related Quantities

Initial draft in progress

15
Task Group 4: Training Program

• Standards for Examiner Qualifications and Training:

• Minimum Requirements for an Examiner Training Program

• Firearms & Toolmarks

• Other Examinations: serial number restoration, distance determination

• Competency Testing

• Continuing Education

• Minimum Qualifications for Firearm/Toolmark Examiners

• Trainees

Task Group Name: Training Program

Task Group Chair Name: Doug Murphy*

Members: Bill Demuth, Melissa Oberg, 

Affiliate Members: Jodi Marsanopoli

Task Group Chair Contact Information: dpmurphy@fbi.gov

16

*Doug Murphy accepted leadership of this task group in late 2017
Training Task Group Progress

Document Title Status

Minimum Qualifications of Firearm/Toolmark 

Examiners Trainees

Final Vote of SAC prior to ASB

Standard Firearms Examinations Training Program Initial Draft in progress

17
Task Group 5: Standard Criteria for 

Source Conclusions

• Development of minimum standards for: 

• The criteria used to draw conclusions regarding the comparison of a 

questioned toolmark to a known or unknown source tool

• The supporting documentation for such examinations and conclusions 

• The reporting of source conclusions

Task Group Name: Standard Criteria for Source Conclusions

Task Group Chair Name: Eric Collins

Members: Mike Neel, Max Morris, Ron Nichols

Affiliates: John Murdock, Nancy McCombs, Gerhard Wevers

Task Group Chair Contact Information: ecoll@so.cccounty.us 

18
Source Conclusions Task Group 

Progress

Planned Actions Status

Standard Range of Source Conclusions and Criteria for Firearm 

and Toolmark Examinations

Adjudicating comments from 

RCs and SAC

Standard for Supporting Documentation of Source Conclusions 

– INITIAL DRAFT

Draft in progress

Expression of Source Conclusions – INITIAL DRAFT Draft in progress

Verification of Source Conclusions – INITIAL DRAFT Draft in progress

19
Topic 6: Terminology

• Reviewed existing glossaries to determine if terms and definitions 

are appropriate and sufficient for use and make adjustments or 

additions were necessary. 

• Working with other disciplines in attempt to standardize 

terminology

Task Group Name: Terminology

Task Group Chair Name: Bill Demuth

Members: Cassey Allen, Doug Murphy

Task Group Chair Contact Information: Bill_Demuth@isp.state.il.us 

20
Planned Actions Approx. Deadline

OSAC Subcommittee on Firearms and Toolmarks Glossary Adjudicating Comments from 

RCs and SAC

21

Terminology Task Group Progress
Research Needs

• In Jan 2016, FSSB asked that we highlight topics appropriate for 

additional research. Updated March 2018. Eight topics 

highlighted:

• Blind Verifications Needs Assessment

• Cognitive Bias: To What Extent Does It Affect Firearm and Toolmark 

Comparison Outcomes

• Consistency of Examiner Evaluation of Distance Determination GSR 

Patterns

• Effect of New Technology on Quantitative Consecutive Matching Striae 

ID Criteria

• Evaluation of Examiner Conclusions Utilizing a Five-Point Scale

• Resolution Requirements for 3D Virtual Comparison Microscopy

• Assessment of Examiners' Toolmark Categorization Accuracy

• Study to Assess the Accuracy and Reliability of Firearms and Toolmark 

Examinations

• More details available on website

22
Summary

• Progress starting to be tangible: five documents sent to external 

SDO (Academy Standards Board)

• Other task groups have documents far along in the OSAC 

internal review; expect more documents to be sent to SDO soon

• New standards being drafted (e.g. virtual microscopy, trigger pull 

uncertainty, training program)

• Future considerations: smoothing transitions 

• Onboarding new leadership (e.g. task group chairs, executive 

committee) and new members. Will be working to improve this process 

as term limits come up.

23
Term Limits

• Term limits: will be adding new members in coming years. 

24
New members are needed

• Term limits: will be adding new members. If interested, please 

apply:

• https://www.nist.gov/osac-application-form

25
Questions?

26