Firearms and Tool Marks

Presented by:

Johannes A. Soons

Core Team Members

Zhe Chen1, Brian Renegar1, Richard Silver1, John Song1, Michael Stocker1

, 

Robert Thompson2, Ted Vorburger1, James Yen3, Nien-Fan3 Zhang, Alan Zheng1

1Physical Measurement Laboratory

2Special Programs Office

3Information Technology Laboratory
Disclaimer

Certain commercial equipment, instruments, or materials are identified in 

this presentation to specify the experimental procedure adequately. Such 

identification is not intended to imply recommendation or endorsement by 

the National Institute of Standards and Technology (NIST), nor is it intended 

to imply that the materials or equipment identified are necessarily the best 

available for the purpose.
Firearm and Tool Mark Identification

• Were the evidence tool marks produced by the same tool?

• Was an evidence mark produced by the evidence tool?

Source: SWGGUN

Firearm “tools”

• Revolver

• Pistol

• Rifle

• Shotgun

Common tool mark evidence

• Bullet

• Cartridge case

aegisacademy.com

Non-firearm tools

Common tool mark evidence

• Screwdriver

• Pry bar

• Wire cutter

• Pliers

• Lock pick

• Lock

• Safe

• Window still

• Wire

Source: SWGGUN
Bullets and Cartridge Cases – Regions of Interest

Land Engraved Areas (LEAs)

Bullets have striated tool marks from the barrel rifling.

Firing pin impression

Breech face impression

Ejector mark

Casings constitute over 90 % of the 

NIBIN national database entries for 

firearm identification. Aperture shear (striae)

Cartridge cases have impressed and striated tool marks from various sources

From Centralia College – H. Ebiari
Firearm Identification – Current Practice

Compare class characteristics - Measurable features 

that indicate a restricted group source

Compare individualizing tool marks (subjective):

• Does the agreement exceed the best agreement 

demonstrated between tool marks from different tools?

• Is the agreement consistent with the agreement 

demonstrated by tool marks from the same tool?

Render an opinion:

• Identification

• Exclusion

• Inconclusive

• Unsuitable

AFTE Theory of Identification, AFTE Journal – Volume 43, Number 4, Page 287, 2011.

Toolmarks on cartridge cases

Striated toolmarks on bullets
Current Practice is Under Scrutiny

• NAS 2009 “..the decision of the toolmark examiner remains a 

subjective decision based on unarticulated standards and no 

statistical foundation for estimation of error rates.” 

• PCAST 2016: “PCAST finds that firearms analysis currently falls 

short of the criteria for foundational validity, because there is 

only a single appropriately designed study to measure validity 

and estimate reliability.”

• PCAST 2016: “A second – and more important – direction is ... to 

convert firearms analysis from a subjective method to an 

objective method...”

• The National Research Council, “Strengthening Forensic Science in the United States—A Path Forward”, Washington DC, 2009.

• President’s Council of Advisors on Science and Technology, “Forensic science in criminal courts: Ensuring scientific validity of feature-comparison methods”, Washington DC, 2016.
NIST Firearms and Tool Marks Focus Area

Goals:

• Metrology infrastructure for objective firearm and tool mark examination 

• Scientifically-justified protocols to quantify the weight of the evidence. 

Focus:

• Measurement methods, quality assurance, and standards.

• Objective comparison metrics and algorithms

• Knowledge base for similarity and variability of tool marks

• Quantitative expressions for the weight of evidence.
From 2D to 3D

• Higher reproducibility and focus on actual topography

• Measure once, compare often

• Well suited for numerical analysis

• Already common for database search

• Virtual comparison microscopy is ready for case work

Comparison microscopy Virtual comparison microscopy

2D reflectance microscopy images 3D topography images
Physical Standards for Measurement 

Traceability and Quality control

• Provide SRM bullets and cartridge cases.

• Provide reference images for comparison.

• Laboratories regularly check their 

measurements with the reference.

SRM 2461 Standard 

Cartridge Case

SRM 2460 Standard Bullet

Enter measured image into NIBIN Track similarity score with reference image

T. Vorburger, et al., “The Second National Ballistics Imaging Comparison (NBIC-2),” J. Res. Natl. Inst. Stand. Technol., 2014.

SRM 2460a

Standard Bullet Replica
Standards for Individual Error Sources and 

Measurement Uncertainty Evaluation

mirror

2D grid

roughness Half-sphere

Artifact Suite

Chirped target

Results are applicable to a 

wider range of measurands 

and enable check of 

instrument specifications
Objective Similarity Metrics

Area or Profile Similarity

(e.g., correlation coefficient)

Number/Quality of Matching 

Features

Congruent Matching Striae (CMS)

Reference Sample Questioned Sample Reference Sample Questioned Sample
Congruent Matching Cells

Breech face impressions from different firearms

Breech face impressions from the same firearm

24 CMCs

0 CMCs

Filtered Reference Surface A

A1 A2 A3 A4

A5 A6 A7

A8 A9 A10

A11 A12 A13

A14 A15 A16

A17 A18 A19

A20 A21 A22 A23 A24 A25

A26 A27 A28 A29

-1500 -1000 -500 0 500 1000 1500

X - Position [ m]

-1500

-1000

-500

0

500

1000

1500

Y - Position [ m]

-21.95

-1.85

-1.00

0.00

1.00

1.85

5.55

m Filtered Compared Surface B

B1 B2 B3

B4

B5 B6 B7

B8 B9 B10

B11

B12

B13

B14

B15

B16

B17

B18

B19

B20

B21

B22

B23

B24

B25

B26

B27

B28

B29

-2000 -1500 -1000 -500 0 500 1000 1500 2000

X - Position [ m]

-2000

-1500

-1000

-500

0

500

1000

1500

2000

Y - Position [ m]

-8.67

-2.06

-1.25

0.00

1.25

2.06

8.98

m

Filtered Reference Surface A

A1 A2 A3 A4

A5 A6 A7

A8 A9 A10

A11 A12 A13

A14 A15 A16

A17 A18 A19

A20 A21 A22 A23 A24 A25

A26 A27 A28 A29

-1500 -1000 -500 0 500 1000 1500

X - Position [ m]

-1500

-1000

-500

0

500

1000

1500

Y - Position [ m]

-21.95

-1.85

-1.00

0.00

1.00

1.85

5.55

m Filtered Compared Surface B

B1

B2

B3

B4

B5

B6

B7

B8

B9

B10

B11

B12B13

B14

B15B16

B17

B18

B19

B20

B21

B22

B23

B24

B25

B26

B27

B28

B29

-1500 -1000 -500 0 500 1000 1500 2000

X - Position [ m]

-1500

-1000

-500

0

500

1000

1500

Y - Position [ m]

-4.94

-1.81

-1.00

0.00

1.00

1.81

7.51

m

J. Song, “Proposed NIST Ballistics Identification System 

(NBIS) using 3D Topography Measurements on Correlation 

Cells”, AFTE Journal, 45 (2), 184-194, 2013.
Open-access research database of firearm tool marks on 

bullets and cartridge cases:

• Firearms representing major class/subclass characteristics.

• Consecutively manufactured firearm components.

• Firearm firing many rounds (persistence/decay).

• Firearm firing different ammunition brands.

• Firearms known to present identification challenges.

www.nist.gov/forensics/ballisticsdb
1 1 1

5

1

3

6

2 2

3

1 1

2 2

4

5 5

2 2

1

0

10

20

30

40

50

Relative Frequency KM (%)

Frequency of Comparison Scores

612

116

10

C = 6

0 5 10 15 20 25 30 35

Number of Congruent Matching Cells

0

20

40

60

80

100

Relative Frequency KNM (%)

Same Source - KM (50)

Different Source - KNM (738)

Ruger

Characterizing the Weight of Evidence

• Characterize score distributions 

for known matching and known 

non-matching comparisons

• Characterize the weight of 

evidence for a particular score

• Error rates

• Likelihood ratio

• ....

J.F. Song, et al., "Estimating Error Rates for Firearm Evidence 

Identifications in Forensic Science," Forensic Science 

International, Vol. 284, pp. 15-32, (March 2018)
5 5

1 1

2

1 1

3

1

2 2 2

4

1

3

4 4

1

2 2

3 3

2

1 1 1 1

4

1 1

0

10

20

30

40

50

Relative Frequency KM (%)

Frequency of Comparison Scores

446

174

35

3

C = 6

0 5 10 15 20 25 30 35 40 45 50

Number of Congruent Matching Cells

0

20

40

60

80

100

Relative Frequency KNM (%)

Same Source - KM (65)

Different Source - KNM (658)

1 1 1 1

4

1 1 1 1

2 2 2

7

3

5

1

5

6

5

1

5

1

2

4

2

1

2

1

0

10

20

30

40

50

Relative Frequency KM (%)

Frequency of Comparison Scores

548

119

12 2

C = 6

0 5 10 15 20 25 30 35 40 45 50

Number of Congruent Matching Cells

0

20

40

60

80

100

Relative Frequency KNM (%)

Same Source - KM (69)

Different Source - KNM (681)

2

1

2 2 2 2

1 1 1

4

2

4 4

5

3 3

1

3

1 1

0

10

20

30

40

50

Relative Frequency KM (%)

Frequency of Comparison Scores

558

125

21

1

C = 6

0 5 10 15 20 25 30 35

Number of Congruent Matching Cells

0

20

40

60

80

100

Relative Frequency KNM (%)

Same Source - KM (45)

Different Source - KNM (705)

2

1

2

1 1 1

2

1 1 1 1 1 1 1 1

3

2 2 2

4

3

1 1

2

1 1 1

2 2

0

10

20

30

40

50

Relative Frequency KM (%)

Frequency of Comparison Scores

616

79

9 1

C = 6

0 5 10 15 20 25 30 35 40 45 50

Number of Congruent Matching Cells

0

20

40

60

80

100

Relative Frequency KNM (%)

Same Source - KM (45)

Different Source - KNM (705)

1 1 1

5

1

3

6

2 2

3

1 1

2 2

4

5 5

2 2

1

0

10

20

30

40

50

Relative Frequency KM (%)

Frequency of Comparison Scores

612

116

10

C = 6

0 5 10 15 20 25 30 35

Number of Congruent Matching Cells

0

20

40

60

80

100

Relative Frequency KNM (%)

Same Source - KM (50)

Different Source - KNM (738)

• No sample triage.

• Consistent distribution of 

CMC scores for differentsource comparisons.

• Results Indicate potential for 

low false positive error rates.

Initial Results “Population Study” (CMC Breech Face)

Colt Glock Ruger

Smith & Wesson Sig Sauer
Major Activities

1. Quality assurance, reference artifacts, and 

documentary standards

2. Metrics and algorithms for objective identification

3. Tool mark database for research and validation

4. Quantitative evaluation weight of evidence

5. Specifications for NIBIN interoperability

6. Tool mark identification for non-firearm tools

ATF

1 1 1

5

1

3

6

2 2

3

1 1

2 2

4

5 5

2 2

1

0

10

20

30

40

50

Relative Frequency KM (%)

Frequency of Comparison Scores

612

116

10

C = 6

0 5 10 15 20 25 30 35

Number of Congruent Matching Cells

0

20

40

60

80

100

Relative Frequency KNM (%)

Same Source - KM (50)

Different Source - KNM (738)
Thank You

Zhe Chen, Wei Chu, Brian Renegar, Richard Silver, John Song, Michael Stocker, 

Robert Thompson, Ted Vorburger, James Yen, Nien-Fan Zhang, Alan Zheng

soons@nist.gov