Response to the President’s Council of Advisors on Science and Technology (PCAST)

Submitted by: Firearm/Toolmark Subcommittee of the National Institute of Standards 

and Technology (NIST) Organization of Scientific Area Committees (OSAC)

Date: December 23, 2015

Question 1: What studies have been published in the past 5 years that support the 

foundational aspects of each of the pattern-based forensic science methods, including (but 

not limited to) latent print analysis; firearms/toolmarks; shoe/tire prints; bitemark analysis; 

questioned documents? What studies are needed to demonstrate the reliability and validity 

of these methods?

The following are literature citations for studies published in the past five years that provide 

foundational support to the discipline of firearm and toolmark comparison. Although these 

citations respond specifically to this Council’s focus within the last five (5) years, it should be 

noted that a plethora of important literature has been generated outside this time constriction, 

which was reported in 2011 to the Research, Development, Testing and Evaluation 

subcommittee on Forensic Science Interagency Working Group (RDT&E IWG).

Scientific practice demands that possible exceptions be researched and published (efforts to 

test or falsify), and that a large body of confirmatory evidence from training programs, 

experimentation, etc., will forever remain unpublished.

It is the opinion of The Firearms/Toolmarks subcommittee of the Organization of Scientific Area 

Committees (OSAC) that the profession and science of firearm and toolmark comparison rests 

on a solid scientific foundation. The citations below represent a minor selection from a much 

larger body of work that encompases nearly a century of research and experiential knowledge. 

Despite this confidence, the professional community continues to perform new research and 

welcomes the scientific method of vigilant and rigorous testing of the underlying principles of 

the discipline. New studies using three dimensional measurement instruments and comparison 

software have provided objective data that supports the range of conclusions used by the 

profession. 

A short summary or abstract follows each citation.

Firearms Identification, Bullets
Intelligent Automation, Incorporated, “A Statistical Validation of the Individuality of Guns Using 

High Resolution Topographical Images of Bullets”, National Institute of Justice Grant #2006-DNBX-K030, October, 2010.

This was a study of marks on fired bullets by a topography based (3D) automated system. This 

study continued the analysis of a previous 2005 NIJ bullet study and validated the original 

premise of Firearm/Toolmark ID. This study also concluded that 1) the ability to determine that 

a given bullet was fired from a specific barrel depends on the individual barrel itself and not 

only on the brand of its manufacture, and 2) the performance of the automated analysis system 

used in this study is not representative of that of a trained firearms examiner as humans have a 

remarkable ability to perform pattern matching that is difficult to be replicated in any 

automated system.

Fadul, T. G., “An Empirical Study to Evaluate the Repeatability and Uniqueness of 

Striations/Impressions Imparted on Consecutively Manufactured Glock EBIS Gun Barrels”, AFTE 

Journal, Volume 43, Number 1, Winter 2011, pp. 37-44.

This paper describes an empirical study of ten consecutively manufactured Glock barrels 

containing the Enhanced Bullet Identification System (EBIS). Study consisted of test sets sent to 

238 examiners from 150 laboratories in 44 states and 9 countries that were designed to test the 

examiner’s ability to correctly identify fired bullets to the barrel that fired them. The results 

from 183 of these examiners produced an error rate of 0.4%. This study validated the 

repeatability and uniqueness of striated markings in gun barrels, as well as the ability of a 

competent examiner to reliably identify fired bullets to the barrels that marked them.

Mikko, D., et al., “Reproducibility of Toolmarks on 20,000 Bullets fired through an M240 

Machine Gun Barrel”, AFTE Journal, Volume 44, Number 3, Summer 2012, pp. 248-253.

This article discusses the reproducibility of toolmarks on 7.62mm high velocity bullets fired 

through a single M240 machine gun barrel. Over the years, there have been several research 

studies and published articles pertaining to consecutively manufactured rifled barrels and the 

ability to microscopically identify bullets as having been fired through the same barrel of a 

firearm; however, to the knowledge of the authors, there has not been any in-depth 

microscopic study pertaining to 20,000 bullets being fired through a single rifled barrel and 

subsequently identified to that particular barrel. This study was designed to provide credible 

evidence in regards to the reproducibility and uniqueness of striations on the bearing surfaces 

of fired bullets. Despite changes to the reproducibility of some of the individual markings over 

the course of the study, the authors were able to correctly identify the barrel of origin for each 

of the collected fired bullets. See subsequent related article: 
Mikko, D. and Miller, J., “An Empirical Study/Validation Test Pertaining to the Reproducibility of 

Toolmarks on 20,000 Bullets Fired Through M240 Machine Gun Barrels”, AFTE Journal, Volume 

45, Number 3, Summer 2013, pp. 290-291.

Chu, et al., “Automatic Identification of Bullet Signatures Based on Consecutive Matching Striae 

(CMS) Criteria”, Forensic Science International, Volume 231, 2013, pp. 137-141.

This paper described a study of fired bullet markings from ten consecutively manufactured 

firearm barrels by an automated 3D signature analytic method. This study used 3D topography 

image capture technology with acquisitions that were cross-correlated to existing firearm 

Consecutive Matching Striae (CMS) identification criteria. Results provided a fairly objective test 

that demonstrated support for these firearm CMS criteria.

Monkres, J, et al., “Comparison and Statistical Analysis of Land Impressions from Consecutively 

Rifled Barrels”, AFTE Journal, Volume 45, Number 1, Winter 2013, pp. 3-20.

The validity and reliability of firearm and toolmark analysis has been debated, often revolving 

around the subjectivity of the methods examiners use. This study attempts to evaluate 

examiners’ conclusions through objective computer analysis. Bullets, knowns and unknowns, 

fired through ten consecutively rifled barrels were used for the study. Unknown bullets were 

identified to the barrels from which they had been fired using traditional comparison 

techniques. Each land impression (LI) was photographed, and the distances of the prominent 

striae to one shoulder of the LI were measured using computer software. Two methods of 

selecting measurable striae were used. The data from these measurements was then converted 

into a barcode representative of the LI from which it was taken. Barcodes were subjected to 

Principle Component Analysis (PCA), and a Support Vector Machine (SVM) was employed to 

evaluate the computer's ability to correctly identify which LI was represented by the barcode, 

based on SVM analysis error rate (ideal error rate =5%). Optimal error rate varied based on 

selection technique, with 19.444% and 1.149% being the optimal values obtained by each 

method. The second result, generated by the majority of bullets analyzed, indicated the 

computer was able to adequately group barcodes according to their common origins, 

supporting the examiner's identifications. This research and described methodology may 

provide support for the reliability of firearm and toolmark analysis.

Wong, C., “The Inter-Comparison of 1,000 Consecutively-Fired 9mm Luger Bullets and Cartridge 

Cases from a Ruger P89 Pistol Utilizing both Pattern Matching and Quantitative Consecutive 

Matching Striae as Criteria for Identification”, AFTE Journal, Volume 45, Number 3, Summer 

2013, pp. 267-272.

Previous studies have investigated the effect of consecutive firing of firearms to determine how 

the wear on barrels and breechfaces would affect the identification of fired bullets and 
cartridge cases. This study was conducted to determine if the toolmarks on fired bullets and 

cartridge cases would change significantly after firing 1,000 cartridges through a Ruger P89 

9mm Luger semiautomatic pistol, while using both pattern matching and quantitative 

consecutively matching striae (QCMS) as identification criteria during the comparison process. 

While there were some differences between the toolmarks on the bullets and cartridge cases 

throughout the firing sequence, each bullet and cartridge case was successfully identified to the 

first bullet or cartridge case.

Mikko, D. and Miller, J., “An Empirical Study/Validation Test Pertaining to the Reproducibility of 

Toolmarks on 20,000 Bullets Fired Through M240 Machine Gun Barrels”, AFTE Journal, Volume 

45, Number 3, Summer 2013, pp. 290-291.

This article is a follow-up to an article that was published in the AFTE Journal-Volume 44, 

Number 3-Summer 2012, titled “Reproducibility of Toolmarks on 20,000 Bullets fired through 

an M240 Machine Gun Barrel”. Using a second M240 Machine gun with its original barrel, along 

with a new spare barrel assembly, thirty (30) additional bullets were test fired through both 

barrels and subsequently inter-compared blindly by four firearm and toolmark examiners, one 

of which had just completed his formal two-year training period. Additionally, the recovered 

(60) test fired bullets from both barrels were also mixed with the 127 bullets recovered during 

the test firing of 20,000 bullets in the reproducibility study and examined by the four firearm 

and toolmark examiners in a blind test study, in order to determine whether or not the 

examiners could correctly identify or eliminate the bullets as being fired through the correct 

barrel. A total of 164 questioned fired bullets were examined, which resulted in 164 correct 

answers from the participants in the study (zero percent error rate).

Rahm, J., “Evaluation of an electronic comparison system and implementation of a quantitative 

effectiveness criterion”, Forensic Science International, Volume 214, 2012, pp 173–177.

The basis of an expansive database and electronic comparison system (Evofinder) used by the 

BKA in Germany is evaluated and a mathematical value is proposed to rate the correlation 

quality. This effectiveness criterion can be valuable to give an objective assessment of different 

electronic comparison systems. Additionally, the applicability of the system on different calibres 

and land engraved area (LEA) width is discussed. The so called scores are also on disposition 

and their benefit to a decision-making is debated. The article also shows results for cartridge 

cases. 

Firearms Identification, Cartridge Cases
LaPorte, D., “An Empirical Validation Study of Breechface Marks on .380 ACP Caliber Cartridge 

Cases Fired from Ten Consecutively Finished Hi-Point Model C9 Pistols”, AFTE Journal, Volume 

43, Number 4, Fall 2011.

An empirical study was conducted using ten (10) consecutively finished Hi-Point model C9 slides 

and one frame acquired from the Hi-Point Manufacturing Company in Mansfield, Ohio. The ten 

(10) slides were mounted on the frame and test fired to obtain cartridge cases for comparison. 

The test fired cartridge cases were microscopically examined, evaluated and compared for class 

and individual characteristics that resulted from the manufacturing process. Prominent 

striations were evident on each test-fired cartridge case. These resulted from sanding of the 

breech face. The variations that occur during the manufacturing process of sanding result in 

unique, identifiable, individual breech face marks devoid of subclass influence. A limited 

validation study was conducted after the empirical study. Correct associations were made 

during this limited study.

Thompson, R., Song J., Zheng A., and Yen J., “Cartridge Case Signature Identification Using 

Topography Measurements and Correlations: Unification of Microscopy and Objective 

Statistical Methods”, National Institute of Standards and Technology, Presented at the 18th 

European Network of Forensic Science Institutes (ENFSI) Conference, Lisbon, Portugal, October, 

2011.

A comparison microscope employing the standard optical comparison method and confocal 

microscopy, with subsequent cross-correlation topography analysis, were used to correctly 

identify cartridge cases fired from ten consecutively made pistol slides. 

Subsequent cross correlation function analysis and statistical analysis of match and non-match 

scores correctly identified the fired cartridge cases back to their respective known slide source 

in 19 of 20 occasions with one inconclusive result. Results of the mathematical determination 

of slide source were compared to the validated results from the microscopic comparisons.

Petraco, D. K., et al., “Application of Machine Learning to Toolmarks: Statistically Based

Methods for Impression Pattern Comparisons”, NIJ/NCJRS Document #239048, Award #2009-

DN-BX-K041, July 2012.

This was a statistical study that evaluated 3D quantitative surface topographies of toolmarks, 

consisting of fired cartridge cases, screwdriver and chisel striations, generated using confocal 

microscopy. Principal component and canonical variate analysis, as well as support vector 

machine methodology, was used to objectively associate these toolmarks with the tools that 

produced them. Estimated toolmark identification error rates were approximately 1% using 

these algorithmic methods. The findings of this objective and quantitative scientific research 

support the general conclusions codified in the AFTE Theory of Identification.
Mayland, B., and Tucker, C., “Validation of Obturation Marks in Consecutively Reamed 

Chambers”, AFTE Journal, Volume 44, Number 2, Spring, 2012, pp. 167-169.

This study of fired cartridge cases from ten consecutively manufactured firearms was 

conducted to determine the reproducibility and reliability of obturation marks from reamed 

chambers for identification purposes. Results of this empirical study, which consisted of sixtyfour (64) participants from nineteen (19) national laboratory systems, effected a sensitivity 

rating of 0.927.

These results demonstrate that obturation markings imparted on fired cartridge cases can be 

used as a reliable means of identification to the firearm that marked them.

Stowe, A., “The Persistence of Chamber Marks from Two Semiautomatic Pistols on Over 1,440 

Sequentially-Fired Cartridge Cases”, AFTE Journal, Volume 44, Number 4, Fall 2012, pp. 293-

308.

A Browning Hi-Power semiautomatic pistol and a Hi-Point model C semiautomatic pistol were 

test fired a total of 1,440 times each, and the chamber marks imparted to the fired cartridge 

cases were examined. Ammunition used included cartridges with cases made of aluminum, 

brass, and nickel-plated brass. Microscopic comparison of the chamber marks revealed that 

they were reproducible and identifiable up to 960 firings and that the metallic composition of 

the cartridge case does affect the reproducibility of the chamber marks.

Grom, T. L., “IBIS Correlation Results of Cartridge Cases Collected Over the Course of 500 Firings 

from a Glock Pistol”, AFTE Journal, Volume 44, Number 4, Fall 2012, pp. 361-363.

This study examines the capability of the IBIS system to find known matching fired cartridge 

cases that have been produced after the moderate use of a Glock firearm. A total of 500 

cartridges were fired from a Glock pistol. The individual characteristics of the breech face and 

firing pin persisted throughout the firings, and IBIS was able to properly correlate the known 

match within the top twenty results for each cartridge case entered.

Weller, T. J., et al., “Confocal Microscopy Analysis of Breech Face Marks on Fired Cartridge 

Cases from 10 Consecutively Manufactured Pistol Slides”, Journal of Forensic Sciences, Volume 

57, Number 4, July 2012, pp. 912-917.

This was a study of 90 test fired cartridge case specimens from ten consecutively manufactured 

pistol slides. A total of 8010 comparisons were conducted by using confocal microscopy with a 

3D cross-correlation analysis logarithm. The average match scores were 0.82 with the average 

non-match scores 0.20. There was no overlap of scores between matching and non-matching 

test scores. This study provided objective data that supports the AFTE Theory of Identification. 
Cazes, M. and Goudeau, J., “Validation Study Results from Hi-Point Consecutively Manufactured 

Slides”, AFTE Journal, Volume 45, Number 2, Spring 2013, pp. 175-177.

This study was designed to determine whether trained firearm and tool mark examiners could 

identify eight unknown fired cartridge cases to one of five consecutively manufactured 9mm HiPoint model C-9 pistol slides. The five slides were used to create a total of twenty-six (26) test 

sets, each containing a known/control set and an unknown set of fired cartridge cases. The 

participants were informed that the firing pin impressions, extractor marks, and ejector marks 

should not be used for identifying purposes, as the frame of the firearm (including the firing 

pin) was the same for all test sets. A total of sixty-nine (69) responses were received from 

participants that took part in the study. Over three-fourths of the participants used the 

technique of pattern matching only to complete this study, while the remainder used both 

pattern matching and consecutive matching striae (CMS). All of the participants reported 

correct results. There were no inconclusive responses and no incorrect responses validating the 

hypothesis that firearms examiners could differentiate between consecutively manufactured 

Hi-Point slides.

Fadul, T., et al., “An Empirical Study to Improve the Scientific Foundation of Forensic Firearm 

and Tool Mark Identification Utilizing Ten (10) Consecutively Manufactured Slides”, AFTE 

Journal, Vol. 45, Number 4, Fall 2013, pp. 376-389. 

This was an empirical study of marks produced from 10 consecutively Ruger brand 

manufactured pistol slides by 217 firearm examiners from 46 states and the District of 

Columbia. Results of this study established an error rate of less than 0.1%, and validated 

toolmark durability as these slides maintained their individual signature after multiple firings. 

Yong, J., et al., “Further Investigations into the Permanence of Breechface Recess and Other 

Marks on Cartridge Cases Discharged from 9mm Calibre Walther P99 Pistols”, AFTE Journal, 

Volume 46, Number 2, Spring 2014, pp. 138-142.

This report describes the permanence of the toolmarks on cartridge cases discharged from 9 

mm calibre Walther P99 pistols. Three weapons that were subjected to extensive firing in the 

years 2010 and 2012 were used for the study. The cartridge cases expended from the firearms 

in these two years were examined in order to verify whether the marks on them have been 

persistent. Results have shown that breechface recess marks, firing pin impression and firing 

pin aperture shear marks showed reproducibility. In addition, all the marks above except the 

breechface impression held sufficient individual characteristics for identification. Thus, the 

identity of the weapon from the expended cartridge cases from Walther P99 pistols after 

extensive firing could be determined. Significantly, the breechface recess marks presented 

themselves in all the three weapons as useful for comparison.
Stroman, A., “Empirically Determined Frequency of Error in Cartridge Case Examinations Using a 

Declared Double-Blind Format”, AFTE Journal, Volume 46, Number 2, Spring 2014, pp. 157-175.

This paper describes a no-gun empirical study of fired cartridge cases to determine the 

frequency of error in firearms identification using a declared double-blind testing format; i.e., a 

declared test containing blind elements. Seventy-four of seventy-five examiners accurately 

identified the questioned fired cartridge cases to the respective known specimens with no false 

positives. This study also demonstrated that examiners were able to accurately evaluate 

breechface markings avoiding mis-identifications from substantial subclass marks borne by the 

cartridge cases. 

Baldwin, D.P., et al., “A Study of False-Positive and False-Negative Error Rates in Cartridge Case 

Comparisons”, USDOE Technical Report # IS-5207 (April 7, 2014)

This report provides the details for a study designed to measure examiner error rates for false 

identifications and false eliminations when comparing an unknown to a collection of three 

known cartridge cases. Volunteer active examiners were provided with 15 sets of 3 known + 1 

unknown cartridge cases fired from a collection of 25 new Ruger SR9 handguns. The 

ammunition was all Remington 9-mm Luger. Responses were received from 218 participating 

examiners. The rate of false negatives was estimated as 0.367%. The overall rate of false 

positives was estimated as 1.01%. However, most of the errors were reported by a small 

number of examiners; that is, individual examiners have varying error rates. Laboratory error 

rates may be significantly lower than these individual rates if quality assurance procedures are 

applied that can effectively reduce or eliminate the propagation of false positives reported by 

individuals.

Song, J., “Proposed ‘Congruent Matching Cells (CMC)’ Method for Ballistic Identification and 

Error Rate Estimation,” AFTE Journal, Volume 47, Number 3, Summer 2015, pp177-185

Based on the concept of correlation cells, a Congruent Matching Cells (CMC) method is 

proposed for ballistic identification and error rate estimation using three sets characteristic 

parameters of the paired correlation cells: cross correlation function maximum CCFmax, spatial 

registration positions in x-y and registration phase angle θ. The proposed CMC method can be 

used for correlation of both geometrical topographies and optical images. The CMC parameters 

and algorithms are in the public domain and subject to open tests. Based on the CMC method, 

an error rate procedure for ballistic identifications is described, which uses binomial 

distributions to model correlation results for both matching and non-matching image pairs. 

Chu, Tong and Song, “Validation Tests for the Congruent Matching Cells (CMC) Method Using 

Cartridge Cases Fired with Consecutively Manufactured Pistol Slides”, AFTE Journal, Volume 45, 

Number 4, Fall 2013, pp. 361-366.
This was a study of ten (10) consecutively manufactured slides using 3D topography technology 

with correlations of paired breech marking correlation cells to establish firearm identifications. 

Test results showed significant separation between KM and KNM distributions without any false 

positive or false negative identification. 

Firearm and Toolmark Identification Theoretical

Wevers G., et al., “A Comprehensive Statistical Analysis of Striated Tool Mark Examinations; 

Part 2: Comparing Known Matches and Known Non-Matches using Likelihood Ratios”. AFTE 

Journal, Volume 43, Number 2, Summer 2011, pp. 137-145. 

In this article, a potential model for increasing the objectivity in the interpretation of toolmarks 

was explored using consecutively matching striae (CMS) and Bayesian inference. Given the 

nature of the data, standard statistical thinking suggests that Bayesian inference is likely to be 

the most powerful method of interpretation. The unavoidable paucity of data for high CMS runs 

for the known non-match condition was handled using a small advance in modelling. The 

resulting likelihood ratios showed some, but incomplete, separation between the known match 

and known non-match conditions. Although promising, the resulting incomplete separation 

between known matches and known non-matches was thought to represent limitations of the 

CMS summary of the complete pattern and limitations of the modelling used.

Petraco, D. K., et al., “Addressing the National Academy of Sciences’ Challenge: A Method for 

Statistical Pattern Comparison of Striated Tool Marks”, Journal of Forensic Sciences, Volume 57, 

Number 4, July 2012, pp. 900-911.

Toolmark test specimens from nine slotted screwdrivers were encoded into high-dimensional 

feature vectors and analyzed by multiple statistical pattern recognition methods. The statistical 

methods used, which are widely known and accepted in academic applications, rely on few 

assumptions of the data’s underlying distribution, can be accompanied by standard confidence 

levels, and are falsifiable. Correct classification rates of at least 97% were achieved.

Bunch, S. and Wevers, G., “Application of likelihood ratios for firearm and toolmark analysis”; 

Science and Justice, Volume 53, Issue 2, June 2013, pp. 223-229.

Historically firearm and toolmark examiners have rendered categorical or inconclusive opinions 

and eschewed probabilistic ones, especially in the United States. The authors of this article 

suggest this practice may no longer be necessary or desirable, and outline an alternative 

approach that is within a comprehensive logical/Bayesian paradigm. Hypothetical forensic and 
non-forensic examples are provided for readers who are practicing firearm and toolmark 

examiners, and the strengths and weaknesses of both approaches are considered.

Kerkhoff, W. , et al., The Likelihood Ratio Approach in Cartridge Case and Bullet Comparison,” 

AFTE Journal, Volume 45, Number 3, Summer 2013, pp 284-289

This article summarizes the different aspects of the discussion that led to the implementation 

of the likelihood ratio approach to firearms identification by the Firearms Section of the 

Netherlands Forensic Institute (NFI). The authors’ (three firearms examiners and a statistician) 

perspectives on the use of this approach in cartridge case and bullet comparison are shared.

Toolmark Identification

Bachrach B., Jain A., Jung S., Koons R.D., “A Statistical Validation of the Individuality and 

Repeatability of Striate Tool Marks: Screwdrivers and Tongue and Groove Pliers”, Journal of 

Forensic Sciences, Volume 55, Number 2, March 2010, pp. 348-357.

This study statistically validated the original premise of individuality in Toolmark Identification 

by analyzing statistical distributions of similar values resulting from the comparison of Known 

Matches (KM) and Known Non-Matched (KNM) pairs of striated toolmarks. This quantifiable 

analysis of KM and KNM toolmark similarity distributions showed nearly error-free 

identifications.

Chumbly, L. S., et al., “Validation of Tool Mark Comparisons Obtained Using a Quantitative, 

Comparative, Statistical Algorithm” Journal of Forensic Sciences, Volume 55, Number 4, July 

2010, pp. 953-961.

A statistical analysis and computational algorithm for comparing pairs of toolmarks by 

profilometry data was conducted. Toolmarks produced by 50 sequentially made screwdrivers, 

at selected fixed angles, were analyzed both empirically (by practicing examiners) and by 

established computational algorithms. The results of these comparisons, as well as a 

subsequent blind study with the participating examiners, showed scores of good agreement 

between the algorithm and human experts. It was also noted that in some of the examination 

phases, examiner performance was much better than the algorithm.

Petraco, N.D.K., et al, “Estimation of Striation Pattern Identification Error Rates by Algorithmic 

Methods, AFTE Journal, Volume 45, Number 3, Summer 2013, pp. 235-244.

This was a computational study that used algorithmic methods of toolmark striation patterns 

produced by screwdriver tips and firearm firing pin apertures in determining error rates. 
Multivariate analysis, as well as support vector machine methodology, was used to objectively 

associate these toolmarks with the tools that produced them. Estimated toolmark identification 

error rates were approximately 1% using these algorithmic methods. The findings of this 

objective and quantitative scientific research support the general conclusions codified in the 

AFTE Theory of Identification.

Zheng, X.A., et al, “2D and 3D Topography Comparisons of Toolmarks Produced from 

Consecutively Manufactured Chisels and Punches”, AFTE Journal, Volume 46, Number 2, Spring 

2014, pp. 143-147.

This paper described an automated blind study of toolmarks from consecutively made chisel 

and punches utilizing 2D and 3D topography analysis. hese analytical comparative results were 

expressed as a maximum value of the normalized Cross Correlation Function (CCF). Based on 

the CCF metric, all of the toolmarks were correctly identified to the tool that produced them. 

This study provides additional objective scientific support for the validity of Toolmark 

Identification. 

Ekstrand, et al, “Virtual Tool Mark Generation for Efficient Striation Analysis”, Journal of 

Forensic Sciences, Volume 59, Number 4, July 2014, pp. 950-959.

This was a follow-up study on Zhang and Chumbley’s research (See Emerging Research Section) 

regarding the development of virtual toolmarks by a 3-D computer simulation that would allow 

for the development of highly predictable toolmark characterizations. The initial study involved 

the production of test toolmarks by six screwdriver tips that were then compared by a 

previously developed statistical algorithm. 

Preliminary experimental results indicate that the use of a manipulative, virtual tool could 

provide quantitative data for the characterization of tool marked surfaces that would improve 

the scientific basis of toolmark identification. These results support the present theory and 

conclusions held in Toolmark Identification. 

King, E., “Validation Study of Computer Numerical Control (CNC), Consecutively Manufactured 

Screwdrivers”, AFTE Journal, Volume 47, Number 3, Summer 2015, pp. 171-176.

The purpose of this study was to perform a validation study to determine if screwdrivers that 

are consecutively manufactured using the computer numerical control (CNC) process can be 

identified by trained forensic examiners after having their class characteristics reproduced by 

striated toolmark samples. The results were based on participation from seven members of the 

Scientific Working Group for Firearms and Toolmarks (SWGGUN) and yielded an error rate of 

0.00%. This result provides support of toolmark identification in the scientific community, thus 

complying with the Daubert standard. These results further demonstrate the CNC consecutive-
manufacturing process did not eliminate the individual or class characteristics of the 

screwdrivers and does not interfere with the ability of examiners to correctly associate tools 

and the marks they leave on working surfaces.

M. Baiker, et al., Quantitative comparison of striated toolmarks, Forensic Science International 

Volume 242, 2014, pp 186–199.

In this study, an automated method was presented for objective comparison of striated marks 

of screwdrivers. The combination of multi-scale registration (alignment) of toolmarks, that 

accounts for shift and scaling, with global cross-correlation as objective toolmark similarity 

metric renders the approach robust with respect to large differences in angle of attack and 

moderate toolmark compression. The performance of the method was evaluated using 3D 

topography scans of experimental toolmarks of 50 unused screwdrivers. Known match and 

known non-match similarity distributions are estimated including a large range of angles of 

attack (15, 30, 45, 60 and 75) for the known matches. It was demonstrated that the system has 

high discriminatory power, even if the toolmarks are made at a difference in angle of attack of 

larger than 15 degrees. The probability distributions were subsequently employed to determine 

likelihood ratios.

Fracture matching

Claytor D., “Validation of Fracture Matching Through the Microscopic Examination of the 

Fractured Surfaces of Hacksaw Blades”, AFTE Journal, Volume 42, Number 4, Fall 2010, pp. 323-

334. 

This study was a validation of a fracture matching method utilizing two consecutivelymanufactured hacksaw blades fractured eleven times and inter-compared. Two hundred fiftythree (253) topical comparisons were conducted between forty-four (44) fractured edges. 

Additional fractured hacksaw blade test specimens were produced and sent to examiners 

around the world yielding three hundred-thirty (330) test results. 

Emerging Research

Bolton-King, R., et al., “What are the Prospects of 3D Profiling Systems Applied to Firearms and 

Toolmark Identification?,” AFTE Journal, Volume 42, Number 1, Winter 2010, pp. 23-33.

This paper details a comparative pilot study of 3D (three dimensional) imaging technologies for 

potential application in forensic firearms and toolmark identification; as such it reviews the 

most up-to-date profiling systems. In particular, the paper focuses on the application of 3D 
imaging and recording technology as applied to firearm identification, being a specialised field 

within the discipline of toolmark identification. Each technology under test employs a different 

technique or scientific principle to capture topographic data i.e. focus-variation microscopy, 

confocal microscopy, point laser profilometry and vertical scanning interferometry. To 

qualitatively establish the capabilities and limitations of each technology investigated, standard 

reference samples were used and a set of specific operational criteria devised for successful 

application in this field. The reference standard crucially included and centred on was the 

National Institute of Standards and Technology (NIST) 'standard bullet'. This was to ensure that 

evaluation represented the practical examination of ballistic samples i.e. fired cartridge cases 

and bullets. It is concluded that focus-variation microscopy has potentially the most promising 

approach for a forensic laboratory instrument, in terms of functionality and 3D imaging 

performance, and is worthy of further investigation.

Chu, W., et al., “Selecting Valid Correlation Areas for Automated Bullet Identification System 

Based on Striation Detection”, Journal of Research of the National Institute of Standards and 

Technology, Volume 116, Number 3, May-June 2011.

This paper detailed a study on fired bullet markings using automated bullet identification 

systems that employ an edge detection algorithm and selection process that locates the edge 

points of significant toolmark features was conducted. Results of this study validated the 

differentiation ability of individual characteristics if a proper striation threshold length could be 

established. 

Gambino, C., et al., “Forensic Surface Metrology: Tool Mark Evidence,” Scanning, Volume 33, 

2011, pp. 272–278.

Over the last several decades, forensic examiners of impression evidence have come under 

scrutiny in the courtroom due to analysis methods that rely heavily on subjective morphological 

comparisons. Currently, there is no universally accepted system that generates numerical data 

to independently corroborate visual comparisons. This research attempted to develop such a 

system for tool mark evidence, proposing a methodology that objectively evaluates the 

association of striated tool marks with the tools that generated them. In this study, 58 primer 

shear marks on 9 mm cartridge cases, fired from four Glock model 19 pistols, were collected 

using high-resolution white light confocal microscopy. The resulting three-dimensional surface 

topographies were filtered to extract all "waviness surfaces"-the essential "line" information 

that firearm and toolmark examiners view under a microscope. Extracted waviness profiles 

were processed with principal component analysis (PCA) for dimension reduction. Support 

vector machines (SVM) were used to make the profile-gun associations, and conformal 

prediction theory (CPT) for establishing confidence levels. At the 95% confidence level, CPT 
coupled with PCA-SVM yielded an empirical error rate of 3.5%. Complementary, bootstrapbased computations for estimated error rates were 0%, indicating that the error rate for the 

algorithmic procedure is likely to remain low on larger data sets. Finally, suggestions were made 

for practical courtroom application of CPT for assigning levels of confidence to SVM 

identifications of tool marks recorded with confocal microscopy.

Song, J., et al., “Development of Ballistics Identification- from Image Comparison to Topography 

Measurement in Surface Metrology”, Measurement Science and Technology, Volume 23, 

Number 054010, March, 2012.

This was a systematic study of direct measurement and correlation of surface topography on 

fired bullet markings. Based on this on this system, a prototype for bullet signature 

measurement and correlation was developed that has demonstrated superior correlation 

results for bullet signature identifications. 

Yammen, S., and Muneesawang, P., “Cartridge Case Image Matching Using Effective Correlation 

Area Based Method,” Forensic Science International, Volume 229, 2013, pp. 27-42.

A firearm leaves a unique impression on fired cartridge cases. The cross-correlation function 

plays an important role in matching the characteristic features on the cartridge case found at 

the crime scene with a specific firearm, for accurate firearm identification. This paper proposed 

that the computational forensic techniques of alignment and effective correlation area-based 

approaches to image matching are essential to firearm identification. Specifically, the reference 

and the corresponding cartridge cases are aligned according to the phase-correlation criterion 

on the transform domain. The informative segments of the breech face marks are identified by 

a cross-covariance coefficient using the coefficient value in a window located locally in the 

image space. The segments are then passed to the measurement of edge density for computing 

effective correlation areas. Experimental results on a new dataset show that the correlation 

system can make use of the best properties of alignment and effective correlation area-based 

approaches, and can attain significant improvement of image-correlation results, compared 

with the traditional image-matching methods for firearm identification, which employ cartridge 

case samples. An analysis of image-alignment score matrices suggests that all translation and 

scaling parameters are estimated correctly, and contribute to the successful extraction of 

effective correlation areas. It was found that the proposed method has a high discriminant 

power, compared with the conventional correlator. This paper advocates that this method will 

enable forensic science to compile a large-scale image database to perform correlation of 

cartridge case bases, in order to identify firearms that involve pairwise alignments and 

comparisons.
Zhang, S. and Chumbley, L.S., “Manipulative Virtual Tools for Tool Mark Characterization”, 

NCJRS Document #241443, Award # 2009-DN-R-119, March 2013.

Research on the development of virtual toolmarks by a 3-D computer simulation that would 

allow for the development of highly predictable toolmark characterizations. The initial study 

involved the production of test toolmarks by six screwdriver tips that were then compared by a 

previously developed statistical algorithm. Preliminary experimental results indicate that the 

use of a manipulative, virtual tool could provide quantitative data for the characterization of 

tool marked surfaces that would improve the scientific basis of toolmark identification.

Grieve, T. et al, “Objective Comparison of Toolmarks from the Cutting Surfaces of Slip-Joint 

Pliers” AFTE Journal, Volume 46, Number 2, Spring 2014, pp. 176-185.

In this paper, experimental results from a statistical analysis algorithm for objectively 

comparing toolmarks via data files obtained using optical profilometry data were described. 

The algorithm employed has successfully been used to compare striated marks produced by 

screwdrivers. In this study, quasi-striated marks produced by the cutting surfaces of slip-joint 

pliers were examined. Marks were made by cutting both copper and lead wire. Data files were 

obtained using an optical profilometer that uses focus variation to determine surface 

roughness. Early efforts using the comparative algorithm yielded inconclusive results when the 

comparison parameters used were the same as those employed successfully for screwdriver 

marks. Further experiments showed that the algorithm could successfully be used to separate 

known matches from non-matches by changing the comparison parameters. Results are 

presented from the analysis of the copper wires.

Riva, F. and Champod C., “Automatic Comparison and Evaluation of Impressions Left by a 

Firearm on Fired Cartridge Cases”, Journal of Forensic Sciences, Volume 59, Number 3, May 

2014, pp. 637-647.

This paper reported on an automated study of marks contained on fired cartridge cases from 

seventy-nine (79) 9mm Luger caliber pistols were conducted using 3D surface topography 

analysis and coupled to a bivariate evaluative model to assign likelihood ratios. The purpose of 

this analytic system was to conduct an objective comparative analysis with a robust statistical 

evaluation basis to the results. The system reflected a very high discriminating ability between 

the known and non-known specimens. This study also reflected very low rates of misleading 

evidence depending on the firearm considered. 

McClarin, D., “Adding an Objective Component to Routine Casework: Use of Confocal 

Microscopy for the Analysis of 9mm Caliber Bullets”, AFTE Journal, Volume 47, Number 3, 

Summer 2015, pp. 161-170.
The Alabama Department of Forensic Sciences (ADFS) procured a confocal microscope for the 

purpose of incorporating three-dimensional (3D) topographical analysis into routine casework. 

The purpose of employing such a technique was to assist the firearm and toolmark examiner by 

complementing routine analysis with an independent objective analysis. This article covered 

the research procedures conducted using confocal microscopy at the ADFS.

Weller, T., et al., “Introduction and Initial Evaluation of a Novel Three-Dimensional Imaging and 

Analysis System for Firearm Forensics” AFTE Journal, Volume 47, Number 4, Fall 2015, pp. 198-

208.

This paper presents a set of matching experiments conducted using a novel 3D imaging and 

analysis system for cartridge cases, TopMatch. The system utilizes the GelSight photometric 

stereo sensor to measure micron scale surface geometry and a novel feature-based matching 

algorithm to score the geometric similarity between measured surfaces. The matching 

algorithm separately considers the impressed breech face impression and the striated aperture 

shear and then combines their similarity into a single confidence score. The system 

demonstrates excellent recall rates with no false positives across a set of experiments involving 

290 firearms and 700 cartridge cases from 24 firearms manufacturers. This was the first 

publication describing this new technology and the first round of matching results. 

Improvements to the imaging and matching algorithms are already underway.

Spotts, R., et al., “Angular Determination of Toolmarks Using a Computer-Generated Virtual 

Tool” Journal of Forensic Sciences, Volume 60, Number 4, July 2015, pp. 878-884.

A blind study was conducted to determine whether virtual toolmarks created using a computer 

could be used to identify and characterize angle of incidence of physical toolmarks. Six 

sequentially manufactured screwdriver tips and one random screwdriver were used to create 

toolmarks at various angles. An apparatus controlled the tool angle. Resultant toolmarks were 

randomly coded and sent to the researchers, who scanned both tips and toolmarks using an 

optical profilometer to obtain 3D topography data. Developed software was used to create 

virtual marks based on the tool topography data. Virtual marks generated at angles from 30 to 

85° (5° increments) were compared to physical toolmarks using a statistical algorithm. Twenty 

of twenty toolmarks were correctly identified by the algorithm. On average, the algorithm 

misidentified the correct angle of incidence by −6.12°. This study presents the results, their 

significance, and offers reasons for the average angular misidentification.

Hamby, J., et al., “Evaluation of GLOCK 9mm Firing Pin Aperture Shear Mark Individuality Based 

On 1,632 Different Pistols by Traditional Pattern Matching and IBIS Pattern Recognition” Journal 

of Forensic Sciences, DOI:10.1111/1556-4029.12940, (In Press) 2015.
Over a period of 21 years, a number of fired GLOCK cartridge cases have been evaluated. A 

total of 1,632 GLOCK firearms were used to generate a sample of the same size. Our research 

hypothesis was that no cartridge cases fired from different 9-mm semiautomatic GLOCK pistols 

would be mistaken as coming from the same gun. Using optical comparison microscopy, two 

separate experiments were carried out to test this hypothesis. A subsample of 617 test-fired 

cases were subjected to algorithmic comparison by the Integrated Ballistics Identification 

System (IBIS). The second experiment subjected the full set of 1632 cases to manual 

comparisons using traditional pattern matching. None of the cartridge cases were "matched" by 

either of these two experiments. Using these empirical findings, an established Bayesian 

probability model was used to estimate the chance that a 9-mm cartridge case, fired from a 

GLOCK, could be mistaken as coming from the same firearm when in fact it did not (i.e., the 

random match probability).

Spotts, R. and Chumbley, L. S. (2015), Objective Analysis of Impressed Chisel Toolmarks. J 

Forensic Sci, 60: 1436–1440. doi:10.1111/1556-4029.12863

Historical and recent challenges to the practice of comparative forensic examination have 

created a driving force for the formation of objective methods for toolmark identification. In 

this study, fifty sequentially manufactured chisels were used to create impression toolmarks in 

lead (500 toolmarks total). An algorithm previously used to statistically separate known 

matching and nonmatching striated screwdriver marks and quasi-striated plier marks was used 

to evaluate the chisel marks. Impression toolmarks, a more complex form of toolmark, pose a 

more difficult test for the algorithm that was originally designed for striated toolmarks. Results 

show in this instance that the algorithm can separate matching and nonmatching impression 

marks, providing further validation of the assumption that toolmarks are identifiably unique.

Reviews

Gerules, G. , et al., “A survey of image processing techniques and statistics for ballistic 

specimens in forensic science,” Science and Justice, Volume 53,2013, 236–250

This paper provides a review of recent investigations on the image processing techniques used 

to match spent bullets and cartridge cases. It is also, to a lesser extent, a review of the 

statistical methods that are used to judge the uniqueness of fired bullets and spent cartridge 

cases. The authors reviewed 2D and 3D imaging techniques as well as many of the algorithms 

used to match these images. They also provided a discussion of the strengths and weaknesses 

of these methods for both image matching and statistical uniqueness. The goal of this paper 

was to be a reference for investigators and scientists working in this field.
Vorburger, T.V. , J. Song, and N. Petraco, “Topography measurements and applications in 

ballistics and tool mark identifications” Surface Topography: Metrology and Properties, Vol. 4, 

No. 1, 2015. 

The application of surface topography measurement methods to the field of firearm and 

toolmark analysis is fairly new. The field has been boosted by the development of a number of 

competing optical methods, which has improved the speed and accuracy of surface topography 

acquisitions. The authors describe some of these measurement methods as well as several 

analytical methods for assessing similarities and differences among pairs of surfaces. They also 

provide a few examples of research results to identify cartridge cases originating from the same 

firearm or tool marks produced by the same tool. Physical standards and issues of traceability 

are also discussed.

Aitken, C., et al. Communicating and Interpreting Statistical Evidence in the Administration of 

Criminal Justice, Part 1., Fundamentals of Probability and Statistical Evidence in Criminal 

Proceedings, Royal Statistical Society, 2010, 121p.

Statistical evidence and probabilistic reasoning today play an important and expanding role in 

criminal investigations, prosecutions and trials, not least in relation to forensic evidence 

(including DNA) produced by expert witnesses. Guide No 1 was designed as a general 

introduction to the role of probability and statistics in criminal proceedings, a kind of vade 

mecum for the perplexed forensic traveller; or possibly, ‘Everything you ever wanted to know 

about probability in criminal litigation but were too afraid to ask’. It explains basic terminology 

and concepts, illustrates various forensic applications of probability, and draws attention to 

common reasoning errors (‘traps for the unwary’).

Question 2: Have studies been conducted to establish baseline frequencies of characteristics 

or features used in these pattern-based matching techniques? If not, how might such studies 

be conducted? What publicly accessible databases exist that could support such studies? 

What closed databases exist? Where such databases exist, how are they controlled and 

curated? If studies have not been conducted, what conclusions can and cannot be stated 

about the relationship between the crime scene evidence and a known suspect or tool (e.g., 

firearm)?

Creating baseline frequency studies is a difficult proposition in the field of Firearms and 

Toolmarks Examination due to the dynamic nature this type of evidence presents. Given there 

can be no degree of control over the absence or presence of affected surface areas that may 
contain baseline marks makes the use of a standard frequency database difficult. However, in 

recent years research has been and continues to be conducted using computer technology to 

begin formulating criteria and to assist in creating objective, measurable standards for 

identification within the field. 

There are published papers and books examining the relative frequency of toolmark evidence 

(Question # 2 References, #’s 1-11). These studies concluded the chance of a coincidental 

match to be low, and that a high degree of similarity between two toolmarks provides a strong 

basis for a conclusion of common origin. These studies remain theoretical in nature and are not 

applied to toolmark casework in the forensic laboratory. There are a large number of random 

and changing factors in tool (and firearm) manufacturing. Therefore the goal of producing a 

statistical model or mathematical equation that can accurately predict toolmark variance

remains elusive. The marks used by toolmark examiners are random in nature, and thus 

establishing a probability model requires an empirical statistical approach. This is a stark 

contrast to DNA that uses a generative model (the Hardy-Weinberg equation). Despite these 

difficulties, scientists continue to research the concepts of frequency, probability, likelihood 

ratios and automated comparisons in field of toolmark identification (see Question 2 

References, #’s 12-28).

NIST in collaboration with the FBI and crime labs across the U.S. is currently compiling a 

database of known test fired bullets and cartridge cases, and will be the curator of this set of 

reference samples. The purpose of the database, as outlined by NIST at 

http://www.nist.gov/forensics/ballisticsdb/, is to foster the development and validation of 

measurement methods, algorithms, metrics, and quantitative confidence limits for objective 

firearm identification. Furthermore, the database is intended to improve the scientific 

knowledge base on the similarity of marks from different firearms and the variability of marks 

from the same firearm, and ease the transition to the application of three-dimensional surface 

topography data in firearms identification. This database will serve as a useful set upon which 

different search and analysis software can be compared.

Additionally, the lack of frequency data or the ability to express an opinion as a likelihood ratio 

does not automatically lessen a scientific conclusion. Many of humankind’s greatest scientific 

discoveries did not enjoy the benefit of a probability distribution but rather utilized detailed 

observations from clearly reasoned experimental design. It has historically been, and remains, 

a primary goal of the firearm and toolmark profession to support practitioners’ conclusions 

with objective or statistical criteria. However, the fact that work remains does not make the 

current state of toolmark comparison bad science. 
Question 3: How is performance testing (testing designed to determine the frequency with 

which individual examiners obtain correct answers) currently used in forensic laboratories? 

Are performance tests conducted in a blind manner? How could well-designed performance 

testing be used more systematically for the above pattern-based techniques to establish 

baseline error rates for individual examiners? What are the opportunities and challenges for 

developing and employing blind performance testing? What studies have been published in 

this area?

In firearms and toolmark identification, performance testing (as defined above) is determined 

by a series of different tests and experiments. First, the overall reliability of a trained examiner 

to correctly differentiate and associate items based on the comparison of microscopic 

toolmarks has been demonstrated through nearly a century of empirical research, validation 

tests, and proficiency test data. Furthermore, over the past decade, research using 3D 

topographical data and comparison algorithms provides strong, statistical support for the 

firearm and toolmark examiners experiential knowledge. 

Within the laboratory, firearm and toolmark examiner training is often the most rigorous and 

time-intensive of all the forensic disciplines. A typical trainee will train for at least two years 

prior to performing any casework. Once the trainee has completed their training, they will be 

presented a series of competency tests. Following successful completion of these tests, they 

will advance onto performing monitored/supervised casework, after which they advance to 

journeyman level status and are qualified to perform full casework. Typically, post training, 

examiners are required to complete (at least) one proficiency test a year in each discipline they 

are qualified. Data has been collected from published results from commercial proficiency tests 

providers. This data has been used to evaluate potential error rates within the field. However, 

this data must be used with caution as the commercial providers do not control for the level of 

training or prior competency before issuing a test and recording the results. The evaluation of 

an individual examiner’s performance on proficiency tests is often monitored by a laboratory

quality assurance manager (Question #3 References, [1]). 

The proficiency test is generally not blind; however, the correct answer is not known by the 

examiner. In order to combat some of the challenges in providing a truly blind test, some 

laboratory systems do periodically test examiners blindly through a re-examination process or a 

blind verification process. However, these practices anticipate a consensus opinion, and the 

answer is not one grounded in truth like the current proficiency test method. It would be 

extremely difficult to produce a truly blind test in a forensic laboratory. These considerations 

and complications of implementing blind forensic proficiency tests are well outlined in the 

articles by Peterson, et al (Question #3 References, [2,3]). To highlight some of the difficulties: 

The test provider(s) would have to produce fake reports, evidence, packaging, and all other 
documentation in order to make the evidence appear “real”. Additionally, with many 

laboratory systems carrying case work backlogs, in order to not bias the examiner and treat the 

test blindly, it would have to be subject to the same timeliness criteria as other cases. This task 

alone is herculean given the patchwork nature of United States Forensic Laboratories. 

Furthermore, law enforcement investigators would have to submit requests to examine this 

evidence, and then the laboratory would have to ensure each examiner is provided a test, but 

do so in a “blind” manner. It is our opinion that this is not a practical use of laboratory 

resources (both cost and manpower). We are only aware of the studies referenced above 

(Peterson et al). 

Question 4: What are the most promising new scientific techniques that are currently under 

development or could be developed in the next decade that would be most useful for 

forensic applications? Examples could include hair analysis by mass spectrometry, advances 

in digital forensics, and phenotypic DNA profiling.

The Firearms/Toolmarks subcommittee of the Organization of Scientific Area Committees 

(OSAC) has established a Task Group to study and evaluate the research and development of 

instruments and software that can accurately measure and compare microscopic toolmarks and 

provide statistical weight to the comparison. This technology has the potential to provide 

greater objectivity and statistically-supported conclusions to the science of firearms and 

toolmark comparison. 

Question 5: What standards of validity and reliability should new forensic methods be 

required to meet before they are introduced in court?

In anticipation of the role that technology will play in the near future for Firearm and Toolmark 

Examination, the Firearms/Toolmarks subcommittee of the Organization of Scientific Area 

Committees (OSAC) is in the process of writing and publishing validation standards for the 

implementation of new technology in the firearms and toolmark laboratory. 

Question 6: Are there scientific and technology disciplines other than the traditional forensic 

science disciplines that could usefully contribute to and/or enhance the scientific, technical 

and/or societal aspects of forensic science? What mechanisms could be employed to 

encourage further collaboration between these disciplines and the forensic science 

community?
The Organization of Scientific Area Committees, established by the National Institute of 

Standards and Technology (NIST) has as a primary goal to answer this very question. The 

majority of forensic science disciplines have now been brought together within one entity with 

a purpose of establishing scientifically sound standards of practice within each discipline. The 

ability to share knowledge and research and to collaborate between like disciplines is now a 

greater possibility which will only serve to enhance the technical and societal impacts of 

forensic science. 

Specifically within the discipline of firearms and toolmarks comparison, our profession has 

begun collaboration with computer scientists utilizing machine learning algorithms. Machine 

learning is a subdiscipline of computer science that utilizes probability and statistics to develop 

algorithms for pattern recognition. Since the comparison of toolmarks is the comparison of 

patterns, the collaboration between firearms and toolmark examiners and computer scientists 

is a collaboration that has started to produce interesting research papers (examples cited in 

Question #6 References [1-10]). 

Metrology is a second discipline that has enhanced the science of firearm and toolmark 

identification. Metrology is the science of measurement. In order to use computer pattern 

recognition algorithms to compare toolmarks, the toolmarks must be accurately measured. 

This is where the metrology scientists have (and will) help the forensic community evaluate and 

implement the best technology for the task at hand (examples cited in Question #6 References 

[11-18]).

We believe the firearms and toolmark examiner community, in collaboration with the 

disciplines above, has a good understanding of the problems and potential solutions facing our 

profession. The problem is finding time and funding to conduct the necessary research. The 

vast majority of the research published in forensic science journals is based on volunteered 

time and conducted by a few dedicated individuals. This country would be wise to implement a 

broader source of forensic science research funding (e.g. NSF and NIH). One way to accomplish 

this task would be to increase the research funding already provided by the NIJ. If the goal is to 

have forensic science research move forward faster, forensic science research needs to be a 

viable full time career option. 

Question #2 References:

[1] Collins, E., et. al. “How Unique Are Impressed Toolmarks? An Empirical Study of 20 Worn 

Hammer Faces”; AFTE Journal, 2005, Vol 37(4), pp.252-295. 

[2] Stone, R. “How Unique Are Impressed Toolmarks?” AFTE journal, 2003, Vol 35(4), pp.376-

383.
[3] Neel, M., et. al. “A Comprehensive Statistical Analysis of Striated Tool Mark Examinations 

Part 1: Comparing Known Matches and Known Non-Matches”; AFTE Journal, 2007, Vol 39(3), 

pp.176-198.

[4] Wevers G., et al. “A Comprehensive Statistical Analysis of Striated Tool Mark Examinations; 

Part 2: Comparing Known Matches and Known Non-Matches using Likelihood Ratios”. AFTE 

Journal, 2011, Vol 43(2), pp.137-145. 

[5] Uchiyama, T. “The Probability of Corresponding Striae in Toolmarks”;AFTE Journal, Vol 24(3), 

pp.273-290.

[6] Howitt, D. “A Calculation of the Theoretical Significance of Matched Bullets”; Journal of 

Forensic Sciences, 2008, Vol 53(4), pp.868-875. 

[7] Biasotti, A. A. “A Statistical Study of the Individual Characteristics of Fired Bullets”; Journal of 

Forensic Sciences, 1959, Vol 4(1), pp.34-50. 

[8] Deinet, W., "Studies of Models of Striated Marks Generated by Random Processes", Journal 

of Forensic Sciences, 1981, vol. 26(1), pp.35-50.

[9] Brackett, J. W. "A Study of Idealized Striated Marks and their Comparisons using Models", 

Journal of the Forensic Science Society, vol. 10 (1), January, 1970, pp. 27-56.

[10] Hatcher, J.S., et al., "Firearm Investigation Identification and Evidence", The Stackpole 

Company, 1957 P. 389 P.380.

[11] Heard, B. J., "Handbook of Firearms and Ballistics", Wiley & Sons, 1997, pp. 136-141

[12] Bunch, S., et al. “Application of likelihood ratios for firearm and toolmark analysis”; Sci 

Justice, 2013, 53(2) pp.223-9.

[13] Riva, F., et al. “Automatic comparison and evaluation of impressions left by a firearm on 

fired cartridge cases” J Forensic Sci, 59(3)637-47.

[14] Weller, T., et al. “Introduction and Initial Evaluation of a Novel Three-Dimensional Imaging 

and Analysis System for Firearm Forensics” AFTE Journal, 2015, 47(4) 198-208.

[15] Chu W., et al. "Validation Tests for the Congruent Matching Cells (CMC) Method Using 

Cartridge Cases Fired with Consecutively Manufactured Pistol Slides", AFTE Journal, 2013, 45(4), 

361-366.

[16] Petraco, D. K., et al., "Addressing the National Academy of Sciences Challenge: A Method 

for Statistical Pattern Comparison of Striated Tool Marks", J. Forensic Sci, 2012, 57(4), 900-911.

[17] Petraco D. K., et al., "Application of Machine Learning to Toolmarks: Statistically Based 

Methods for Impression Pattern Comparisons", NIJ/NCJRS Document #239048, Award #2009-

DN-BX-K041, July 2012.

[18] Bachrach, B. “A Statistical Validation of the Individuality and Repeatability of Striated Tool 

Marks: Screwdrivers and Tongue and Groove Pliers” J. Forensic Sci 2010 55(2) pp.348-357.

[19] Bachrach, B. "Development of a 3D-Based Automated Firearms Evidence Comparison 

System", Journal of Forensic Sciences, Vol. 47(6), November 2002, pp. 1253-1264.
[20] Chu, W., et al., "Automatic Identification of Bullet Signatures Based on Consecutive 

Matching Striae (CMS) Criteria", Forensic Science International, 231, 2013, Pp. 137-141

[21] Banno, A., et al. “Three Dimensional Visualization and Comparison of Impressions on Fired 

Bullets,” Forensic Science International, Vol.140, 2004, pp. 233–240.

[22] Bolton-King, R., et al., “What are the Prospects of 3D Profiling Systems Applied to Firearms 

and Toolmark Identification,” AFTE Journal, Vol. 42, No. 1, 2010, pp. 23–33

[23] Gambino, C., et al., “Forensic Surface Metrology: Tool Mark Evidence,” Scanning, Vol. 33, 

2011, pp. 272–278.

[24] Petraco, N., et al. “Estimates of Striation Pattern Identification Error Rates by Algorithmic 

Methods,” AFTE Journal, Vol. 45, No. 3, 2013, pp. 235–244.

[25] Yammen, S., and Muneesawang, P., “Cartridge Case Image Matching Using Effective 

Correlation Area Based Method,” Forensic Science International, Vol. 229, 2013, pp. 27–42.

[26] Zheng, A., Soons, J., Thompson, R., Villanova, J., and Kakal, T., “2D and 3D Topography 

Comparisons of Toolmarks Produced from Consecutively Manufactured Chisels and Punches,” 

AFTE Journal, Vol. 46, No 2, 2014, pp.143–147.

[27] Spotts R., et al. “Angular Determination of Toolmarks Using a Computer-Generated Virtual 

Tool” J. Forensic Sci 2015 60(4) pp. 878-884.

[28] Hamby, J., et al. “Evaluation of GLOCK 9 mm Firing Pin Aperture Shear Mark Individuality 

Based On 1,632 Different Pistols by Traditional Pattern Matching and IBIS Pattern Recognition” 

J. Forensic Sci, doi:10.1111/1556-4029.12940.

Question #3 References

[1] Murphy, D., “CTS Error Rates, 1992-2005 Firearms/Toolmarks”, Presented at the 41st 

Association of Firearm and Tool Mark Examiners (AFTE) Training Seminar, Henderson, NV, May 

5, 2010. 

[2] Petersen, J.L., et al. “The feasibility of external blind DNA proficiency testing. I. Background 

and findings” J. Forensic Sci, 2003, 48(1) pp. 21-31.

[3] Petersen, J.L., et al. “The feasibility of external blind DNA proficiency testing. II. Experience 

with actual blind tests.” J. Forensic Sci, 2003, 48(1) pp. 32-40.

Question #6 References

[1] Grieve, T. “Objective Comparisorements of NIST SRM 2460 standard bullets by four 

techniques.” Meas Sci Technol 2006;17: pp. 500–3.

[17] Ma, L., et al, “NIST Bullet Signature Measurement System for RM (Reference Material) 

8240 Standard Bullets” J. Forensic Sci 2014 49(4): pp. 649-59 .

[18] Song, J., et al. “Three Steps towards Metrological Traceability for Ballistics Signature 

Measurements” Meas. Sci. Review 2009 10(1) pp. 19-21.n of Toolmarks from the Cutting 

Surfaces of Slip-Joint Pliers” AFTE Journal 2014 46(2) pp. 176-185.
[2] Riva, F., et al. “Automatic comparison and evaluation of impressions left by a firearm on 

fired cartridge cases” J Forensic Sci, 59(3) pp.637-47.

[3] Roth, J., et al. “Learning-based Ballistic Breech Face Impression Image Matching” 

Conference: the IEEE Seventh International Conference on Biometrics: Theory, Applications and 

Systems (BTAS 2015), At Arlington, Virginia

[4] Weller, T., et al. “Introduction and Initial Evaluation of a Novel Three-Dimensional Imaging 

and Analysis System for Firearm Forensics” AFTE Journal, 2015, 47(4) pp. 198-208.

[5] Petraco, D. K., et al., "Addressing the National Academy of Sciences Challenge: A Method for 

Statistical Pattern Comparison of Striated Tool Marks", J. Forensic Sci, 2012, 57(4), pp. 900-911.

[6] Petraco, N., et al. “Estimates of Striation Pattern Identification Error Rates by Algorithmic 

Methods,” AFTE Journal, Vol. 45, No. 3, 2013, pp. 235–244.

[7] Chu, W., et al., "Automatic Identification of Bullet Signatures Based on Consecutive 

Matching Striae (CMS) Criteria", Forensic Science International, 231, 2013, Pp. 137-141

[8] Song, J. “Proposed "NIST Ballistics Identification System (NBIS)" Based on 3D Topography 

Measurements on Correlation Cells” AFTE Journal 2013 45(2) pp. 184-194

[9] Chu, W., et al. "Validation Tests for the Congruent Matching Cells (CMC) Method Using 

Cartridge Cases Fired with Consecutively Manufactured Pistol Slides", AFTE Journal, 2013, 45(4), 

pp. 361-366.

[10] Spotts, R., et al. “Angular Determination of Toolmarks Using a Computer-Generated Virtual 

Tool” J. Forensic Sci 2015 60(4) pp.878-884.

[11] Zheng, X., et al. “Applications of surface metrology in firearm identification” Surf. Topogr.: 

Metrol. Prop. 2014(2) 10pp.

[12] Weller, T., et al. “Confocal Microscopy Analysis of Breech Face Marks on Fired Cartridge 

Cases from 10 Consecutively Manufactured Pistol Slides” J. Forensic Sci 2012 57(4)pp. 912-917.

[13] Gambino, C., et al., “Forensic Surface Metrology: Tool Mark Evidence,” Scanning, Vol. 33, 

2011, pp. 272–278.

[14] Song, J., et al. “Topography Measurements and Performance Comparisons between NIST 

SRM 2460 Standard Bullet Masters and BKA Bullet Replicas” AFTE Journal 2012 44(3) pp. 208-

217.

[15] Vorburger, T.V.,et al. “Surface topography analysis for a feasibility assessment of a national 

ballistics imaging database. A report prepared for the National Academies Committee to assess 

the feasibility, accuracy, and technical capability of a National Ballistics Database.” NISTIR 7362 

2007; pp. 1–171.

[16 ] Song, J., et al. “Correlation of topography measurements of NIST SRM 2460 standard 

bullets by four techniques.” Meas Sci Technol 2006;17:500–3.

[17] Ma, L., et al. “NIST Bullet Signature Measurement System for RM (Reference Material) 

8240 Standard Bullets” J. Forensic Sci 2014 49(4):649-59 .
[18] Song, J., et al. “Three Steps towards Metrological Traceability for Ballistics Signature 

Measurements” Meas. Sci. Review 2009 10(1)19-21.

Sincerely,

Andy Smith

Chair

OSAC Firearm/Toolmark Subcommittee