1

Abstracts

Measurement Science & Standards in

Forensic Firearms Analysis

July 10-11, 2012

National Institute of Standards and Technology (NIST), 100 Bureau Drive, Gaithersburg, MD 20899

Room: Building 101 – Portrait Room

Current State of Firearms Analysis: Review of the Field Including 

Strengths & Limitations

Current Technology Used in the Laboratory

William E. Demuth II

Illinois State Police

The purpose of this presentation is to provide the non-Firearms Examiner an introduction to the 

instrumentation typically found in the modern Firearms Unit. This will consist of an explanation 

of the types of measurements performed, how the data is collected, and what conclusions may be 

drawn from these measurements. 

To accomplish this, a brief description and history of the two main types of microscopes 

(stereomicroscope and comparison microscope) will be presented along with a discussion of the 

various measuring devices used in the section. Particular emphasis will be placed on the 

combinations of instrumentation used to measure the land and groove impressions found on fired 

bullets. The presentation will conclude with an explanation of firearms barrel length and overall 

length measurements along with a brief description of the application of measurement 

uncertainty to the collection and reporting of this data.
2

Human Performance Issues in Firearms/Toolmarks

Douglas Murphy

FBI Laboratory

This presentation will explore the decision process employed by examiners in Forensic Firearms 

Analysis, with emphasis on the more subjective elements. Also included will be error rate 

information from various sources, and a look at how future technology may improve the 

decision process.

Introduction to Measurement Science Advances in Firearms 

Analysis

Optical Methods of Surface Measurement

Theodore Vorburger

National Institute of Standards and Technology (NIST)

Ballistics examinations rely heavily on optical methods, but there is a confusing number of 

optical methods for measuring surface topography that could be applied to examinations of fired 

cartridge cases and bullets. We will try to describe work in this field from two points of view: 

the active documentary standardization of optical surface measurement methods and the 

applications of these methods to firearms examination. Four microscopy methods will be 

described: confocal, focus variation, coherence scanning interferometry, and chromatic confocal. 

Measurement Science Advances in Firearms Analysis

Confocal Microscopy of Bullet Surfaces

David Howitt

University of California - Davis

The application of confocal microscopy to the determination of the distribution of the striations 

on the surface of a bullet are complicated by the curvature of the surface and the shallow depth 

of the surface contours. The consequences to the determination of an accurate three-dimensional 

reconstruction of the surface are very long acquisition times and a great deal of information at a 

resolution far beyond what is normally distinguishable in a conventional bridge microscope. 

These combine to render the technique cumbersome and far from straightforward to interpret in 

terms of the optical features that are typically employed by firearms and toolmark examiners. 

Another drawback to the technique is that the algorithms that perform the reconstruction appear 

to have been empirically refined to optimize the analysis of flat surfaces and this coupled with 
3

the same bias in the analytical software makes the application of confocal microscopy to the 

development of a functional database far from straightforward. 

Focus Variation

Brad Etter

Alicona Corporation

In this presentation an optical 3D surface measurement technology (Focus Variation) is 

presented. This technique combines the small depth of focus of an optical system with vertical 

scanning to provide topographical and color information from the variation of focus. Novel and 

unique algorithms reconstruct this into a single 3D data set with accurate topographical 

information. “Z” resolution can be as low as 10nm making the instrument ideal for surface study 

of both homogeneous and compound materials. Examples of real world firearm measurements 

will also be presented.

Surface Topography Measurement using GelSight Elastomeric Sensor

Marcus Brubaker

Cadre Research Labs

This talk will present the new GelSight sensor technology which provides rapid surface 

topography measurements down to the micron scale. The technology uses an elastomeric or gelbased sensor with a coating of known reflectance which enables detailed measurement of surface 

topography independent of the sample’s material. The system is substantially less expensive and 

smaller than existing alternatives, making both desktop and portable devices feasible. This talk 

will describe the GelSight sensor technology and demonstrate its capabilities. The specific 

application of matching breech face impressions using a novel feature-based surface matching 

approach will be presented, including the preliminary results of a small-scale study.

Application of Thermal Infrared Imaging to Ballistic Toolmark Identification

Francine Prokoski, PhD

Infrared Identification Incorporated (IRID)

Every object continuously radiates thermal energy unless its temperature is at absolute zero. The 

amount it radiates depends on its temperature and aspects of its surface including: shape, 

material composition, roughness, texture, color, surface treatments and coatings. Those factors 

combine to determine the emissivity of the object's surface, which is a measure of how 

efficiently it radiates heat. Non-contact infrared imagers sensitive at thermal wavelengths focus 

and record surface emissions to produce apparent temperature images that contain evidence of 

emissivity variations. In most common applications for infrared imaging, the intent is to obtain 

true temperatures, and emissivity variations must be calibrated out of the sensor data. When the 

intent is ballistic toolmark identification, temperature variations are removed to produce a 

surface map of emissivity variations that comprise the toolmarks.
4

Their ability to reliably image surface emissivity variations makes thermal infrared imagers 

particularly well-suited for toolmark detection and replication. Other important aspects of 

thermal IR imagers for toolmark identification include: 

 They do not require, and are insensitive to, visible light 

 They do not require any incident illumination or activation

 They are free from illumination-induced artifacts such as glare, shadow, and speckle. 

 There is no need to adjust illumination angle to image toolmarks on a specific object

 This removes variations associated with lighting adjustments by different human 

observers based on their personal visual acuity, training, and habits 

 Collection and comparison of emissivity maps can be performed reliably, accurately, 

and quickly by fully automated systems 

 This facilitates large collection efforts to support statistical analyses.

The science and engineering behind thermal infrared imaging is well-established and a vast body 

of textbooks and peer-reviewed publications provide a rigorous scientific basis that can be 

applied to toolmark identification with the goal being to improve measurement validity, 

accuracy, and reliability. R&D efforts have been designed to test the following hypotheses: 

 First, that surface texture variations associated with firearm-induced toolmarks on 

cartridge cases and bullets, when imaged using thermal infrared cameras, provide 

a proper basis for toolmark identification. 

 Second, that the reliability of emissivity mapping provides for meaningful 

quantitative correlations between surfaces being compared, with analyses of 

statistical distributions supporting likelihood estimates that surface toolmarks 

have a common origin. 

 Third, that emissivity maps, being valid representations of surface topography 

features, can be meaningfully compared against other representations such as 

two- and three-dimensional visible light images. Specifically, that emissivity 

maps can be accurately compared against legacy NIBIN images. 

The National Academy of Sciences (NAS) 2009 report, “Strengthening Forensic Science in 

the United States”, outlines the need to improve the scientific foundations of the forensic 

disciplines, particularly those dependent on qualitative analyses and expert interpretation of 

observed patterns. Emissivity maps from thermal infrared imagers have been shown to 

provide reliable and accurate identification of cartridge casings and bullets, resulting in 

significantly reducing both false positive and false negative candidate selections presented 

for expert examination. A plan to achieve evidentiary acceptance needs to be developed,

detailing additional testing, peer review, dissemination of results, replication by third 

parties, formulation and testing of a scientific model, and preparations for a 

Daubert hearing.

KEYWORDS: toolmark, bullet, cartridge case, identification, infrared metrology, thermal infrared
5

Advances in Image and Statistical Analysis in Firearms Analysis

Comparison and Interpretation of Impressed Marks Left by a Firearm on 

Cartridge Cases

Christophe Champod

University of Lausanne, Switzerland

Recent years have been characterized by a series of publications in the field of firearms 

investigation questioning the reliability and objectivity of such examination. The main claim is 

that the field is dominated by a subjective and poorly articulated interpretation stage. This 

research investigates new solutions to decrease the subjective compound affecting the 

interpretation that follows the comparison of marks left by a firearm on the surface of spent 

cartridge cases. To do that, an automatic comparison system based on 3D measurements has 

been developed and coupled to a bivariate interpretative model allowing likelihood

ratio assignment.

Keywords: Firearms identification, likelihood ratio, cartridge cases, 3D topographies, 

interpretation.

Consecutively Matching Stria: The Development by Empirical Testing of 

Numerical Criteria for the Identification of Striated Toolmarks

John Murdock and Al Biasotti

The conclusions drawn from this training and research were published in 1997 in a chapter 

entitled “Firearms and Toolmark Identification” by Biasotti and Murdock which appeared in the 

two volume set “Modern Scientific Evidence – The Law and Science of Expert Testimony”. The 

author’s conservative quantitative criteria for identification are:

 In three dimensional toolmarks when at least two different groups of at least three 

consecutive matching striae appear in the same relative position, or one group of six 

consecutive matching striae are in agreement in an evidence toolmark compared to a test 

toolmark.

 In two dimensional toolmarks when at least two groups of at least five consecutive 

matching striae appear in the same relative position, or one group of eight consecutive 

matching striae are in agreement in an evidence toolmark compared to a test toolmark. 

For either of these criteria to apply, however, the possibility of subclass characteristics must be 

ruled out. These same conclusions appear, unchanged, in Chapter 35, now authored by Biasotti, 

Murdock and Moran, in the now five volume set of Modern Scientific Evidence (MSE) 

published 2009-2010.

The research that led up to what co-authors Biasotti and Murdock called conservative 

quantitative criteria for the identification of striated toolmarks (QCMS) was described only in a 
6

general way in the 1997 MSE chapter. The purpose of this paper is to describe more specifically 

the training and research that led up to the formation of the QCMS criteria. Regrettably, this 

should have been done before publishing the conclusions, but the opportunity to help prepare the 

American Jurisprudence to shoulder their gatekeeping role under Daubert, by co-authoring the 

chapter in MSE, was one that Biasotti and Murdock could not pass up. 

3D Surface to 2D Barcode

David Howitt

University of California - Davis

This concept for a bullet database is a straightforward extension of the concept of consecutive 

matching striae (CMS) that has long been promoted in California since the work of Biasotti in 

the 1950s when he established that the proportion of matching striae found in the one to one 

comparison of the land impressions of bullets could not be used as a determination of whether 

the bullets were fired from the same gun. Although Biasotti was never able to determine the 

cause of this counterintuitive finding we now know that it is related to the variation in the total 

number of striae and the presence of subclass features on the bullet surfaces. 

The ramifications of this to the determination of the validity of the various approaches to the 

construction of databases for firearms identification are not all intuitively obvious and these will 

be discussed along with the aspects of 3D surface to 2D barcode conversion that pertain 

to them.

Bullet Signature Identification Using Topography Measurements and 

Correlations; the Unification of Microscopic and Mathematical Comparisons 

Robert M. Thompson, BS, Wei Chu, PhD, John Song, MS, Theodore Vorburger, PhD, Alan Zheng, 

MS, T. Brian Renegar, MS, Richard Silver, PhD

A 2D and 3D Topography Measurement and Correlation System was developed at NIST for 

certification of NIST Standard Reference Material (SRM) 2460/2461 Bullets and Cartridge 

Cases. Based on this system, a prototype system for signature measurement and correlation of 

fired bullets has been recently developed at NIST for bullet identifications. The 3D topography 

data of the land engraved areas (LEAs) of fired bullets are captured by a commercial confocal 

microscope. The LEAs were processed by the “edge detection” method to determine the 

“striation density”, by which the surface area with low striation density on the LEA could be 

masked out from correlation. The modified 3D micro-topography data on the remaining “valid 

correlation areas” are compressed into a 2D profile which represents the 2D ballistics signature 

of the LEA. A correlation program using two methods has been developed for matching the 

paired profile signatures: the Consecutive Matching Striae (CMS) method used by many firearm 

examiners and the Cross Correlation Function maximum (CCFmax) method developed by NIST 

based on analysis methods in surface metrology.

In July 2010, a set of 20 known-matching bullets fired from 10 consecutively manufactured 

barrels (two bullets from each barrel) were tested. Their 3D topography images were captured 

by the confocal microscope at NIST, and correlated by the prototype ballistics identification 
7

system using the CCFmax as a correlation indicator. The correlation result was excellent: 

correlation values of all ten pairs of known-matching bullets scored highest on all correlation 

lists, yielding a correct identification rate of 100%. For the 60 pairs of matched LEAs (each 

bullet includes six LEAs), correlation values of matching LEAs scored highest on 59 out of 60 

correlation lists, yielding a correct identification rate for individual LEAs of 98.3%.

In August 2010, an additional set of 15 unknown matching bullets fired from the same set of 10 

barrels was blind tested. These bullets were correlated with the 20 known-matching bullets 

mentioned above, making the total number of correct matching pairs equal to 30 (15 × 2). Both 

the CCF and CMS method were used and showed excellent correlation results. When using the 

CMS method, one matching pair did not meet the selected CMS criterion (3X) for a “match”, 

and 29 out of 30 pairs of matching bullets were correctly identified, yielding a correct 

identification rate of 96.7%. When using the CCF method, all 30 pairs of matching bullets 

scored at the topmost position on their respective correlation lists, yielding a correct 

identification rate of 100 % (Note: the CMS criteria were applied to topography images here and 

not to traditional reflectance microscopy images). The comparable performances of both 

mathematical models point to the potential unification of decades of CMS empirical data and 

new surface metrology systems.

Advances in Image and Statistical Analysis in Firearms Analysis –

Cont.

Proposed “NIST Ballistics Identification System (NBIS)” Based on 3D 

Topography Measurements on Correlation Cells

John Song

National Institute of Standards and Technology (NIST)

The proposed “NIST Ballistics Identification System (NBIS)” using 3D topography 

measurements on correlation cells can facilitate high accuracy and fast ballistics identification 

and evidence searches. The correlation cells can identify “valid correlation areas” and eliminate 

“invalid correlation areas” from identification. The proposed “synchronous processing” can 

significantly increase correlation speed. Based on the concept of correlation cells, a Congruent 

Matching Cells (CMC) method using three identification parameters is proposed for ballistics 

and toolmark identifications and for high accuracy and fast ballistics evidence searches. The 

proposed method can be used for correlations of both geometrical topographies and optical 

intensity images. All the parameters and algorithms are in the public domain and subject to open 

tests. An error rate reporting procedure can be developed that can greatly add to the scientific 

support for the firearm and toolmark identification specialty, and give confidence to the trier of 

fact in court proceedings.
8

Principal Component Analysis and Other Methods

Nicholas Petraco

John Jay College of Criminal Justice and the Graduate Center

Over the last decade, forensic firearm and toolmark examiners have encountered harsh criticism 

that there is no accepted methodology to generate numerical “proof” that independently 

corroborates their morphological conclusions. Our research focus is to investigate the validity of 

toolmark pattern analysis from an objective, algorithmic and numerical perspective that is fully 

peer reviewed by the wider scientific community and can withstand the scrutiny of the 

adversarial legal system under which the United States operates. With these goals in mind, our 

approach is to use 3D microscopy and multivariate based machine learning techniques. Machine 

learning been successfully exploited for decades in industries to make good, data-based, mission 

critical decisions.

We have already begun to study various types of data using these techniques. A research 

database was assembled (and continues to grow) which consists of 3D striation and impression 

patterns on fired cartridge cases, and screwdriver and chisel striation patterns. This database is 

now available to registered users: http://toolmarkstatistics.no-ip.org/toollab/index.php. Machine 

learning techniques have been applied to a large portion of the primer shears (cartridge cases) 

and screwdriver striation patterns collected thus far. Principal component analysis, combined 

with support vector machine methodology is used to associate these toolmarks with the tools that 

created them. These techniques were chosen not only for their acceptable performance, but also 

because of the minimal assumptions they place on the validity of their conclusions. In our view 

the assumptions of many types of statistical techniques will be their “Achilles’ heel” when 

presented in the adversarial U.S. court system.

U.S. Courts are charged to consider the “...known or potential rate of error...” of the 

“...particular scientific technique...” being presented (Daubert v. Merrell Dow Pharmaceuticals 

Inc., 509 U.S. 579 (1993), at 594). Estimated toolmark identification error rates were computed 

using test sets and well-known techniques of cross-validation and bootstrapping. In order to 

improve the error rate estimation methodology, a wavelet based simulator for 1D toolmark 

“profiles” was created as a temporary measure to help build-up datasets as real toolmark data 

continues to be acquired. 

Many practitioners have also asked for and are interested in numerical measures of quality, of 

algorithmically generated associations between tools and toolmarks. In our strong opinion, there 

is no consensus in the general scientific community as to how to do this, such that practical 

applications will withstand long run scrutiny in the cauldrons of U.S. courtrooms. Our approach 

thus far is two-fold. As a “frequentist” based approach, conformal prediction theory is used to 

assign orthodox confidence levels to each toolmark identification (Vovk 2005, Petraco 2012a,b). 

For a “Bayesian” based approach we are pursuing an empirical Bayes’ methodology. Modern 

machine learning methods output a voluminous amount of information when executing a 

discrimination task. This massive amount of output allows one to leverage Efron’s empirical 

Bayes’ model for an estimate of the posterior error probability of a toolmark ID in a scientifically 

testable way (Efron 2006). Technically, the posterior error probability (PEP) gives an estimate 
9

that the tool truly did not generate the toolmark. This is interesting from a philosophical point of 

view, however, we are strong believers in Gelman and Shalizi’s statement that “...posterior 

model probabilities ...[are]... useful as tools for prediction and for understanding structure in 

data, as long as these probabilities are not taken too seriously.” (Gelman 2012). Thus we 

interpret a PEP value associated with the algorithmic association between a tool and toolmark as 

a goodness-of-fit.

Finally, future directions for our research such as invariant feature extraction and general wavelet 

based “de-noising” will be presented, as well as advocacy for an forensic algorithm 

“Architectural Review Board” (ARB) akin to the Khronos group (www.khronos.org) or 

Boost.org (www.boost.org) familiar from industrial open standard and generic computing. Such 

an ARB can review and put official “stamps of approval” on computational/statistical techniques 

used for court proceedings. 

References

 Vovk V., Gammerman A., & Shafer G. (2005). Algorithmic learning in a random world. 

1st ed. Springer, New York.

 Petraco N. D. K., Chan H., De Forest P. R., Diaczuk P., Gambino C., Hamby J., 

Kammerman F., Kammrath B. W., Kubic T. A., Kuo L., Mc Laughlin P., Petillo G., 

Petraco N., Phelps E., Pizzola P. A., Purcell D. K. and Shenkin P. “Final Report: 

Application of Machine Learning to Toolmarks: Statistically Based Methods for 

Impression Pattern Comparisons”. National Institute of Justice, Grant Report: 2009-DNBX-K041; 2012a.

 Petraco N. D. K., Shenkin P., Speir J., Diaczuk P., Pizzola P. A., Gambino C., and 

Petraco N. “Addressing the National Academy of Sciences’ Challenge: A Method for 

Statistical Pattern Comparison of Striated Tool Marks”. J Forensic Sci 2012b;57(4):900-

911.

 Efron, B. “Size, power and false discovery rates”. Ann Stat 2007;35(4):1351-1377.

 Gelman A. and Shalizi C. R. “Philosophy and the practice of Bayesian statistics”. Brit J 

Math Stat Psyc 2012; To appear.

Measurement Uncertainty & Certainty of Conclusions

Uncertainty: Some Concepts and a Firearms Example

James Yen

National Institute of Standards and Technology (NIST)

The concept of uncertainty as usually applied in a metrology context like at NIST will be 

discussed. Using the context of firearms similarity scores, there will be a discussion of error 

rates for false positives and false negatives. Statistical and graphical methods of summarizing 

distributions of matching and non-matching correlation scores will be described.
10

Conclusion Statements and How to Introduce Evidence (DNA Example)

Mike Ambrosino

U.S. Attorney’s Office for the District of Columbia

This presentation will discuss the evolution of forensic testimony in the District of Columbia 

over the past decade. How the DNA wars and various NRC reports have shaped the manner in 

which scientific conclusions are communicated to the jury.

ATF’s International Correlation Server Project 

Martin Ols

ATF Laboratory

The International Correlation Server Project is an agreement between the United States and 

Mexico and the United States and Canada to share their Integrated Ballistics Identification 

Systems (IBIS) database images. Once connected, it will allow our neighboring countries to 

query our IBIS database as well as the US to query the Mexican and Canadian databases.

Industry Panel on Future of the Discipline: Automated Ballistic 

Search & Identification Systems

ScannBI Technology USA: Evofinder® - Network Based Digital Microscope

Sergey Perunov

ScannBI Technology USA

By this presentation ScannBI Technology USA informs community about Evofinder®system – a 

serious player on the market of Ballistics Identification Systems. It's accented that right imaging 

technology preselected as well as self-designed mechanics, optics and electronics provide high 

quality of initial data produced by the system – digital objects images as in 2D as in 3D. In turn it 

leads to high efficiency of ballistic expertise – as in automated as in manual modes. Some 

samples of objects images are exhibited as well.
11

Special Business Centre Co. Ltd (SBC): What can be named a multipurpose 

expert’s workplace?

Yury Ilyasov

Special Business Centre Co. Ltd (SBC Co. Ltd.)

Examination of microrelief of markings left on bullets and cases by firearms, further conclusion 

about matched markings and possible identification of the firearm: all this is an uneasy, 

responsible and very often energy and time consuming process, and what is most important –

deciding the fortunes of people.

In our report we would like to touch on the prospects of development of firearms identification 

from the point of view of experts’ needs and try to answer the question: What can be named a 

multipurpose expert’s workplace?

We are going to tell about the developments of our company, which optimize the process of 

ballistic examination. They are:

 New version of our scanners POISC-Advanced, opening new possibilities for criminalists 

(new functions of the interface, special software products “Audit” and GIS-module”);

 New equipment POISC-F (Photo-Module), allowing working with photo/video images of 

digital evidence (bullets and cases) in on-line mode, making all possible measurements of 

ammunition geometry and marking left on them for identification of the firearms model.

 Analytical Search and Reference Database “Guns Explorer”, allowing integrating and 

sharing all possible information related with firearms and crimes.

Forensic Technologies Incorporated (FTI): The challenges of capturing and 

successfully comparing bullet images

Alain Beauchamp

Forensic Technology Incorporated

In recent years, the evolution of technology has simplified the measurement of micron-level 

topography. Indeed, several 3D sensors are available for researchers in the forensic ballistics 

field. It is no longer a challenge to capture a high-resolution microscopic 3D image of a bullet 

section or cartridge case.

The comparison of bullets under a comparison microscope is a complex process, especially when 

they are considerably deformed. Each manipulation of the physical samples, such as to shift 

along a bullet's surface to view another section, requires a number of re-adjustments and forces 

the firearm examiner to keep a mental map of the bullet to determine the correct phase and draw 

conclusions. Having a single clear view of the entire circumference can definitely help accelerate 

comparison and determine overall agreement.

There is a huge technological leap between capturing one isolated section of a bullet with a 

sensor and creating a continuous 3D representation of the bullet surface. Some of the challenges 
12

include automated repositioning and tracking of surface regardless of deformations, image 

stitching, lighting and focus consistency, and others.

The research scientists at Forensic Technology have made breakthroughs in automating the 

capture of bullets of almost any shape, and in generating images for effective comparison. This 

presentation describes the key steps that overcame the challenges of automated bullet image 

capture, as well as looks at this technology's performance in terms of correlation 

and visualization.

Infrared Identification Incorporated (IRID): Application of Thermal Infrared 

Imaging to Ballistic Toolmark Identification

Francine Prokoski, PhD

Infrared Identification Incorporated (IRID)

See previous abstract

Role of Standards and Validation to Extend Quantitative 

Measurement into Practice

Physical Standards, Calibrations and Traceability

T. Brian Renegar

National Institute of Standards and Technology (NIST)

Traceability and Quality Control have become increasingly important in forensic work. More 

and more scrutiny is being placed on the scientific measurements and research that are performed 

in firearms laboratories. Laboratory Accreditation is also now a requirement in many cases. The 

National Institute of Standards and Technology (NIST) has been developing two physical 

standards to be used for calibration and quality control of ballistics imaging equipment. These 

are the Standard Reference Material (SRM) 2460 Standard Bullet and 2461 Standard 

Casing. The two precision manufactured standards were developed at NIST and provide a key 

method for firearms labs to satisfy the requirements of laboratory accreditation and quality 

control by establishing traceability and control limits. In addition, the standards enable 

correlations between remote sites, each using a Standard Bullet or Casing. This allows 

additional control checks and diagnostic services to be performed in establishing nationwide 

control and quality assurance of the imaging systems.
13

Adopting New Technology within an Existing Quality Assurance System

Erich Smith

FBI Laboratory

A general overview of the accreditation requirements and responsibilities under ISO/IEC 17025, 

General Requirements for the Competence of Testing and Calibration Laboratories, for the 

validation of novel methods and technologies being incorporated into an existing quality 

assurance system.