Symposium Workshop List

All workshops will take place on Monday, July 24, 2017

Times listed are tentative and subject to change

Lunch is scheduled from 12:30 PM - 1:30 PM, and break times will be scheduled by workshop instructors. Lunch and breaks are on your own; the NIST 

cafeteria is open from 7:30 AM until 3:00 PM.

Workshop 1: Workshop 2: Workshop 3 Workshop 4: Workshop 6: Workshop 8:

8:30 "Comparing Open- "A Primer on Proper "A Risk Based Approach to 

Source Free-of-Charge Root Cause Analysis Measurement Uncertainty & 

Software Programs for and Its Role in Data Integrity in Forensic Drug 

Assessing the Weight Forensic Science Error Analysis"

of Evidence" Management"

Instructors: David Cirullo, Klaus 

Fritsch, Thomas Rohrer, Tucker 

Rubino

Instructors: Carole Chaski, 

Katherine Ramsland, Gregg 

McCrary

Instructor: Robert Langendoen

(Mettler Toledo LLC) (Institute for Linguistic Evidence) (Eagle Valley Search Dogs)

Instructors: Instructors: Instructors: 8:30 AM - 12:30 PM 8:30 AM - 12:30 PM 8:30 AM - 12:30 PM

12:30

Ramit Plushnick-Masti 

and LaShon Beamon

Keith Inman and 

Norah Rudin

 Emma Dutton, Bryan 

Forward, Josh Spatol Seats: 25 max Seats: 50 max Seats: 25 max

LUNCH Workshop 5: Workshop 7: Workshop 9:

1:30 "Automated Liquid Handling 

Essentials for the Forensic 

Practitioner: An Interactive 

Course"

8:30 AM - 5:30 PM 8:30 AM - 5:30 PM 8:30 AM - 5:30 PM

Instructors: Kevin Miller and 

Kimberly Sturk-Andreaggi, Eric 

Sindelar, Sean Oliver, Brandon 

Bare

Instructors: Charlotte Word, 

Lewis Buzzel III, Scott Scoville

Instructors: Prem Rachakonda 

and Bala Muralikrishnan

(Hamilton Robotics) (private consultant) (NIST)

1:30 PM - 5:30 PM 1:30 PM - 5:30 PM 1:30 PM - 5:30 PM

5:30 Seats: 16 max Seats: 20 max Seats: 30 max Seats: 12 max Seats: 60 max Seats: 25 max

Monday

(Houston Forensic 

Science Center, Inc.)

(California State 

University, East Bay)

(CA Department of Justice 

/ Bureau of Forensic 

Services)

"The Scent Picture"

"Sources of Bias for Forensic 

Investigations: Spotting and 

Limiting Their Influence"

"Errors with Evidence and the 

Legal System – Be Prepared!"

"Techniques to Evaluate 

Terrestrial Laser Scanner 

Errors"

"Overcoming the Next 

Crisis: Media Relations 

and Crisis 

Communications in 

the Forensic Services 

Sector"
Overcoming the Next Crisis: Media Relations and Crisis Communications in the Forensic Services Sector

Instructors: Ramit Plushnick-Masti and LaShon Beamon (Houston Forensic Science Center)

Full day Workshop 8:30 AM - 5:30 PM, 16 seats

The goal of this workshop is to teach crime lab directors, public information officers, and other top-level managers how to prepare for and deal with the inevitable next crisis. 

What, how, and when do we release information to the media? How do we answer questions from reporters most effectively? What communication strategies should a crime lab 

have in place to allow it to better handle a crisis? Through interactive, hands-on activities combined with tips, rules of the trade, and other strategies, this workshop will help crime 

lab directors put in place a communications plan and team that can guide them through the good, the bad and the ugly.

During this workshop, participants will practice responding to media inquiries on camera in a realistic crisis communications scenario, and will receive constructive feedback 

from the instructor and other workshop participants on their delivery and responses to difficult questions.

Comparing Open-Source Free-of-Charge Software Programs for Assessing the Weight of Evidence

Instructors: Dr. Keith Inman (CSUEB) and Dr. Norah Rudin (Forensic DNA Consulting)

Full Day Workshop 8:30 AM - 5:30 PM, 20 seats

Amultitude of open-source free-of-charge software programs for assessing the weight of evidence are now available to the forensic DNA community. These programs 

encompass discrete models as well as those capable of automatically modeling a multitude of variables such as peak height and stutter. It can be challenging to learn how each of 

these programs work, to gain an understanding of the capabilities and limitations of each, and to compare them with each other.

We are the creators of one such program (Lab Retriever), and have been fortunate to connect and collaborate with the creators of some of the other programs (LRmix Studio, 

likeLTD v.6, European Forensic Mixtures). As part of a recent NIJ grant, we have created a set of 820 low-template complex mixtures that are useful for various types of 

validation, including comparison of software programs. These samples will be made publicly available, both as raw data (.fsa files) and as analyzed data (.csv files).

Ideally, participants will bring laptops on which one or more of the programs have been downloaded and installed. This workshop will provide a basic introduction to each of 

the four software programs mentioned above. We will then proceed to run samples from the data set above, and conduct an in-class comparison of the same samples run on various 

programs, and under varying assumptions. We will explore the similarities and differences and discuss the underlying reasons for any differences. We will also discuss 

expectations, limits, and the significance of any differences within the context of the inferences they generate. Finally, we will present a large scale comparison of these software 

programs that we have performed.

*Attendees will need to bring laptops with pre-loaded software. Links to download the software will be provided before the workshop.

A Primer on Proper Root Cause Analysis and Its Role in Forensic Science Error Management

Instructors: Emma Dutton (ANAB), Josh Spatola (CA Dept. of Justice/Bureau of Forensic Service), and Bryan Forward (Senior Criminalist, CA)

Full Day Workshop 8:30 AM - 5:30 PM, 30 seats

All organizations regardless of size or mission are prone to "errors" and then having to manage those errors. Forensic science laboratories are no exception. The forensic 

science community, however, must be ever so diligent in actively identifying, understanding and correcting such errors due to the impact that the quality of the work has on the 

criminal justice community. Root cause analysis is a process used to define, evaluate, and systematically analyze a problem (i.e., error) to determine the underlying reason(s) for 

the problem; the output of which is the input to corrective action. Proper error management requires a thorough and robust corrective action process; consequently, it is essential 

for root cause analysis to be thorough and robust in order for the corrective actions to be effective. 

This workshop will provide participants with the basic knowledge and skills to perform root cause analysis and to effectively implement appropriate corrective actions to 

eliminate and prevent the problem from recurring. During this workshop, we will evaluate the corrective action process, define root cause analysis and discuss the philosophy and 

purpose of root cause analysis. We will outline the basic steps of root cause analysis and describe an effective approach for performing root cause analysis. We will learn the 

difference between correction vs. corrective action, the process of asking ‘why’ at least five times to determine the underlying reason(s) for the problem, and learn why ‘blaming 

the individual’ is missing the point. Attendees will acquire skills and learn an approach for evaluating and improving the effectiveness of a management system through effective 

root cause analysis. In class exercises will be used and numerous opportunities for discussion and Q&A are included. 

Forensic specific examples will be provided. These examples will demonstrate how a thorough root cause analysis benefits the laboratory organization, the laboratory 

employees, and the laboratory customers by providing continual improvement opportunities. 
Root cause analysis is a skill that must be learned, and a process that requires continuous improvement and resources. Too costly, some might say. Are you willing to accept 

the risk of not doing root cause analysis well?

A Risk Based Approach to Measurement Uncertainty & Data Integrity in Forensic Drug Analysis

Instructors: Tucker Rubino, Dr. Ian Ciesniewski, Dr. Klaus Fritsch, and Thomas Rohrer (Mettler Toledo LLC)

Half Day workshop 8:30 AM - 12:30 PM, 25 seats

Weighing is a key step in qualitative and quantitative drug analysis and strongly influences the integrity of the final result. The standout prerequisite for traceable and accurate 

weighing is the effective calibration of weighing instruments, which also comprises the estimation of measurement uncertainty. Historically, many laboratories set their own 

calibration procedures due to the lack of nationally or globally recognized calibration guidelines. Based on international cooperation from subject matter experts in the field of 

metrology, efforts were made to globally harmonize the methodology to calibrate weighing instruments. The major benefit is the ability to estimate measurement uncertainty at the 

time the instrument is calibrated and also provide guidance to estimate uncertainty during day-to-day usage. This resulted in the calculation of the minimum weight becoming a 

key parameter for quantitative drug analysis. The minimum weight is the smallest amount of net substance that needs to be weighed in order to achieve a specified degree of 

accuracy. It ensures the overall analysis is not negatively impacted by an insufficient sample weight.

To help meet requirements set forth by regulatory organizations it is also important to understand the benefits of incorporating the weighing process with an integrated data 

management system. In recent years an increasing number of assessments and internal audits have revealed incomplete data, lack of audit trails, and falsification of results. While 

most labs have turned toward LIMS systems with the idea of replacing the manual workflow, these systems are designed primarily to aggregate result data from an array of 

analytical tests - not automate and document bench top workflows or provide traceability to the measurement.

Regulatory organizations have recognized the advantages of electronic data systems and have increasingly established further controls for the use of such systems all the way 

down to bench top instruments. The goal of reducing errors, simplifying processes, and reinforcing compliance can become challenging when directly integrating and automating 

bench top instruments. As regulators continue to tighten auditing approaches, it is critical for forensic scientists to understand the key issues surrounding data integrity.

This workshop is intended for Lab Managers, Quality Control Managers, and Drug Chemistry Forensic Scientists. It will provide an overview on the harmonization of 

calibration procedures and the resulting concept of incorporating a minimum weight with measurement uncertainty. It will discuss the criteria for data integrity based on recent 

guidance issued by various regulatory agencies. It will provide practical solutions to improve data management processes and address many data integrity weaknesses typically 

found in Forensic laboratories.

Attendees will walk away with a better understanding of the fundamentals of weighing and the importance of a risk based approach to measurement uncertainty. Additionally, 

individuals will grasp a better understanding of data integrity and the management of weighing results while learning about common issues that arise in Forensic Labs.

Automated Liquid Handling Essentials for the Forensic Practitioner: An Interactive Course

Instructors: Dr. Kevin Miller*, Eric Sindelar*, Kimberly Sturk-Andreaggi†, Sean Oliver†, and Brandon Bare*(

*Hamilton Robotics, †Armed Forces DNA 

Identification Lab)

Half Day Workshop 1:30 PM - 5:30 PM, 12 seats

Automation in a forensic laboratory offers advantages over manual processing including increased throughput, robust sample tracking, greater precision and a reduction in 

error. However, the uncertainty surrounding automated liquid handling – which may include the instrument itself, pipetting behaviors and volumes, the physical properties of 

liquids being pipetted, environmental factors within the laboratory, and even operator skill – can seem daunting. This half day workshop is designed for Criminalists, Quality 

Assurance Officers, and Forensic Laboratory Managers who want to accomplish more with automated liquid handling platform(s) in their laboratory. Workshop facilitators include 

scientists, engineers, and forensic practitioners who employ a balanced mix of discussion and activities designed to address each of the above factors with the aim of optimizing 

liquid handling performance. 

The workshop will begin with the fundamentals of automated liquid handling, which provides participants with an overview of liquid handling and how volume is determined 

on automated platforms with a review of the factors that affect liquid delivery. Emphasis is placed on: 1) classification of liquids based upon their physical properties and how 

those properties can affect pipetting performance, 2) the methods by which liquids are handled, exploring critical components of liquid transfer steps and examining how various 

parameters influence performance, and 3) how volumes are monitored and measured, with a review of the various approaches used to ensure proper liquid handling and to maintain 

assay performance. 
The discussion will then move to important considerations when automating assays within the forensic laboratory. These include workflow design, including sampling, sample 

preparation, measurement, data evaluation and data transcription process, as well as forensic validation, instrument verification, and performance checking within the ISO 17025-

regulated forensic environment.

Major concepts are supported by three interactive exercises. In the first of these exercises, which deals with measurement, workshop participants will use a robotic workstation 

and employ a gravimetric approach in order to gather volume transfer data, and then use these data to assess trueness and precision of the volume transfer. The second exercise 

emphasizes liquid classification. Here, participants will run a computer simulation of robotic liquid verification and workshop instructors will demonstrate how to make 

adjustments to correction curves in order to account for trueness. The final exercise looks at how three forensic laboratories chose to validate robotic platforms within their 

laboratories. The initial verification process will be explained in terms of the IQ/OQ process that a robotic provider goes through, followed by handoff to the forensic practitioner 

who performs PQ (and any additional OQ) and validation for casework. Finally, ongoing preventative maintenance performed by the robotic provider and routine checks 

performed by crime lab personnel will be discussed in terms of best practice. 

The workshop will conclude with a question and answer discussion session.

At the end of the workshop, each participant should be able to:

1. Describe the nature of the measurements that are performed by automated liquid handling platforms, and distinguish the quality features that can be used to mitigate 

uncertainty in these measurements;

2. Examine the role of liquid classes in automated liquid handling, and the relationship that liquid class libraries have on liquid handling behavior; and

3. Create a plan to evaluate the quality of a forensic automated liquid handling workflow.

Sources of Bias for Forensic Investigations: Spotting and Limiting Their Influence

Instructors: Dr. Carole Chaski (Institute for Linguistic Evidence), Dr. Katherine Ramsland (DeSales University), and Gregg McCrary (Behavioral Criminology 

International)

Half Day Workshop 8:30 AM - 12:30 PM, 50 seats

This workshop will use facts and insights from cognitive psychology and linguistics to demonstrate ways in which cognitive and linguistic biases can influence investigative 

and interpretive judgment. With improved awareness, participants will learn potential solutions for reducing bias. Experts on this panel will present strategies for identifying 

common sources of bias and for strengthening anti-bias techniques. Participants will be invited to describe their own experiences and to request suggestions.

Interpretive filters, or biasing mechanisms in human cognition assist with efficient information processing, but they can also result in erroneous assumptions, especially as they 

inform our cognitive maps. The concept of cognitive mapping describes how we become habituated to a specific perspective. It means that we encode, recall, and recognize our 

“situated existence” according to familiar parameters. We grow used to these frames, which guide how we physically and emotionally process the world. These maps become our 

cognitive positioning system (CPS), the center from which we mentally navigate for problem solving and decision-making. Since the brain does not naturally self-reflect, we are 

often blind to our limits and errors. 

People who make judgments that significantly affect others’ lives must understand the potential impact of their CPS. Decisions that feel good might be accepted as right, even 

truth, prematurely ending or diverting the investigative process. Yet feelings can arise merely from what is familiar or what seems clear or complete. The panel will discuss some 

of the most common processing factors that influence our CPS. 

Professor Katherine Ramsland will describe how the CPS develops from mental maps, showing how things learned from families, culture, and subcultures subtly inform our 

personal interpretations. People seek clarity and closure, which can hasten judgments and trigger errors. Certain perceptual sets pose a persistent challenge for investigators, due to 

reliance on gut instinct, need for closure, and expectation. This can be as true for post-incident consultants as for front-line investigators. Although mental shortcuts (heuristics) 

reduce complexity and help to organize information, this same benefit can introduce error. Researchers have examined many types of heuristics, attention dynamics, and cognitive 

illusions that subtly impact decision-making. 

Former FBI Supervisory Special Agent Gregg McCrary will discuss how the cognitive traps specific to law enforcement investigations result in errors of misclassification, 

tunnel vision, and confirmation bias, which affect observation, interpretation, interviewing and interrogations. Complex tasks like criminal investigation should be undertaken 

carefully, with eyes and mind open to new information and improved methods. Among the anti-bias strategies to be presented are teaching how to form and analyze competing 

hypotheses, identifying unsupported assumptions, and spelling out areas of ambiguity as attractants for heuristics.

Linguist Carole Chaski will describe a strategy, based in experimental psycholinguistics, for overcoming cognitive bias. In experimental psycholinguistics, initiation of bias is 

known as semantic priming. Semantic priming happens when a person is “primed” or “made ready” to interpret language and events in a specific interpretation. A welldocumented psycholinguistic result is the particular manipulation of semantic interpretation based on stimuli that prime certain interpretations over others. Semantic priming can 

be driven by word choice, cohesion, presupposition and other very common linguistic structures; in fact, we are constantly using semantic priming techniques in our daily 
conversations in order to persuade our conversation partners but also to simplify and take short-cuts in conversation. Chaski presents a psycholinguistic strategy for overcoming the 

cognitive bias triggered by semantic priming. This psycholinguistic strategy – the metalinguistic filter—relies on basic awareness of semantic priming techniques and a strong 

scientific methodology as bias correctives. Chaski provides examples, throughout the forensic workflow, of both semantic primes triggering cognitive bias and the metalinguistic 

filter de-biasing the workflow.

Errors with Evidence and the Legal System – Be Prepared!

Instructors: Dr. Charlotte Word (Forensic DNA Consultant), Lewis H. Buzzell III, J.D. (4th Judicial Circuit of Florida), and Scott Scoville, J.D. (Orange County, 

CA), 

Half Day Workshop 1:30 PM - 5:30 PM, 60 seats

The primary goal of this workshop is to foster effective communication between forensic scientists and lawyers with the aim of achieving an enhanced understanding of the 

scientific data in all cases impacted by forensic scientists. A main focus of this workshop will be to learn more about the responsibilities and duties of the forensic scientist in 

relationship to the roles and responsibilities of the prosecutor and defense attorney in routine legal matters, with a particular focus on how best to address errors that have been 

discovered in or outside of the laboratory. Information regarding neutrality, objectivity, transparency and best practices when communicating about the evidence, testing data, 

interpretations and conclusions used by the stakeholder within the judicial system will be provided. The presentations and discussions should be relevant to any forensic science 

discipline and will provide a strong foundation for all attendees interested in learning more about how to manage errors. 

The three presenters for this workshop, which include a defense attorney, a prosecutor and a forensic scientist, have a wide range of experience and expertise with a wealth of 

knowledge and familiarity regarding the presentation of forensic science evidence inside and outside of the courtroom. While having significantly different roles, all three 

presenters are united in their objective of ensuring that all forensic science examinations, evaluations, reports and testimony are accurate, credible and can be relied upon by all 

users of the information when effectively communicated. 

The intended audience for this workshop is anyone wanting to gain a better understanding of how to handle and communicate errors within the legal system, including forensic 

scientists from any discipline, laboratory management, quality assurance managers, law enforcement, attorneys, judges, and victim advocates. There are no prerequisites for this 

workshop. 

Participants of this workshop will gain a greater understanding and appreciation of the roles of the defense attorney and the prosecutor in their necessary interactions with 

forensic scientists. The advocacy role of each attorney, and especially the role of the forensic science expert to advocate for reliable science, best practices and transparency of the 

forensic science information related to a case, will be discussed. Information will be provided regarding the possible additional responsibilities required of the forensic scientist 

when retained or subpoenaed by the prosecution vs. defense attorney due to the legal responsibilities of the attorneys. Participants will gain a better understanding of the 

continuous obligation to provide discovery to the prosecutor before, during and after a case has been adjudicated, including during post-conviction relief matters to ensure the 

defense attorney has access to all relevant information. The workshop will better prepare the forensic scientist for communicating test results, conclusions and any errors during 

pre-trial discussions with the prosecution and defense attorneys and during testimony.

The Scent Picture

Instructor: Robert Langendoen, Eagle Valley Search Dogs

Half Day Workshop 8:30 AM - 12:30 PM, 25 seats

Since the beginning of time we have relied on the dogs unique abilities...paramount of which is their keen sense of smell. Today we utilize our K9 partners for this but still 

have only scratched the surface of what they do and how they do it so remarkably well. This discussion will cover the anatomy and mechanics of how dogs gather scent and the 

most recent studies as to what scent is. More importantly however we will take a look at describing what the scent picture represents to the dog, how it becomes relevant to our 

profession as Handlers and how we train the dog for specific jobs to benefit mankind and Law Enforcement. 

The errors we generally encounter are centered around how the dogs are trained for various types of scent detection and will be of interest to those whose job it is to have the 

proper resources available for crime scenes or suspected cold case crime scenes where Human Remains may be present in various forms including clandestine graves...even 150 yr 

old bones!

Recent research has identified over 2 dozen variables in decomposition odor which include Volatile Organic Compounds (VOC’s). The studies done by Dr. Vass, et.al. at the 

University of Tennessee where they have a “Body Farm” and papers written by Katherine Bamford PhD give us a better understanding of the complexity of odor detection and the 

use of K9’s in this process.
While we are likely only beginning to understand the complex nature of decomposition odor it is clearly evident that K9s are a valuable asset in the detection of Human 

Remains and will continue to be so for many years into the future. While mechanical devices are being sought after to simulate what the dogs nose does, none have come close to 

their ability to detect odors on a scale of parts per billion! The key is proper training and the ability of Handlers to recognize subtle detection clues by the K9 and the event of a 

recognized final indication. 

The broader implications center around Standards and Methods used for K9 training so that when evidence is presented in the courts it cannot be disputed.

Techniques to Evaluate Terrestrial Laser Scanner Errors

Instructors: Prem Rachakonda, and Bala Muralikrishnan (NIST)

Half Day Workshop 1:30 PM - 5:30 PM, 25 seats

Forensic photography has been one of the most important tool for investigators over many decades. They preserve the history of the scene and provides a way for investigators 

to revisit the scene. However, it is difficult to ascertain any objective measurements with high confidence from such photographs. Terrestrial Laser Scanners (TLSs) offer a way for 

investigators to revisit the scene and enable them to measure many features such as a trajectory of a bullet, track width of a vehicle etc. Because they also capture intensity data, 

they offer a way to reconstruct the photographs in any desired orientation from the measured data. 

TLSs do suffer from some issues that can cloud the veracity of the measurements. Their construction and data analysis procedures could introduce errors that have to be 

characterized. To standardize the measurements from TLSs, the Dimensional Metrology Group (DMG) at the National Institute of Standards and Technology (NIST) has been 

working with TLS manufacturers to develop a new documentary standard for these class of instruments. The existing ASTM E2938-15 standard for 3D imaging systems only 

addresses the performance of these instruments in the ranging direction, however, most forensic scans are used for measurements in many directions. 

The new proposed standard addresses the issues with the performance evaluation of these instruments for any spatial distance measurement. This standard is being developed 

under the auspices of ASTM and is presently in the pre-balloting phase. In May 2016, a run-off meeting was held at NIST where five manufacturers brought their instruments and 

tested them per the procedures in the draft standard. 

NIST is a major contributing member of this committee and possesses unique expertise for this task due to our prior experience with laser trackers that could be applied for 

evaluating TLSs. Extensive work was done at NIST to model the behavior of these instruments and many of the test positions being recommended for the standard are based on 

this work. 

In this workshop, we will discuss the errors and accuracy of TLSs and other efforts to characterize the performance of TLS systems to ensure that they meet the needs of the 

forensics community.

Purpose: Disseminate information about error sources in Terrestrial Laser Scanners (TLSs) and methods to evaluate these instruments for use in forensic applications. 

Intended audience: Forensic community (No prerequisites needed)

Issues: Understand various sources of errors while using TLSs

Learning outcomes: 

1. Learn about the use of TLSs in forensic applications.

2. Learn about the sources of the errors due to the construction of the instrument, targets, data analysis procedures etc.

3. Learn about the methods to test these instruments to evaluate the performance in an objective manner.