The Future State of Handwriting 

Examinations: A Roadmap to Integrate 

the Latest Measurement Science and 

Statistics

John Paul Jones II

Program Manager

National Institute of Standards and Technology

AAFS Annual Meeting

February 20, 2014

Seattle, WA
Agenda

• National Institute of Standards and Technology

• MSSFHA Planning Team & Conference

• MSSFHA Facilitated Session – Future, Barriers, Roadmap

• Recommendations
National Institute of Standards and Technology

• Advances measurement science, standards, and technology 

in ways that −

• Promote U.S. innovation and industrial competitiveness

• Enhance economic and physical security

• Improve our quality of life 

• Founded in 1901 as the National Bureau of Standards

• Within the U.S. Department of Commerce

• Non-regulatory agency Gaithersburg Campus
National Institute of Standards and Technology

•Non-regulatory status enables 

an important role as a convener 

that facilitates collaboration 

between agencies of the 

Federal Government, industry, 

private organizations, and state 

and local governments.
Agenda

• National Institute of Standards and Technology

• MSSFHA Planning Team & Conference

• MSSFHA Facilitated Session – Future, Barriers, Roadmap

• Recommendations
Measurement Science and Standards in 

Forensic Handwriting Analysis (MSSFHA) 

Conference & Webcast

• Held June 4-5, 2013, at NIST in Gaithersburg, MD 

with live webcast.

• The purpose of this conference was to enhance 

the current state of forensic handwriting analysis 

by exploring of the latest advancements in 

measurement science and the latest research 

investments in quantitative analysis capabilities.

• Archived: http://www.nist.gov/oles/handwriting.cfm
MSSFHA: Collaborative Effort

• Designed by a planning team consisting of representatives 

from:

• American Academy of Forensic Sciences –Questioned 

Document Section,

• American Board of Forensic Document Examiners 

(ABFDE),

• American Society of Questioned Document Examiners 

(ASQDE),

• Federal Bureau of Investigation (FBI) Laboratory,

• National Institute of Justice (NIJ),

• National Institute of Standards and Technology (NIST),

• Scientific Working Group for Forensic Document 

Examination (SWGDOC).
MSSFHA: Planning Team

• Jane A. Lewis – AAFS QD Section

• Kirsten Singer – ABFDE

• John Paul Osborn – ASQDE

• Rigo Vargas – SWGDOC

• JoAnn Buscaglia – FBI Laboratory

• Heather Waltke – National Institute of Justice (NIJ)

• John Paul Jones – National Institute of Standards and

• Technology (NIST)

• Contract Support – Kristen Mehamed with SAIC
Quantitative Systems for Forensic

Handwriting Analysis

• WANDA – A Measurement Tool for Forensic Document 

Examiners

• Katrin Franke, PhD – Gjovik University College

• FLASH-ID – Forensic Language Independent Analysis System for 

Handwriting Identification

• Mark Walch – Gannon Technologies Group

• CEDAR FOX and iFOX

• Sargur Srihari, PhD – SUNY Buffalo

• D-Scribe

• Matthias Schulte-Austum - Siemens AG

• SIFT – Scale Invariant Feature Transform

• Jeffrey Woodard, PhD – The MITRE Corporation

Note: FISH is another quantitative system, however

no presentation was delivered during the event
95 Answered
Advances in Statistics for Handwriting Analysis #1

• Trends in Frequency Occurrence of Handwriting and Hand 

Printing Characteristics

• Thomas W. Vastrick - Private FDE

• Error, Confidence and (Un)certainty – Deconstructing Authorship 

Opinions Using a Forced-Call Testing Protocol

• Brent Ostrum - Canada Border Services Agency

• Understanding Individuality of Handwriting Using Score-Based 

Likelihood Ratios

• Christopher Saunders, PhD - South Dakota State University

• The Development of Individual Handwriting Characteristics and 

the Statistical Evaluation of Different Combination Likelihoods of 

These Individual Characteristics

• Lisa Hanson - Minnesota Bureau of Criminal Apprehension
• Handwriting Evidence Evaluation Based on the Shape of 

Characters: Application of Multivariate Likelihood Ratios

• Raymond Marquis, PhD - Institute of Forensic Science, University 

of Lausanne

• Statistical Basis to Determine Probabilities of Occurrence of 

Handwriting Characteristics

• Kirsten Singer, D-ABFDE - Department of Veteran Affairs, OIG

• Sargur Srihari, PhD - Buffalo

Advances in Statistics for Handwriting Analysis #2
96 Answered
96 Answered
Agenda

• National Institute of Standards and Technology

• MSSFHA Planning Team & Conference

• MSSFHA Facilitated Session – Future, Barriers, Roadmap

• Recommendations
Facilitated Session: The Questions

• What does the future state of handwriting analysis look like?

• What are the barriers to implementing the future state?

• What does a roadmap to achieve the future state look like?

• Facilitated By:

• John Paul Osborn, Rigo Vargas, John Paul Jones II
Facilitated Discussion Disclaimer

•This presentation summarizes the 

concluding discussion, including 

conversations in the room and 

comments submitted online. Although 

not every participant agreed with every 

comment, this summary gives a 

general sense of the discussions.
What does the future state of handwriting 

analysis look like?
What does the future state of handwriting 

analysis look like? (1)

• The discipline will incorporate more quantitative analysis tools 

during the examination process to access and compare 

handwriting characteristics.

• FDEs will employ the use of statistical models to explain 

significance of conclusions based on the uniqueness of 

observed and measured handwriting characteristics.

• Researchers will publish more studies involving the use of 

quantitative analysis in peer-reviewed journals.

• More peer-review of casework and proficiency testing 

requirements as all forensic disciplines move toward mandatory 

accreditation and certification.
What does the future state of handwriting 

analysis look like? (2)

• Proficiency tests will better simulate actual casework.

• NOTE: Some FDEs believe that peer review and proficiency 

testing will be required by the court system.

• Previously established standards will be updated or validated 

as new technology is used to test long-standing practices.

• More formal use of statistics determining uncertainty and 

strength of similarities between writings.

• Opinion scale currently in use will change...
What does the future state of handwriting 

analysis look like? (3)

• As research demonstrates a better understanding of the 

frequency of characteristics in handwriting samples, FDEs will 

be able to formalize their conclusions in a more uniform, 

quantifiable fashion.

• Note: Automated comparison systems may be considered 

separate from statistical models, as automated systems can 

facilitate the matching of a known writer with questioned 

documents without necessarily generating statistics.

• Ultimately FDE’s can use statistics and automated systems 

to complement their current practices and to enhance the 

way they review cases – but neither can replace humans.
Think about it...

• In current practice, when an FDE determines the genuineness, 

simulation, or disguise of a handwriting sample, the FDE is 

often mentally calculating the frequency of observations of 

certain strokes and specific features, studying the geometrical 

outlines and areas of letters, developing ratios between 

extenders and small letters, noting the variability of indicators...

• The FDE conclusions already involve approximate mental 

“statistical observations” but they do not include a metrical 

evaluation of the parameters.

• Participants believe it is quite reasonable to start generating on 

paper the calculations that FDE’s usually perform mentally 

without any actual measuring.
What does the future state of handwriting 

analysis look like? (4)

• Courts will demand more statistical data with conclusions like 

DNA model.

• Reporting of results in the criminal justice system is changing 

toward degrees of confidence, and FDEs will have to deal 

with statistics to show this.

• Note: statistics may help with the attorneys and court 

however may not be helpful to jurors... Many FDEs will 

instead present conclusions with images that present a 

clearer picture to the jury than statistical jargon would.

• Society will move away from formalized education on 

handwriting which may result in more unique handwriting 

styles and may encourage hand-printing. 
What are the barriers to implementing the

future state?
What are the barriers to implementing the

future state? (1)

• Statistics may be too complicated for the jury to 

understand and may be manipulated by attorneys to show 

excessive doubt.

• Many FDEs lack statistical training – makes explaining 

probabilities or ratios challenging.

• Field is aging and current FDEs may struggle to accept the 

new procedures and technologies.

• Some labs experience decreasing caseloads and staffing 

in QD while DNA units in the same lab are expanding.
What are the barriers to implementing the

future state? (2)

• Crime scene investigators may not know the value the FDE 

can provide for a case.

• More cases going civil route than criminal.

• Research is expensive and FDEs have challenges 

connecting with top researchers – much research has been 

done by FDEs outside their normal casework without 

additional funding, often focusing on case-specific 

questions. (changing recently)

• Many FDE research publications are in professional 

journals with targeted readership or presented during 

discipline specific conferences.
What does a roadmap to achieve the future

state look like?
What does a roadmap to achieve the future

state look like? (1)

• Increased research to validate existing methods and develop new 

technologies. 

• NIJ continues support and other agencies like NSF should invest 

to encourage larger projects which will result in publications in top 

tier peer-reviewed, general science journals.

• FDEs and statisticians collaborate more regularly to identify the 

statistical methods that will best help casework and testimony –

test with real-world data. 

• FDEs receive formal training in statistics. (Some larger labs have 

associations with statisticians)
What does a roadmap to achieve the future

state look like? (2)

• More collaborations through peer-review of casework and 

proficiency testing – increases validity of conclusions.

• Collaborations with academic institutions with students writing 

their masters thesis on a QD topics. Allows professors to 

investigate larger issues and also creates the next generation of 

FDEs. 

• Campaign to attract the next generation of FDEs. 

• Reaching into academic institutions generates opportunities to 

collaborate with groups outside of your discipline that may be 

working on common challenge areas. (i.e. statistics, psychology, 

etc.).
What does a roadmap to achieve the future

state look like? (3)

• Technology advances and validation of methods can result in 

new opportunities (digital signatures) and increased 

caseloads. This is a good thing when some labs are reporting a 

decrease.

• As existing technological aids are tested and validated, FDEs 

should incorporate them into casework with a full understanding 

of the value they offer.

• More training for law enforcement officials on FDE capabilities 

and your value proposition. 

• Labs must allow FDEs to be trained on latest techniques. 
What does a roadmap to achieve the future

state look like? (4)

• Although modern communication is more often typed rather 

than handwritten, additional technological advancements 

could allow FDE’s to analyze high-resolution electronic 

signatures and to conduct linguistic analysis on documents.

• FDE’s must continue to review their processes and standards 

to support existing and forthcoming technology.

• Everyone needs datasets – need a repository...

• To conduct reliability and reproducibility studies on FDE 

performance and newly developed quantitative measurement 

systems.
Agenda

• National Institute of Standards and Technology

• MSSFHA Planning Team & Conference

• MSSFHA Facilitated Session – Future, Barriers, Roadmap

• Recommendations
Recommendations 1-3

1. FDEs should receive basic statistical training relevant to 

forensic handwriting analysis to help formalize the use of 

statistics in casework and to communicate statistical findings to 

the court.

1. Through SWGDOC and the American Statistical Association’s 

ad hoc committee on forensic science, FDEs should 

collaborate with statisticians to develop better statistical 

models to address handwriting analysis challenges.

1. NIJ should continue its enthusiastic support of forensic 

handwriting analysis research, and other agencies should 

contribute to this support. 
Recommendations 4-6

4. FDEs should collaborate with academic institutions and 

should leverage graduate students to conduct studies 

on case-specific questions.

5. Researchers should strive to publish forensic 

handwriting analysis studies in top-tier, peer-reviewed 

journals to invite the level of scrutiny and acceptance 

experienced by other scientific fields. 

6. Through SWGDOC or OSAC, FDEs should develop an 

expanded and standardized list of conclusions with 

degrees of confidence based on scientific research.
Recommendations 7-10

7. FDEs and researchers should consider establishing an opensource data set with anonymous handwriting samples similar 

to real-world casework to use in testing and validating 

technological tools and statistical models. 

8. NIST should consider serving as a repository for handwriting 

datasets and should make this material available to FDEs and 

researchers to test new technological developments.

9. Through SWGDOC or OSAC, FDEs should revise and develop 

standards that apply to the latest technological advances.

10. Professional associations that represent FDEs should consider 

launching campaigns to attract younger scientists to the 

discipline.
Other Ideas

• SWGDOC

• http://www.swgdoc.org

• American Statistical Association ad Hoc Forensic 

Committee

• http://www.amstat.org/policy/forensicscience.cfm

• National Commission on Forensic Science (NCFS)

• Policy issues

• Organization of Scientific Area Committees (OSAC)

• http://www.nist.gov/forensics/osac.cfm

• National Science Foundation “Dear Colleague Letter”

• http://www.nsf.gov/pubs/2013/nsf13120/nsf13120.jsp

• Optical Character Recognition Technology Partners
Organization of Scientific Area Committees (OSAC)

SAC = Scientific Area Committee

Sub = Subcommittee

Imaging Technologies Sub Firearms & Toolmarks 

Sub 

Questioned Documents 

Sub

DNA Analysis Sub1 Controlled Substances Sub Anthropology Sub Facial Identification Sub

Disaster Victim 

Identification Sub

Friction Ridge Sub

Fire Debris and Explosives 

Sub (lab)

Materials (Trace) Sub

Medical/Legal Death 

Invest Sub

Blood Stain Pattern 

Analysis Sub

Toxicology Sub

Dogs and Sensors Sub

Footwear 

& Tire Tread Sub

Forensic Science Standards Board (FSSB)

Wildlife Forensics Sub

Geological Materials Sub

Speaker Recognition Sub 

SAC

Biology/DNA

Quality Infrastructure 

Committee (QIC)

SAC

Physics/Pattern

SAC 

Chemistry/ 

Instrumentation

SAC

IT/Multimedia

SAC

Crime Scene/ 

Death Investigation

Fire Scene and 

Explosives Sub 

Legal Resource 

Committee (LRC)

DNA Analysis Sub2

Human Factors 

Committee (HFC)

Gunshot Residue Sub
Questions

John Paul Jones II

National Institute of Standards and Technology

100 Bureau Drive, Mail Stop 8102

Gaithersburg, MD 20899

301.975.2782

john.jones@nist.gov

www.nist.gov/forensics