Page 1 of 5

Measurement Science and Standards in 

Forensic Handwriting Analysis Conference & Webcast

Facilitated Discussion: Raw Comments

June 5, 2013

Overview

During the Measurement Science and Standards in Forensic Handwriting Analysis Conference and 

Webcast on June 4-5, 2013, a facilitated session was held to discuss the future of the forensic science 

discipline of handwriting analysis. In-person event attendees provided verbal input and webcast viewers 

provided comments through email (forensic@nist.gov) and Twitter (#NISTForensics). 

This document includes the aggregated raw comments captured during the facilitated session. We are 

unable to determine the identity, occupation, or associated agency of the individuals that contributed 

specific statements. The following three questions were used to frame the discussion: 

1. What does the future state of handwriting analysis look like?

2. What are the barriers to implementing the future state?

3. What does a roadmap to achieve the future state look like?

What does the future state of handwriting analysis look like?

“Ideal” forensic document examiner (FDE) is able to communicate information to fact finders 

and juries; do numbers allow effective communication? 

Statistics help in practice and in explaining conclusions to attorneys and judges (deposition), but 

maybe not as helpful to juries. Have to explain in a basic way, providing them with the 

knowledge they need to make independent judgment.

Clear picture for the jury, images help more than statistical jargon—handwriting is a visual sort 

of evidence.

Most FDEs have little statistical training, which makes it much harder to explain the statistics to 

the jury.

Analysis and communication to fact-finder are two different things.

Maybe reference the statistical basis of the forensic handwriting discipline in testimony, or 

should statistics be ignored in testimony entirely? 

Individuality of handwriting: can use statistics to prove and analyze and to respond to Daubert, 

but can’t really use in front of juries.

Statistics are helpful for research on evaluation and changing practices based on new research 

and to respond to Daubert. But when talking to juries, use electronic charts and images. FDEs 

also have to learn how to use the courtroom testimony technology because it’s the charts and 

presentation that are important.

Statistics can be misused by the other side to question confidence, can confuse the issue.

Statistics are used in research to validate and improve methods.
Page 2 of 5

Statistics have been used in case work for many years—during the Rice Will Case, AS Osborn 

used a lot of research and calculations, but Osborn left them out of testimony.

Concern with automated ID systems and scanning of docs to get a conclusion with a probability, 

could be misused/misunderstood by laypeople.

There should be a systematic way to convert research into best practices that practitioners can 

use. Originally ASTM and SWGDOC have, and SWGDOC should continue and should be 

constantly revisiting old standards for validity and appropriateness with new technology.

Automated system is a separate issue than statistics: e.g., FLASH-ID automates the matching of a 

known writer and questioned doc, but without creating statistics; more for analysis during 

examination and may provide new information that human examiner would not have collected.

Statistics are really just short-hand ways to describe the world, but it’s different than the way 

most people think. They should serve to compliment the current practices and enhance way 

FDEs review cases, not to replace humans. They help with uncertainty, and most courts won’t 

accept “100% certainty”. FDEs will have to work with technology and statistics to answer tough 

questions in court, but these changes may not dramatically affect casework.

Future of discipline will involve quantitative analysis tools. Statistics will continue to be used 

more as FDEs learn more about them, and continuing research will evolve understanding of the 

discipline.

Lack of formal education on handwriting will affect discipline dramatically. Students are now 

learning keyboarding. 

Adapt or die—FDEs are seeing decrease in case load. Handwriting testimony is at turning point 

now since courts now like and expect statistics (because of DNA) so will probably be asked 

during testimony.

Make the statistics real and understandable to jury.

Opinion scale will likely be adapted, but FDEs don’t yet know where it should go—need more 

research.

Courts may demand certainty in the future, which statistics will help convey.

DNA experts report a statistical analysis, not individualization.

Examiners can measure uncertainty, but they have to convey it properly.

100% certainty is not a reality, but FDEs can appropriately apply statistics to results.

The current FDE reaction to a trend away from binary conclusions is similar to the latent print 

examiner reaction a few years ago. Reporting is changing toward understanding that there are 

degrees of confidence, and examiners have to deal with statistics to show this. During Daubert 

trials, judges will want error rates, even if they are not conveyed to the jury directly.

FDEs are using frequency of characteristics informally to analyze samples already, maybe now is 

time to start formalizing.

Must adapt to improving technology as field evolves to involve more statistical evaluation.

Terminology in opinion scale is level of confidence and demonstrable basis of opinion, not 

statistical judgment. ASTM/SWGDOC standard says it’s your level of confidence in opinion, not 

100% certainty

Lack of formal writing education in schools may make writing more unique and identifiable.

Decreasing caseloads may be lab process problem, specific to certain labs.

Technology can help compare bad scans.

Could be confident that “inconclusive” is the right conclusion. Practitioners shouldn’t state 100% 

certainly given their personal error rates.

In Netherlands, verbal likelihood ratios are now accepted in court, but this took explanation and 

transition. 
Page 3 of 5

What are the barriers to implementing the future state?

Client and court’s level of understanding; they may or may not accept changes to FDE 

testimony.

May not be able to calculate statistics accurately, as they are more than class characteristics.

Level of conclusion language is unclear to jury because it is not standardized and it uses

“probability” without numbers.

Need to avoid getting lost in the statistics or relying solely on them. Instead, use statistics to 

help others and selves determine how reliable the discipline and work is. Statistics should be 

used to supplement testimony.

Older FDEs may be reluctant to or incapable of adopting new technology and methods.

FDEs need a way to understand Bayesian theory work and need teachers to show them.

“Statistics” isn’t one answer, but is a complex discipline with multiple tests. FDEs have to ask 

right question and the statistician has to understand question. Currently, the statisticians don’t 

always agree on the right method to answer the question.

The disciplines need people who can do solid studies, but these studies are expensive and 

require high level of expertise.

Funding and resources are needed for validations, techniques, automated tool development, 

and data collection (small compared to fingerprints).

Detractors and critics pull research energy and funds away from other, more pressing research 

needs. Have to answer the critics for the courts, even though the critics may never be happy. On 

the other hand, current research validating forensic handwriting methods may not have come 

about if not for critics.

Electronic signatures and online signatures may reduce the number of documents FDEs are able 

to examine. But it may allow us to capture information we haven’t had before.

Not enough peer-review document publications in science journals, research requires time and 

money.

Handwriting analysis is discipline of older practitioners. As more labs are shut down, 

practitioners have fewer training and career opportunities.

FDEs need to know who to contact with research ideas/needs, but they have problems 

connecting with these people.

Many labs are dropping QD to get people into DNA.

What does a roadmap to achieve the future state look like?

Educating younger generation and selecting people with proper qualifications, such as degrees 

in science, statistics, computer science, and chemistry.

Further research on the correct statistical methods.

NIJ has been enthusiastically supportive of research so far.

Proficiency testing and peer review may be required for court and are very helpful.

Expand publications to peer-reviewed articles in top-tier journals, even beyond forensic science. 

This will build the discipline as a whole up. FDEs and researchers should look at the trajectory of 

the number of handwriting articles in peer-reviewed journals.
Page 4 of 5

Need more research done by researchers in collaboration with FDEs, not by FDEs who don’t 

have time and training (started already, last ~60 years depended on unfunded practitioners).

Need to define the current state of handwriting analysis, which was covered through the 

conference presentations.

Get away from case-problem-specific research, need to expand beyond tiny steps. Graduate 

students may be a resource for great (cheap!) research for those tiny steps if they are given 

ideas on practitioner needs. This strategy is ongoing at several institutions where students 

develop master’s theses on tiny, specific forensic research questions. 

Ensure that research and protocol development/review continues to proceed.

Need to entice younger students to get involved as FDEs.

Researchers and technology developers need a reasonable number of realistic document 

samples that reflect population to fine-tune and test technology.

Continue to collaborate with other disciplines and fields with other expertise to expand research 

benefits and to provide funding. Continue to fund collaboration through meetings.

Recognize value of tools that are available.

As DNA technology improved, DNA was used for additional crimes—to increase FDE workload,

expand use of technology into high-resolution electronic signatures and linguistic analysis 

(authorship v. writer-ship). Also, teach law enforcement and investigators about FDE 

capabilities.

Collect documents, papers at crime scenes.

Need a better way to integrate statisticians with forensic practice/crime lab; this is happening 

somewhat in bigger, better-funded labs.

Continue and increase interagency and public/private collaboration.

The American Statistical Association’s ad hoc advisory committee on forensic science can help 

connect FDEs to statisticians.

Casework data sets, which are confidentially stored with numbers replacing a person’s name 

and as individual characters instead of whole documents, may be able to be shared with 

researchers.

Additional comments submitted via email and Twitter

Speaking of DNA as a gold standard...we tend to have a problem in our laboratory of documents 

being submitted for latent print or DNA analysis, but not FDE work. Roadmap for the future 

might include more education between the disciplines of our capabilities.

Legitimate training opportunities are scarce. As a profession, we must actively pursue a remedy.

Future of document examination is a merger with computer forensics and digital imaging 

forensics.

Does the roadmap to the future include discussing these aspects at the ASQDE meeting in 

Indianapolis, Indiana, in August?

I think peer review and proficiency testing must be a mandatory aspect moving forward. The 

appropriate construction and extrapolation of proficiency testing to the real working 

environment will be the only way to access individual error rates and will be required by the 

court. Appropriately controlled peer review is demonstrable in reducing error rates and, I 

believe, will also be required for the court.
Page 5 of 5

Are there any legal barriers to entering handwriting from casework (either questioned or known 

documents) into any of the database systems or statistical models like there is in DNA (CODIS) 

and fingerprints (AFIS)?

It seems unlikely that detailed statistical calculations could be force-fit into examinations rather 

than form a foundation.

Law enforcement has reduced the criminal investigations involving forensic document evidence. 

Forgeries of wills and contracts are now relegated to civil courts for investigation and resolution. 

Citizens therefore must now pay out-of-pocket costs for private FDE work, with their tax money 

being applied to fund DNA and other disciplines in crime labs. This evolution in the future of FDE 

is abominable, but reality.

The verbal likelihood ratio doesn’t give an answer to authorship. During the presentations, I 

heard a lot of p-fallacies.

Does it seem reasonable that any FDE would ever actively incorporate statistical calculations 

into individual exam processes?

A preliminary study was presented a while ago on the topic of opinions and how they are 

received and perceived by jurors, i.e. how strongly they felt about what an expert stated and 

their understanding of the gravity of expert statements. Perhaps psychology should 

be considered when considering modifications to opinion scales.

Please let us all remember that nowadays when any FDE makes a call on genuineness or 

simulation or disguise, basically we FDEs have been actually mentally working/calculating on 

frequencies of observations of certain strokes, on number of occurrences of specific features, on 

geometrical outlines and areas of letters, on ratios between extenders and small letters, on 

variability of indicators etc. Ironically, nowadays the FDE calls are already made through 

mentally (that is very approximately) processed "statistical observations," but without an actual 

metrical evaluation of the parameters. So it seems quite reasonable to start and actually 

physically make all those calculations we usually reckon in our minds without carrying out any 

actual measuring.

Future: Effective demonstration material will always be essential to FDE testimony. Most people 

can see features in writing when they are highlighted and explained during FDE testimony.

However, FDEs must also work to be effective in explaining the basics of statistics related to 

study findings important to our field. Failure to have this minimum skillset can diminish the 

credibility of an FDE during an effective cross-examination. It certainly diminishes any claim FDEs

make to following a scientific methodology.

I have used these control charts to present statistics in court. They have been well received and 

understood when explained. This is a discriminator to determine statistically where the Q 

writing lies relative to the Ks. I take several measures. The charts are simply one example used in 

a case. 

I think the stats will only hurt our testimony. It will only help the defense (or other side) to make 

hay out of why you are not 100% positive. How can you explain why the stat is not 100%, which 

only leaves other avenues for the opposing counsel?