Extracting Handwritten Annotations from Printed Documents Via Infrared Scanning
Channel: Research Group Physical-Digital Affordances
Date: 2022-05-17

Despite ever improving digital ink and paper solutions, many people still prefer printing out documents for close reading, proofreading, or filling out forms.
However, in order to incorporate paper-based annotations into digital workflows, handwritten text and markings need to be extracted.
Common computer-vision and machine-learning approaches require extensive sets of training data or a clean digital version of the document.
We propose a simple method for extracting handwritten annotations from laser-printed documents using multispectral imaging.
While black toner absorbs infrared light, most inks are invisible in the infrared spectrum.
We modified an off-the-shelf flatbed scanner by adding a switchable infrared LED to its light guide.
By subtracting an infrared scan from a color scan, handwritten text and highlighting can be extracted and added to a PDF version.
Initial experiments show accurate results with high quality on a test data set of 93 annotated pages.
Thus, infrared scanning seems like a promising building block for integrating paper-based and digital annotation practices.



Andreas Schmid, Lorenz Heckelbacher, and Raphael Wimmer. 2022. Extracting Handwritten Annotations from Printed Documents Via Infrared Scanning. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA '22). Association for Computing Machinery, New York, NY, USA, Article 363, 1–6. https://doi.org/10.1145/3491101.3519872


For more information on the project, feel free to contact us or visit our project website https://hci.ur.de/projects/infrared_scan