Digitizing 125 Years of Images - Suman Roy & William P. Davis
Channel: All Things Open
Date: 2020-12-01

Presented by: Suman Roy & William P. Davis, The New York Times
Presented at All Things Open 2020 - Case Study/Demo Track

Abstract: Photographs first appeared in The New York Times on Sept. 6, 1896 and have been an integral part of our journalism ever since. The New York Times has a vast archive of physical photos stored in hundreds of file cabinets in the “morgue”, organized by folders. This archive has significant historical value, some of these photos can be found nowhere else in the world.

The Times began a project in 2018 to digitize these images in order to preserve them and make the archive more easily searchable and researchable. An index card catalogue was the only way to research the archive. The backs of the images and the folders containing the images have rich information that were not indexed in any form.

The talk will review how we build a system to ingest millions of photos, and make them findable Cloud Storage, Pub/Sub, Kubernetes, Postgres, Golang, Imagemagick and a host of other technologies.

We will also touch on how we built search, which has been a fascinating experience because of the difficulties around OCRing hand-written text and dates.