Vital Records: Uncover the past from historical handwritten records (2020)

Abstract: We present Vital Records, a demonstrator based on deep-learning approaches to handwritten-text recognition, table processing and information extraction, which enables data from century-old documents to be parsed and analysed, making it possible to explore death records in space and time. This demonstrator provides a user interface for browsing and visualising data extracted from 80,000 handwritten pages of tabular data.

## introduction a great deal of human history is detailed in hand-written documents that have yet to be analyzed. extracting information from such documents has, until recently, represented an extremely time-consuming and labour-intensive task. for this reason, there remains a treasure trove of untapped information that could help provide insight into, for example, the impact of industrialization on populations. indeed, some of these records-such as those documenting epidemics-may provide useful context for our understanding of modern problems. the recent advances in handwritten text recognition @xcite , and in deep learning at large, allow now us to automatically process large volumes of documents. the vital records demonstrator, based on a collection of 80,000 pages, brings to life records that were handwritten by more than 700 different priests from 200 parishes in germany from 1848 to 1878. the demo illustrates how state-of-the-art deep-learning methods-handwritten text recognition (htr), table recognition (tr), and information extraction (ie)-can be used to transform these records into a digital format that can be queried and visualized in different ways to enrich our knowledge from previously unexplored sources of information. ## using vital records to trace trends through history vital records allows users to browse and visualize the dataset extracted from these german records using spatio-temporal criteria, in addition to the usual textual search queries (figure 1 ). death records provide a range of useful information. as well as the name, age, date, and cause of death, the records include the profession of the deceased (figure 3 ). with this information in hand, we are able to visualize trends over a given period. for example, we used these death records to trace the evolution of professions between 1847 and 1877. the resulting graph, shown in figure 4 , shows the number of deaths recorded for which the field of profession (stand) contains weaver (weber), shoemaker (schumacher), and miller (müller). the graph may be considered a proxy for estimating the evolution of these professions during this period. it's possible, too, to spot locally relevant characteristics. for example, data extracted from these records shows a high number of glassmakers (glasmacher) around zwiesel, a region that remains well-known for its production of glassware to this day. in addition to the visualization of data using graphs, vital records makes possible the development of temporal animations for which the timestep can be adjusted by year, month, or day. by combining both spatial and temporal information in this way, we are able to visualize the evolution of a given query over space and time. our favourite query is the spread of scarlet fever in 1871. it's also possible to track specific events, such as the opening of a hospital in a parish, or the arrival of a train line. searching for professions containing the string bahn will provide you with a map of the three bahnstrecke, or train lines: passau-regensburg, the bavarian forest railway, and the münchen-simbach. 3 deep-learning and recognition: the technology behind vital records ## future work we have almost finished processing the birth and wedding records, and will be adding them to the demo soon, along with the ability to search across the three types of records. we hope that one outcome of this upgrade will be the automatic generation of family trees. some issues in terms of information extraction are still pending and seem to require a specific step to solve them: for instance the idem/ditto sign occurring regularly in columns in order to avoid to repeat the same value (figure 3 , last column). additionally, a major milestone will be to infer demographic information from these data, such as the population size, and some demographic statistics (birth and death rate, for instance). this can only be computed/estimated when the three types of records are linked, enabling identification of the same person across the three records @xcite . ## conclusion in summary, vital records uses information extracted (via htr and tr machine-learning technology) from archival documents to enable users to visualize trends through history, and even to track specific events. the demonstrator that we have developed, based on data obtained from german parish records between 1848 and 1878, showcases the spatio-temporal capabilities of the demonstrator.