Improving workflows in digital art history: the usefulness of patrimonial images segmentation
Creators: name: Maronet, Léa
name: Truc, Alice
Publisher: name: Zenodo
Year: 2024
Subjects: subject: Art history
subject: History and archaeology
subject: FOS: History and archaeology
subject: Digital humanities

description: Since Johanna Drucker questioned the existence of “digital art history” (Drucker, 2013), the last decade has seen a growing number of art historical projects make use of computer vision methods. However, digital art history remains a fragmented field: until now, these projects prefer to develop their own technical and methodological solutions. This project-based logic prevents the reproducibility of workflows and, thus, requires a significant financial and time investment, large quantities of data and technical skills that are not accessible to all research units, let alone all researchers (Romein et al., 2020, 310).

This logic ultimately hampers the standardization of digital practices, particularly in terms of data recording and algorithm training. Particularly, segmentation of patrimonial images is not subject to any standard or harmonization. It's not a question of trying to standardize the description of image content - an attempt that falls within the realm of ontology - but proposing a way to harmonize the recording of objects of interest within these images. If initiatives are being produced in the field of literary studies (Chagué, Clérice & Romary, 2021), in digital art history, the question remains little addressed (Bardiot, 2021). Yet having corpora of segmented images, whose objects have been identified by their coordinates and recorded in a hierarchical and standardized way, would make it possible to create ground truths that would serve as basis for training new algorithms and be profitable for every workflow in art history.

As workflows in digital art history are generally based on the technique of transfer learning, which reduce the amount of data required to train algorithms and improve the results obtained, this question is crucial in the discipline. However, the algorithms available are for the most part trained on “natural” images, which means they are hardly relevant to the specificity of patriomonial images. The absence of harmonization in the solutions proposed project after project culminates in a bitter realization: the tools available swiftly evolve, rendering them obsolete in the short term. Therefore, it appears vital today, in the age of Open Science, to pool our segmented images so they can serve as ground truths for future research, minimizing model training while enhancing the results obtained.

Segmentation always proposes an interpretation of the images. As such, how can we harmonize data segmentation, while remaining relevant to the epistemological requirements of art history? Asking this question implies putting into tension issues of interoperability and reproducibility, on the one hand, and epistemological reasoning in relation to the tools developed, on the other hand. These questions should not be addressed after tools development but upstream and should accompany the entire workflow (Stutzmann, 2010, 247-278). To address these tensions, the specific needs of digital art history about questions of image segmentation have to be identified, especially in the case of small and limited corpora, and existing standards in other fields will be discussed. The aim is twofold: to produce ready-to-use models and workflows adapted to patrimonial data, and to share data enabling these models to be produced.