Extracting Relations from Ecclesiastical Cultural Heritage Texts (2024)

Abstract: Motivated by the increasing volume of data and the necessity of getting valuable insights, this research describes the process of extracting entities and relations from Italian texts in the context of ecclesiastical cultural heritage data. Named Entity Recognition (NER) and Relation Extraction (RE) are paramount tasks in Natural Language Processing. This paper presents a traditional methodology based on a two-step procedure: firstly, a custom model for Named Entity Recognition extracts entities from data, and then, a multi-input neural network model is trained to perform Relation Classification as a multi-label classification problem. Data are provided by IDS&Unitelm (technological partner of the IT Services and National Office for Ecclesiastical Cultural Heritage and Religious Buildings of CEI, the Italian Episcopal Conference) and concerns biographical texts of 9,982 entities of type person, which can be accessed by the online portal BeWeb. This approach aims to enhance the organization and accessibility of ecclesiastical cultural heritage data, offering deeper insights into historical biographical records.

## introduction in the current landscape, the abundance of data is unprecedented; technological advancements, the internet of things (iot), increasing connectivity, and digitalization are some factors that led to today's scenario. yet, whereas methods of data collection continue to expand, the true value of this phenomenon is not related to the mere accumulation of information, but to the acquisition of meaningful insights. for this reason, there is a major focus on applying innovative techniques to all possible domains; the cultural heritage environment is also experimenting with big attention to exploit new possibilities. in the post-covid era, the national recovery and resilience plan (nrrp) has funded numerous endeavors to underline the impor-tance of preserving and exploring cultural heritage, signaling a great moment for the combination of innovative approaches with domains traditionally less associated with these topics. this study positions itself among those initiatives, aiming to bridge this gap by employing advanced methodologies within the cultural heritage landscape. at the core of this pursuit, a paramount role is played by knowledge graphs, tools based on the concept of knowledge bases. knowledge graphs enable us to navigate the complex and intricate depths of data; for this reason, this work aims at finding the basic components of a knowledge graph, by extracting entities and relations from texts. the proposed methodology uses a traditional approach structured as a two-step process: first named entity recognition (ner), key for pinpointing names, locations, and other text elements, and later sentence-level relation extraction, implemented as a multi-class classification task. a multi-input neural network model is built to leverage a labeled dataset of sentences and entity types. in natural language processing, extracting semantic relationships from text is a very crucial task. this process concerns converting unstructured data (text) into structured. relation extraction (re) can be achieved in several ways; one of them consists of setting the problem as a classification task: relation classification (rc) @xcite , @xcite . relation classification has been approached using pattern-based @xcite , @xcite , or kernel-based methods @xcite . early approaches make use of pipelines that identify entities and then classify relations between pairs using cnns or lstms to capture sentence-level semantics @xcite , @xcite . moreover, current methods for sentence-level relation extraction employ transformer models @xcite like bert @xcite . in the context of art, especially in the last years, increasing importance is reserved for applications of digital tools mainly focused on art discovery or recommendation mechanisms @xcite . in @xcite the authors describe several techniques (such as entity recognition and linking, coreference resolution, time extraction, and artwork extraction) applied to vasari's most important piece "the lives of the artists". in @xcite , instead, nlp techniques have been applied to biographical texts of artists to achieve sentence-pair binary classification, hence connections among artists, without considering the type of the relations, conversely to what is developed in this study. the paper is organized as follows: sections 2 and 3 respectively illustrate the data and the methodology that were employed, starting from the formulation of the problem and continuing with the construction of models, and their evaluation (section 4). the last part shows conclusions and potential future applications. ## data data for this research were provided by the partner company foot_0 and can be accessed through an online website: beweb ( https://beweb. chiesacattolica.it/ ). the portal allows seamless exploration of diverse databases acquired through the joint efforts of unbce foot_1 and the italian dioceses @xcite . this collaborative initiative began in 1996 and has resulted in a census of diocesan and ecclesiastical assets, including archives, libraries, and museums. throughout the years they were able to assemble a database that comprises over 5 million records, including 4 million historical and artistic assets, 66,000 places of worship, 1.5 million library assets, 6,800 archival collections, and 1,588 cultural institutions @xcite . thanks to the help of approximately 3,000 experts, the projects are constantly reviewed and updated. upon validation, the data is integrated into the national database; beweb, therefore, shows data validated from several inventories and facilitates cross-domain navigation of databases making use of the specific descriptive standards for each sector. the project's development involves two main elements: a dataset derived from sector-specific descriptive standards and a clustering system that interconnects terms referring to the same entity across various catalog databases, resulting in clusters identified by cross-domain aggregates: cei foot_2 authority file. hence, the cei authority file (af-cei) can be seen as a centralized repository that integrates authority records from diverse cataloging domains through a clustering mechanism. each domain interacts with the af-cei to establish, associate, or revise clusters. these resulting clusters undergo enrichment by designated reviewers, incorporating additional elements such as alternative nomenclature, biographical and historical annotations, images representing the entity, where present, interrelationships with other af-cei, and potential references to external web resources @xcite . this research focuses on the biographical notes of 9,982 cei authority files categorized as person. ## evaluation of relationship classification to ensure the model grasps relationships effectively without merely memorizing the training data, the evaluation is carried out in two distinct ways. first, evaluation metrics are observed for both training and validation sets during the training process. the validation set is created using 30% of the training data and is used to assess how well the model generalizes to unseen examples. this step aids in identifying any signs of overfitting, where the model might excessively adapt to the training data and struggle to perform well on new data. by validating the model's performance on this subset, we ensure it can make accurate predictions beyond the examples it was trained on. initially, the model starts with a loss of 0.9039 and an accuracy of approximately 61.8%, however, as training proceeds, the model shows notable improvements, with decreasing loss values and increasing accuracy, as well as balanced metrics such as recall, precision, and f1-score. these metrics exhibit positive trajectories across different classes. the model is further evaluated on a new subset composed of 250 manually annotated sentences. out of these 250 sentences, 29 are wrongly predicted, representing an error rate of 11.6%. ## results and conclusion the evaluation of the ner model demonstrates its growing capacity to accurately recognize entities within the provided text data, suggesting that the figure 6 shows that the model excelled in classifying "work/study" and "ecclesiastical title," achieving high true positive counts of 111 and 60, respectively. however, there are notable misclassifications, particularly in the "birth/death/travel" and "kinship" categories. these issues may be attributed to the class imbalances highlighted in figure 5 , suggesting areas for improvement in distinguishing these categories. finally, the classification of 65,289 relations within the complete dataset reveals a meaningful distribution of relationships, with "work/study" as the most frequent and "kinship" as the least common. these findings not only underline the model's ability to accurately recognize and categorize different relationships within the text data, but also align with the patterns that are evident from simply reading the texts, suggesting that the model is well-optimized for the task at hand. cultural heritage has a huge role in shaping national and cultural identity, promoting tourism, and attracting visitors; on the other hand, digital technology enables cultural institutions to offer global access to their collections, including items rarely exhibited due to space constraints or fragility @xcite . therefore, in the era of rapid digital transformation where the volume of data has surged dramatically, the challenge is not merely increasing the number of information but transforming it into valuable insights. whereas digitalization has initiated a new era, the true potential of data can only be unlocked through innovative methodologies that transcend conventional approaches and offer new opportunities for exploration. within this context, the endeavor to employ new tools holds profound significance, because whereas traditional methodologies, reliant on manual studies of literature and documents by academics, have laid the groundwork, they may not have fully revealed all the potentiality of knowledge. this work wants to be an opportunity to bridge this gap, leveraging nat-ural language methodologies to delve deeper into these big collections of data. the present study proposes a two-step procedure to extract entities and relations from text data concerning ecclesiastical cultural heritage. a custom pipeline for named entity recognition and a multi-classification model are exploited with manually annotated data, to extract and classify 65,289 relations. a differentiated evaluation process is performed to assess the significance of the proposed methodology. for future work, we intend to train the model using a larger corpus of annotated data to enhance the classification performance. finally, the ultimate goal of this study is to transform unstructured text into structured data to create a graph database, specifically a knowledge graph. this knowledge graph will serve as a valuable tool for operators studying extensive texts, enabling them to uncover new connections that may not have previously emerged. additionally, the extracted information will be integrated into a recommender system designed to assist users on the beweb platform. beweb is a website that provides access to a vast amount of data concerning ecclesiastical cultural heritage. by offering personalized suggestions, the recommender system aims to enhance user engagement and foster the creation of communities around shared interests.