The CHROME Manifesto: Integrating Multimodal Data into Cultural Heritage Resources
Authors: Francesco Cutugno, Felice Dell'Orletta, Isabella Poggi, Renata Savy, Antonio Sorgente (Università di Napoli Federico II, CNR-ILC, Università Roma Tre, Università di Salerno, CNR-ISASI, 2018)
Topic: Lexical and Semantic Resources and Analysis

Abstract
The CHROME Project aims at English. The CHROME Project aims at collecting a wide portfolio of digital resources oriented to technological application in Cultural Heritage (henceforth CH). The contributions for the realisation of such objective come from the efforts of computer scientists, psychologists, architects, and computational linguists, who constitute an interdisciplinary equipe. We are collecting and analyzing texts, spoken materials, architectural surveys, and human motion videos, attempting the integration of these data in a multidimensional platform based on multilevel annotation systems, game engines importing, and virtualization techniques. As case of study we choose to work on the magic travel along three Charterhouses located in Campania region: S. Martino in Naples, S. Lorenzo in Padula (Salerno) and S. Giacomo, in Capri.

Introduction
Introduction Introduction The CHROME project was born with the intention of creating a framework and methodology to collect, represent and analyze cultural heritage contents and present them through artificial agents whose behavior is inspired by accurate analysis of expert guides, museum curators and tour operators. These gatekeepers are those professional figures possessing a significant amount of knowledge concerning how people should be guided in the exploration of cultural contents. In this sense, they act as mediators between cultural heritage and visitors by using a set of communication strategies, both verbal and non-verbal, aimed at maintaining a high level of engagement and delivering high-quality content. The overall experience of accessing cultural heritage is greatly enriched by these professional figures: their knowledge and experience, therefore, should not be overlooked when designing artificial agents oriented to cultural heritage presentation. As this knowledge is primarily based on experience collected on the field, the CHROME project aims at recording the performance of gatekeepers in a sensible environment so that formal analysis of their behavior can be documented and studied. The result of this process (see Fig. 1), conducted jointly by humanities and computer scientists, will lead to the formalization of a model describing the behaviors adopted by gatekeepers when presenting cultural heritage. This will then be used to control a humanoid robot designed to follow similar presentation strategies. Taking in account this aim, the main goals of the project are to: collect and provide the scientific community with reference datasets to study human-human interaction during the presentation of cultural heritage by professionals; investigate the structure of the texts contained in the collected corpus in order to produce automatic approaches supporting text generation for oral presentations in cultural heritage domain; provide a reference computational model to support development of artificial agents exhibiting coherent and engaging behavioural strategies. In addition to the orality degree of the assembled presentations, special attention will be attributed to non-verbal aspects. Specifically, CHROME will concentrate on enriching the presentation with consistent prosody and gestures. Finally, another goal is to evaluate the impact of these agents in simplifying access to cultural heritage and attract visitors in cultural sites. For the realization of such goals, five research groups are involved in the CHROME projects covering different scientific and humanistic disciplines that complement each other. The equipe is highly interdisciplinary and is formed of linguists (with specific competences in prosody, pragmatics, paralinguistics, and non-verbal behavior analysis), computational linguists and computer scientists (with skills in Artificial Intelligence and Human Machine Interaction) The teams involved in the project are: • UrbanEco (Naples – Federico II) an interdisciplinary team formed by computer scientists, architects, linguists, aiming at collecting 3D architectural surveys and speech and gesture corpora. UrbanEco is also designing multimodal interaction systems; sub-partner linked to this unit is the “Polo Museale della Campania - MiBaCT” the local section of the Italian Cultural Ministry managing more than 30 museums in our region; • ILC (Pisa – CNR) will develop systems for automatically extracting and organizing linguistic and domain knowledge from domainspecific corpora; • UniSa (University of Salerno) will analyze texts and will afford the theme of prosodic analysis of spoken material finalized at speech synthesis issue; • ISASI (Pozzuoli, CNR) will afford the challenge of CH question answering and language generation for the realization of interaction models in natural language; • RomaTre (Roma, University RomaTre), will confront the theme of multimodal communication and gesture analysis. As case of study we choose to work on the magic travel along three Charterhouses located in Campania region: S. Martino in Naples, S. Lorenzo in Padula (Salerno) and S. Giacomo, in Capri. All the texts, the architectural surveys and the audio-video recordings, in other words, all the digital resources that we have and will collect and that we describe in the next sections, concern with these wonderful sites.

Conclusion
CHROME aims at formalizing data collection and annotation paradigms for architectural heritage, in particular the annotation regards texts, video, audio and gestures. From the annotated data, we will: i) perform correlation analysis to identify cross-domain patterns and link them to communicative goals; ii) describe how an expert presenter relates to the physical environment while she describing it; iii) identify which communicative strategies can be mimicked by an artificial agent with the available technology. Possible domains of simulation will the deictic and iconic gestures, face and gaze behaviour; iv) implement a final demonstrator adopting the formalized strategies to generate dynamic presentations for the attending visitors.