###### Abstract  
This paper deals with innovative fruition modalities of cultural heritage sites. Based on two ongoing experiments, four pillars are considered, that is, _User Localization_, _Multimodal Interaction_, _User Understanding_ and _Gamification_. A survey of the existing literature regarding one or more issues related to the four pillars is proposed. It aims to put in evidence the exploitation of these contributions to cultural heritage. It is discussed how a cultural site can be enriched, extended and transformed into an intelligent multimodal environment in this perspective. This new augmented environment can focus on the visitor, analyze his activity and behavior, and make his experience more satisfying, fulfilling and unique. After an in-depth overview of the existing technologies and methodologies for the fruition of cultural interest sites, the two experiments are described in detail and the authors' vision of the future is proposed. cultural heritage; localization; multimodal interaction; augmented reality; affective computing; gamification; cognitive architectures +
Footnote †: journal: _Future Internet_  
The possibility of using wearable and not invasive devices allows for acquiring data of interest, like the location, the current routes, and the visitor's visual attention. Among new facilities, Virtual Reality (VR), Augmented Reality (AR) and natural language interaction systems allow for enriching and extending the visiting experience [2]. Even if the facilities are tied to technologies, both VR and AR can be defined, avoiding specifying particular hardware as in [3] where VR is defined \"a real or simulated environment in which a perceiver experiences telepresence\", and in [4] where AR is defined as a technique to show extra information over the real world, respectively. Between VR and AR, there is a \"virtual continuum\" where mixed reality encompasses everything between reality and a virtual environment [5]. VR and AR have been attracting researchers' interest for several years; however, since their spread depends on several factors, such as technologies, society's needs and applications, the research trend showed peaks and stagnations [6]. On the other hand, these technologies and related applications play a crucial role in widening the potential market. It can be noted that AR and VR were not devised for cultural heritage, in fact, starting from games applications, both VR and AR have been driven by the health sector, education and industry while technologies benefit from the increased computing capacity and the lowering of the devices [7], particularly smartphones [8]. Recently VR and AR, and related technologies have also been adopted in cultural heritage to improve the site's experiences. Nowadays, the wide availability of smart devices with both affordability and high computational power, makes their application very widespread, and relatively inexpensive. Moreover, the use of intelligent data analysis and artificial intelligence, to process the acquired information, makes it possible to effectively profile the visitor, who can consequently receive personalized and non-trivial suggestions and recommendations to continue the visit. These advancements make it possible not only to focus on the visitor but also to transform the cultural site into an intelligent multimodal environment, emphasizing the emotional aspects of the interaction. In our vision, we conceive the cultural heritage site as an intelligent entity embodied in a multimodal interactive environment. This entity can perceive human beings' activities and their feelings, reacting accordingly by talking or by giving sounds, images, and colors as feedback. Such an entity's behavior should be driven by the desire of establishing a social, emotional, engaging connection with the visitors while supporting them in the site fruition.