# A CRITS Foray Into Cultural Heritage: Background Characters For The SHELeadersVR Project

J-B Culié, Bojan Mijatovic, David Panzoli, Davud Nesimovic, Stephane Sanchez, Selma Rizvic

► To cite this version:

J-B Culié, Bojan Mijatovic, David Panzoli, Davud Nesimovic, Stephane Sanchez, et al.. A CRITS Foray Into Cultural Heritage: Background Characters For The SHELeadersVR Project. Creating Lively Interactive Populated Environments Workshop (CLIPE 2024), Apr 2024, Limassol, Cyprus. 10.2312/cl.20241046 . hal-05279593

HAL Id: hal-05279593

https://hal.science/hal-05279593v1

Submitted on 23 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# A CRITS Foray Into Cultural Heritage : Background Characters For The SHELeadersVR Project

J-B. Culié<fnref n="1" />,<fnref n="2" />, B. Mijatovic<fnref n="3" /> ID, D. Panzoli<fnref n="1" />,<fnref n="2" /> ID, D. Nesimovic<fnref n="3" />, S. Sanchez<fnref n="4" />,<fnref n="2" /> ID and S. Rizvic<fnref n="5" /> ID

## Abstract

This article presents CRITS, a software framework designed to enhance virtual environments, particularly in the context of cultural heritage and immersive learning simulations. CRITS enables the easy integration of autonomous, human-like characters into virtual settings, enriching the user's experience by simulating the dynamic activities and social presence of background characters. The framework is showcased through its application in the SHELeaders VR project, which aims to recreate historical settings and narratives centered around medieval female leaders in the Balkans. The article discusses the technical implementation of CRITS, its benefits for creating lively and populated environments, and reflects on potential improvements and future research directions.

## 1. Introduction

Digital twins and virtual environments have the potential to host interactive and compelling experiences in desktop 3D or fully immersive virtual reality (VR). In most contexts, spanning from cultural heritage explorations to training scenarios in technical maintenance, the social dimension is an essential component in the full rendering of a place and the activities that take place in its midst. Populating these environments therefore adds great value, although it entails specific computer skills far beyond those specifically required for storytelling and 3d modelling. The techniques that allow human users to interact with digital characters are diverse and the large variety of goals and situations has so far prevented a unique and universal approach to prevail. Conceptual frameworks, like the Levels of Interactions framework (LoI) [PQD*10], illustrated in figure 1, have guided the design of such characters in the past years, offering simplifying principles that make it easier to overcome complexity.

In this article, we focus on the background characters that bring a virtual environment to life as they come and go between the different areas, attractions or work stations. Although they play a minor role in the cultural transmission associated with the places they inhabit, those agents are nonetheless essential for user's immersion, as they induce a feeling of social presence [KMG22]. The seminal work pertaining to populating a virtual space with a crowd can be traced back to the late 1990s. In [ST07] a crowd of autonomous pedestrians walk around in a recreation of the 1910 original Pennsylvania Station in New York City that was demolished

in 1963. Virtual Romans in [MHY*07] recreate typical daily activities of citizens in lost city of Pompeii. Game AI has also been a recurring inspiration for modelling the behaviours of non-playing characters in a virtual environment. Goal-Oriented Action Planning (GOAP) [Ork03] is a not-so-recent AI technique yet still unparalleled in video games AI where non-playing characters are expected to cope with uncertainty and demonstrate variability in their behaviours. Agents evaluate potential actions based on their ability to achieve desired goals and select the most promising course of action through heuristic search. GOAP offers stunning results al-

<figure><img src="image_2.png" /><figcaption>**Figure 1:** The Levels of Interactions framework categorise the roles of non-playing characters in a virtual environment by defining the instructional opportunities of each type of interaction between the user/learner and a virtual character.</figcaption></figure>

<footnote n="1">¹INU Jean-François Champollion, F-81000 Albi</footnote>

<footnote n="2">²IRIT UMR 5505, F-31400 Toulouse, France</footnote>

<footnote n="3">³Sarajevo School of Science and Technology</footnote>

<footnote n="4">⁴University of Toulouse Capitole, F-31200 Toulouse, France</footnote>

<footnote n="5">⁵University of Sarajevo, Faculty of Electrical Engineering</footnote>

<footnote>© 2024 The Authors.
Proceedings published by Eurographics - The European Association for Computer Graphics.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.</footnote>
though it is a resource-intensive technique that can only be jus-
tified if AI is a crucial part of the game design. In addition, be-
havioural unpredictability is an asset in adversarial video games,
but the lack of control over the NPCs makes it a rather irrele-
vant choice in an immersive training simulation. Behavior trees
(BT) [MH17] are a hierarchical AI technique which is far more
commonly used in game development for decision-making. The
technique is so widespread that it has been integrated into com-
mon video game engines such as unity3d or Unreal Engine. BTs
represent a set of tasks and decisions arranged in a tree structure,
where each node corresponds to a behavior or a condition. The tree
structure allows for clear organization of behaviors and prioritiza-
tion of actions based on conditions, enabling flexible and dynamic
decision-making in complex environments. Nodes can include ac-
tions, conditions, sequences, selectors, and decorators, offering a
versatile framework for modeling diverse behaviors and strategies.
BTs must be manually defined by expert programmers. Although
the task is relatively straightforward for simple rules, programming
can become more challenging and time-consuming with more com-
plex behaviours with intricate decision logic. Finally, crowd mod-
elling techniques for evacuation simulation [ARC13] or urban plan-
ning are known to demonstrate ultra-realistic crowd dynamics, with
hundreds of characters navigating in open spaces or narrow areas.
The equations used accurately describe the flow of people, but are
not sufficient to represent the human activity of a place in a suffi-
ciently realistic way.

The CRITS software is intended to provide a suitable response to the need for designers of immersive learning simulations or serious games about cultural heritage to populate their virtual environments with autonomous characters who give a sense of the ongoing activities inside. To do this, it draws on existing work to propose a method that is easy to implement in a game engine, and runs with a small CPU and memory footprint. The research presented in this article describes the integration of CRITS in such an environment as part of the SHELeaders VR project. We discuss the benefits of the CRITS approach and reflect on its shortcomings and possible corresponding improvements.

## 2. The SHELeaders VR project

Creative Europe SHELeadersVR project [DIG24] aims to recreate the lives and locations of five medieval female rulers from the Balkans. It was not common in medieval times for women to rule. They were mainly performing the roles of kings' wives and had no executive powers. However, in the SHELeadersVR project the domain experts identified five actual queens of Bosnia, Serbia, Montenegro, Albania, and North Macedonia. Through Virtual and Augmented Reality applications the visitors of partner museums and selected historical locations will learn about those queens and their significance.

The VR application is designed as an educational game. It uses Advanced Interactive Digital Storytelling (A-IDS) [RBM23] methodology to convey historical information in an amusing and educational way to all generations of users. The game consists of VR video stories and 3D scenes of reconstructed castles and settlements where the queens used to live. The queens are presented through live actress performances recorded on green screen and

added into 3D reconstructions. The user should watch those digital
stories and look for artifacts related to the narratives. The artifacts
are digitized museum exhibits that belonged to the queens.

The game starts in a virtual scene resembling a medieval castle main hall (Figure 2). The user is surrounded by doors leading to five Balkans countries. After opening each door he/she is placed in the middle of a digital story. When the story ends, the user should find the artifacts in an interactive 3D scene. Non-player characters (NPCs) are asking questions and helping in the quest. After collecting artifacts from all five countries, the sixth door opens and leads into the treasury, where all digital models can be explored.

<figure><img src="image_3.png" /><figcaption>**Figure 3: Albanian queen Vojsava in front of the 3D reconstructed basilica in Kruja**</figcaption></figure>

Most of the gameplays are happening in interior spaces, pop-
ulated by a small number of NPCs. However, in Albania we im-
plemented a part of the gameplay in the exterior of the 3D recon-
structed basilica in Kruja (Figure 3). Below the basilica, a village
is located and it is visible from the user perspective (Figure 4). We
needed to populate that village with CRITS to increase the feeling
of immersion. This is the list of requirements they were supposed
to perform:

* Mow/Harvest grass to a wheelbarrow.

* Once the wheelbarrow is full, an NPC empties it into a defined place.

* Cut trees and put the wooden logs in the pile of wood

<figure><img src="image_4.png" /><figcaption>**Figure 2: SHELeadersVR application.** *The main hall with doors leading to five countries and the treasury*</figcaption></figure>
* Carry water from a well to water containers

* Sell and buy objects

* Drink from water containers

* Eat food

* Use bench

* Talk between NPC

* Walk around the village and/or watch objects

### 3. Crowds for Immersive Training Simulations

CRITS [CSP23] (CRowds for Immersive Training Simulations) is a framework providing generic virtual crowd for immersive environments. It is designed to enhance immersive training environments by integrating virtual humanoid agents. These agents populate the scene, adding life and a sense of ongoing activity, which enhances the realism and immersion for users. The framework's foundation on informed environments and smart-objects ensures its seamless integration and flexibility across a diverse range of training scenarios, from crisis management exercises to learning factories. CRITS focuses on background characters that, while not central to the training's pedagogical goals, contribute significantly to the training environment's authenticity and dynamism (figure 5).

Expanding on the application of CRITS in cultural heritage virtual environments, the framework's ability to provide agents with specific thematic roles adds a significant depth to the user experience. By representing historical figures, local inhabitants, or other pertinent entities, these agents engage in culturally and contextually appropriate behaviors and interactions, thereby offering a more nuanced and comprehensive exploration of the cultural heritage in question. This approach not only enriches the user's engagement with the content but also fosters a deeper appreciation and understanding of the cultural nuances being presented. Consequently, CRITS transforms the virtual environment into a more immersive, informative, and captivating educational platform, bridging the gap between mere observation and interactive learning.

<figure><img src="image_5.png" /><figcaption>**Figure 4:** *The village with CRITS characters walking, as seen in first person from the promontory next to the basilica.*</figcaption></figure>

The decision-making process within CRITS is a component that ensures the autonomous and dynamic behavior of virtual agents within immersive training simulations. This process is based on a combination of Finite State Machines (FSM) and a set of decision

<figure><img src="image_6.png" /><figcaption>**Figure 5:** *CRITS displaying various roles such as farmers, sellers, carriers, customers...*</figcaption></figure>

rules tailored for each state, enabling agents to adapt their actions according to the evolving circumstances of the environment.

Each agent in CRITS is initially assigned a list of tasks to perform, alongside a set of needs and attributes that influence their decision-making. The tasks, needs, and attributes are either randomly selected or predefined, contributing to the diversity of agent behaviors. The environment is populated with smart objects that not only serve as interactive elements but also broadcast "affordances" — embedded semantic properties that define the actions they make possible in the environment. These affordances guide the agents in selecting which interactions (figure 6) to prioritize according to their tasks, needs and personal attributes.

Technically-wise, CRITS currently works with Unity3D, as part of several scripts and resources that can be transplanted in any project, like the SHELeaders VR project for that matter. The implementation of CRITS into SHELeadersVR application was a smooth and friction-less experience. The first step was to import the CRITS folder that contained all the important prefabs and scripts into the SHELeadersVR project. Once that task completed, the next step was to set up all the necessary components of CRITS in the scene. Next, the managers prefab were placed into the scene. These prefabs controls the number of agents spawned (Game Manager) alongside controlling spawn points ( spawn Manager). After adding the managers, the navigation mesh which determine the area where the agents are allowed to move around was baked. To do so, the CRITS developed Tool for baking was used and no problem were encountered.

That was all that was needed to have agents spawn in and roam around. Yet, certain specific jobs needed to be added, such as carrying water, carrying hay, cutting grass, etc. This significantly increased the dynamic and immersion due to agents now being able
<figure><img src="image_7.png" /><figcaption>**Figure 6:** Purple lines represent the communication between an agent and an object. Green lines show the object selected by the agent once the decision making process is achieved.</figcaption></figure>

to perform different and more relevant activities with respect to the context of the game.

## 4. Discussion

As a preliminary qualitative evaluation, the game was tested on a Meta quest 3, tethered to an Intel Core i7-8700 CPU and an RTX 2080 GPU. A group of researchers, both novice and experienced in virtual reality, participated in the educational VR experience to provide relevant feedback. The application uses multiple VR interactions such as grabbing and pulling objects, talking and giving objects to NPCs. These interactions add another layer of depth to the VR experience, enhancing the educational aspect of the game. By allowing two different ways to move in the 3D space (With joy-stick or teleportation) and using smooth transitions when switching scenes, no motion sickness was reported from the users.

Regarding the integration of CRITS, SHELeadersVR's developers managed to quickly and easily add new jobs related to the cultural theme. The two male and female characters 3D models which were packaged as part of CRITS default implementation were utilized with only little adjustment on the material so that the virtual inhabitants are clearly distinguishable from afar. A welcome improvement would consider replacing these standard characters with more accurate depictions of people of the time. In addition, CRITS characters have been integrated into the village so far, but it would be an improvement to add more characters to the outskirts of the basilica among the static figures already in place and closer to the player's zone of play. With more detailed 3D models and bespoke animations, CRITS would be able to go beyond the initial framework of the living background and move on to a higher level of interaction (see LOI) by offering the player the chance to see animations that authentically recreate the activities of the period. Such a functionality is already provided for in the CRITS framework and would not require too much work, apart from the 3D modelling of the characters and associated animations, which remains a major undertaking.

## 5. Conclusion

CRITS is an application that was designed for easily populating an existing environment with human-like characters that would mimic the daily activities of inhabitants. To that end, the SHELeadersVR project designers tasked CRITS with several occupations and activities to reproduce, building on existing areas and 3d models of stalls, workshops or tools. Apart from a few minor adjustments, the integration of CRITS was straightforward, rapid and successful. A preliminary qualitative analysis carried out by researchers (involved in the development of CRITS) established that the presence of characters in the background was an undeniable added value to the exploration of the environment, as a background to the main activities that make up the primary narrative of the project. Future steps of the collaboration will involve a quantitative study involving a more representative audience of users targeted by the cultural heritage application, and based on the collection of data more indicative of the usual measures of presence or immersion.

Immersive training and cultural heritage are two prolific disciplines, with many digital twins or 3d reconstructions crafted each year by expert modelers and artists. Populating these environments adds value to the experiences offered, increasing learner engagement in training courses or immersing visitors even more in historical re-enactments.

## References

[ARC13] ALMEIDA J. E., ROSSETI R. J., COELHO A. L.: Crowd simulation modeling applied to emergency and evacuation simulations using multi-agent systems. *arXiv preprint arXiv:1303.4692* (2013). 2

[CSP23] CULIÉ J., SANCHEZ S., PANZOLI D.: Effortlessly populating immersive training simulations with background characters. In *Extended Reality - International Conference, XR Salento 2023, Lecce, Italy, September 6-9, 2023, Proceedings, Part I* (2023), Paolis L. T. D., Arpaia P., Sacco M., (Eds.), vol. 14218 of *Lecture Notes in Computer Science*, Springer, pp. 172–183. 3

[DIG24] DIGI.BA: Sheleadersvr project web site, 2024. 2

[KMG22] KYRLITSIAS C., MICHAEL-GRIGORIOU D.: Social interaction with agents and avatars in immersive virtual environments: A survey. *Frontiers in Virtual Reality* 2 (2022), 786665. 1

[MH17] MARCOTTE R., HAMILTON H. J.: Behavior trees for modelling artificial intelligence in games: A tutorial. *The Computer Games Journal* 6 (2017), 171–184. 2

[MHY*07] MAÏM J., HAEGLER S., YERSIN B., MÜLLER P., THALMANN D., VAN GOOL L.: Populating Ancient Pompeii with Crowds of Virtual Romans. In *Proceedings of the 8th International Symposium on Virtual Reality, Archeology and Cultural Heritage - VAST* (2007). 1

[Ork03] ORKIN J.: Applying goal-oriented action planning to games. *AI game programming wisdom* 2 (2003), 217–228. 1

[PQD*10] PANZOLI D., QURESHI A., DUNWELL I., PETRIDIS P., DE FREITAS S., REBOLLEDO-MENDEZ G.: Levels of interaction (loi): A model for scaffolding learner engagement in an immersive environment. pp. 393–395. 1

[RBM23] RIZVIC S., BOSKOVIC D., MIJATOVIC B.: Advanced Interactive Digital Storytelling in Virtual Reality presentation of Austrian-Hungarian fortresses around Trebinje. In *Eurographics Workshop on Graphics and Cultural Heritage* (2023), Bucciero A., Fanini B., Graf H., Pescarin S., Rizvic S., (Eds.), The Eurographics Association. 2

[ST07] SHAO W., TERZOPOULOS D.: Autonomous pedestrians. *Graph. Models* 69, 5-6 (2007), 246–274. 1