# Cognitive Archaeology in Virtual Environment

Maxime Dumonteil, Valérie Gouranton, Marc J.-M. Macé, Théophane Nicolas, Ronan Gaugne

► To cite this version:

Maxime Dumonteil, Valérie Gouranton, Marc J.-M. Macé, Théophane Nicolas, Ronan Gaugne. Cognitive Archaeology in Virtual Environment. ARCHERIX 2025 - 1st Workshop on Archaeology and Cultural Heritage through XR, IEEE Virtual Reality (VR), Mar 2025, Saint-Malo, France. pp.1-4, 10.1109/VRW66409.2025.00017. [hal-04943503](https://hal.science/hal-04943503v1)

HAL Id: hal-04943503

https://hal.science/hal-04943503v1

Submitted on 12 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Cognitive Archaeology in Virtual Environment

Maxime Dumonteil *

Univ. Rennes, Inria, CNRS,
IRISA, France

Valérie Gouranton †

Univ. Rennes, INSA Rennes,
Inria, CNRS, IRISA, France

Marc Macé ‡

IRISA, CNRS, Univ. Rennes,
Inria, France

Théophane Nicolas §

Inrap, UMR Trajectoires,
France

Ronan Gaugne ¶

Univ. Rennes, Inria, CNRS,
IRISA, France

<figure><img src="image_2.png" /><figcaption>Figure 1: Comparing the behavior during an archaeological task between different modalities</figcaption></figure>

ABSTRACT

The integration of Virtual Reality (VR) and Extended Reality (XR) has significantly transformed the working methods of archaeologists, enabling experts to explore artifacts in a controlled virtual environment without altering them. However, it may differ when XR technologies are used compared to real-world experiences. This paper introduces a workflow that generates standardized eye-tracking and behavioral data, allowing for a comparison between real and virtual modalities. A user study of visual exploration of an artifact based on this workflow is presented to identify biases in the use of VR tools for archaeology. The paper also outlines ongoing work to explore virtual interactions, such as representing missing or hidden parts of archaeological materials, to study their influence on user behavior. XR offers promising opportunities for enhancing the understanding and interactions with cultural heritage, but further research is needed to extend and refine these methods to ensure they provide a relevant archaeological experience.

**Index Terms:** Virtual Reality, Archaeology, HMD, Immersive room, Behavior.

# 1 INTRODUCTION

Virtual reality (VR) has become increasingly employed in the fields of archaeology and cultural heritage [5, 7, 14]. It facilitates interaction with historical or archaeological material, artifacts and sites from the past without altering them, in a safer and more comprehensive way. For archaeologists and specialists, the integration of virtual reality-based workflows is gradually transforming techniques, offering new methods of interaction, interpretation and exploration. The observation of archaeological material is central to the archaeologist's work. Virtual reality makes it easy to visualize digital copies of such material. In this sense, techniques based on virtual reality, and more generally extended reality (XR), can address

a wide range of observational activities [12].

However, exposure to virtual archaeological representations may induce different knowledge, information and observational behaviors than direct confrontation with the real thing, depending on the use case [9, 15, 21, 10]. This question is becoming particularly critical as recent studies link behavioral studies of the way people look at artifacts with archaeological interpretations of the organization of the society that produced them [1, 2] or how they were used [20].

For this reason, we propose to design an experimental workflow to assess the extent to which the use of virtual reality can adequately reproduce a real archaeological experience. To this end, it is important to identify the main parameters that need to be observed to ensure that the simulation is as close as possible to reality and that the numerical exploration deviates as little as possible from the real experience.

The studies carried out with this workflow will then be used to identify cognitive and behavioral biases introduced by XR environments in the execution of archaeological tasks. Cognitive biases relate to differences in knowledge information deduced from observation tasks. Behavioral biases relate to differences in user activity during these tasks. These two types of biases can lead to differences in interpretation.

In this article, we present a brief overview of the workflow developed in previous work, followed by some results from a user study we conducted on an archaeological use case with this workflow, and we introduce the ongoing work we are pursuing to further explore the field of cognitive archaeology.

2 STUDYING ARCHAEOLOGICAL BEHAVIOR IN XR

The approach that we propose to develop considers user behavior through their gaze activity and interactions with the environment. Gaze activity is analyzed using eye-tracking data such as eye fixations and gaze path. The analysis of the user's interaction with the environment is based on metrics about the position or the distance between the observed object and the user's head. In order to be able to compare the behavior of a user performing an archaeological task in real and virtual environments, we established a generic methodology for extracting standardized and comparable behavioral data across different modalities. Subsequently, this workflow was implemented to conduct a user study aimed at identifying the potential impact of the virtual modality on user behavior.

<footnote>*0009-0004-4267-4289</footnote>

<footnote>†0000-0002-9351-2747</footnote>

<footnote>‡0000-0003-4124-8697</footnote>

<footnote>§0000-0003-0098-9456</footnote>

<footnote>¶0000-0002-4762-4342</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 2: A workflow to compare gaze-related behavior across real and virtual modalities (Figure from [4])</figcaption></figure>

## 2.1 Comparison Workflow

Analyzing user behavior can be a complex task, especially when attempting to do so across different modalities and therefore different devices. In addition, comparing 3D gaze data across modalities presents unique challenges. For example, user movement tracking devices in real and virtual environments produce different data logs in different frames of reference, whether for eye tracking or body movement. Standardized methods of data collection and analysis are required to perform comparable analyses. We have therefore developed a comprehensive data preparation and analysis workflow based on four main steps (Fig. 2):

* **Experimental settings:** this step is related to the definition of the protocol, the constraints and the rules necessary to establish a comparison,

* **Acquisition:** this step is related to the description of the required data for processing and analysis and how to generate them,

* **Processing:** this core step is related to the description of the processes to transform the raw acquired data into the standardized format of 3D gaze data, and behavior data,

* **Interpretation:** this step is related to the analytical techniques to be applied to these data sets in order to produce metrics and graphics.

The complete workflow is detailed in [4]. It should be noted that the design of this methodology is sufficiently generic to be able to apply the process of comparing visual and interactive behavior between real and virtual to other application domains.

## 2.2 User Study Across Modalities

We applied this methodology to an archaeological task of observing artifacts. To this end, we set up a user study experiment in which a corpus of archaeological artifacts was analyzed in three modalities (Fig. 1): a real environment, a virtual environment using a head-mounted VR device (HMD), and a virtual environment using a large CAVE-like immersive room. The aim of this experiment was to compare participants’ behavior (head movements, gaze patterns and task performance) between the three modalities in order to define the potential interpretation biases introduced by the use of VR tools in this domain.

It was essential to clearly define the rules for participant interactions to ensure the experiment remained consistent across different modalities. Similarly, the virtual environments replicate the real environment as faithfully as possible, using for example photogrammetry to digitize the real artifacts.

The results, detailed in [3], highlight several issues that need to be considered when using VR solutions in archaeology for analysis

and interpretation. Most importantly, they also indicate a coherence in user behavior and performance across modalities in performing the task under consideration. Participants’ visual exploration is consistent across modalities, with the same amount of information being obtained about them, despite the variability in the characteristics of the artifact within the corpus selected for the experiment. Statistical discrepancies in the user’s movements were identified, but these can also be mitigated by their small amplitude. The analysis of the eye-tracking data did not reveal any differences in the global gaze pattern, as determined by the classification of the exploration of the different regions of interest on the observed objects. However, differences were found in specific oculomotor behaviors, but they still did not affect the global gaze pattern on the object.

In summary, the results suggest that VR tools can be used without bias in the archaeological context of artifact observation tasks, provided that the virtual features are accurately rendered with the correct geometry and scale of the virtual model.

## 3 GOING FURTHER WITH COGNITIVE ARCHAEOLOGY IN XR

The two contributions presented provide a foundation for studying user behavior when carrying out archaeological tasks with XR devices. At the moment we are interested in the reproduction of real tasks in XR. However, XR environments make it possible to go beyond the simple reproduction of reality and perform tasks that would be difficult to perform in real life. To this end, we are now focusing on two specific use cases: the representation of missing parts of artifacts and the correct visualization of hidden elements in archaeological material.

### 3.1 Representing the missing parts of artifacts

Archaeological objects are often incomplete or damaged. When physical copies of these objects are made, the missing parts are often reconstructed in order to restore a complete element and thus more faithfully represent its actual function or appearance for an exploration and manipulation study. However, how this reconstruction is carried out can be a subject of debate: on the one hand, to remain as faithful as possible, even if this means inducing false representations, or, on the other hand, to move away from fidelity so that there is no confusion between the original elements and those that have been added.

This choice is illustrated by the concept of anastylosis (Fig. 3), which corresponds to the reassembly of an existing artifact (or monument) using fragmentary parts added in a recognizable way in order to ensure that conservation is as accurate as possible [16, 13]. While XR environments facilitate the representation of complete reconstructions of artifacts or monuments, they also allow different visualizations of the reconstructed elements to be proposed, as well as the representation of different reconstruction hypotheses. It is particularly interesting because of its ability to encompass all types of artifacts, regardless of their scale or the number of missing
elements. For example, its application can be proposed in the representation of a complete ruin, filling in the missing or empty sections of the building, thus creating the presumed complete structure [22].

The reconstruction process for a representation in an XR environment can be approached in different ways, with greater implications due to the multiple possibilities offered by 3D modeling. For example, it is possible to hide missing parts or, on the contrary, make reconstructed areas more visible. In addition, the visibility of these added elements can be controlled and adapted by the user [11, 18]. Virtual anastylosis effects represent a valuable tool for experts, as they facilitate the straightforward application and evaluation of diverse reconstruction hypotheses, while also providing a quantitative representation of each hypothesis's reliability.

Existing work already recommends some guidelines for an effective anastylosis effect in an architectural context [16], emphasizing the importance of visually representing the added elements in a way that is clearly distinct from the original parts. In addition, this work points out that it is also important that the modifications made to the digital representation of the original artifacts should maintain a coherence of aesthetic dominants and keep them at the center of the observer's attention. This work also shows that it is preferable to add new elements to the missing parts than to leave the segment empty.

The impact of virtual restoration is influenced by many aspect and choices, such as the luminance values of added parts [16] but also the textures and virtual material, color changes, choice of background or highlight the area. Design choices for the graphical representation of missing parts can thus have a significant impact. We believe that a behavioral user study should provide interesting guidelines for identifying relevant metaphors for virtual restoration. Therefore, this topic will be examined in a future work through this approach. First, we plan to conduct an experiment using reconstructed pottery with anastylosis and the same one in a complete and unaltered version, both in a real and a virtual context. The objective is to better define the impact of the anastylosis effects to establish guidelines for their use in XR applications, thereby expanding the advantages of XR in archaeology. We aimed to apply various types of anasylosis effects in an XR environment, including transparency, border highlighting, color changes, or texture modifications. Our goal is to determine which effect is preferred and why, based on behavioral analysis.

<figure><img src="image_4.png" /><figcaption>Figure 3: Three types of anasylosis used in the experiment: (A) complete; (B) with an empty cavity; (C) with a cavity filled with lighter stone (Figure from [16])</figcaption></figure>

## 3.2 Representing the hidden parts of artifacts

Another interesting aspect made possible by XR environments is the ability to see through to the inside of archaeological material

to access parts inside an artifact, or to visualize the techniques employed in the fabrication of these objects. To access this hidden content, the most common solution is to use medical imaging techniques, in particular CT visualization (Fig. 4) [6, 8], which is a relevant method due to its efficiency and non-destructive nature in capturing the interior of an object. However, the visualization tools associated with medical imaging require certain skills to be handled correctly, and offer mainly 2D rendering, resulting in a lack of depth perception.

This raises the question of how best to represent these hidden elements in order to avoid confusion with others. Special consideration should be given to depth perception, with the aim of making the exploration of artifacts as efficient and natural as possible. The challenge for archaeologists is to visualize and access new information directly, to improve their interpretation and to prepare an excavation in the best possible way.

For example, simple solutions to clearly examine the inside of the object could be to visualize the object in a semi-transparent way, filtering the value from the scan to show only a part of the whole object, or using a virtual window inside and displaying only the inner elements through this window [19, 17]. Another study presents an experiment evaluating the estimation of the position of some elements inside the archaeological material [6]. The aim is to design and evaluate representations of these hidden elements, and their perception so that they can coexist with directly visible parts with a new effect. This study also underlines that it is important to see both the external and internal structure of the artifact in order to understand the production process or to spatially locate the various elements inside. In addition, according to the study, experts confirmed the effectiveness of the application's spatial perception capabilities, as demonstrated by the ability to accurately identify different depth maps, up to eight elements in a row while judging the relative distances between them.

Consequently, a more precise study of the representation of the internal structure of archaeological material, and the resulting quality of depth perception, is a subject that will benefit from the approach based on the study of user behavior in future work. Analysis of the user's oculomotor and interactive behavior should provide a better understanding of the cognitive process used by the expert in such environments. This approach will be used to improve visualization techniques and facilitate the exploration and non-destructive analysis of archaeological material. It is important to highlight a particular scientific issue linked to this type of representation: medical imaging visualization is based on volume rendering, which is rarely used in XR and even less so in conjunction with eye tracking. We therefore first need to extend our methodology to address this problem, by proposing tools for analyzing gaze in a 3D volumetric environment.

<figure><img src="image_5.png" /><figcaption>Figure 4: Mix of representation of a cremation urn; Top Left: Cremation urn. Top, middle and right: slice views from CT scan. Bottom: Volume rendering of the content of the cremation (Figure from [6])</figcaption></figure>
# 4 CONCLUSION

In summary, the XR has become a relevant tool for archaeologists and cultural heritage experts, facilitating new exploration and manipulation techniques that ensure safer and more comprehensive methods, as well as new forms of interaction and access to information about artifacts. The studies presented focus on the improvement and analysis of specific archaeological use cases involving a virtual environment. In order to validate the relevance of using XR environments for archaeology, we proposed a methodology to discern the differences between real and virtual exploration and the parameters that elucidate these dissimilarities. This methodology, which falls within the field of research known as cognitive archaeology, is based on the study of user behavior when carrying out archaeological tasks. We conducted a user study using this methodology in order to identify potential behavioral biases. The initial results encourage us to take this cognitive archaeology approach further in an XR environment. Moreover, we plan to study two specific use cases: the correct representation of a missing part of an incomplete artifact and the representation of hidden parts of an artifact. The workflow will also be implemented to support the analysis of cognitive behavior and to propose visualization and interaction solutions relevant to these two use cases. This will allow to characterize the effectiveness of anastylosis variants and gain a better understanding of the depth perception in a scanned representation, with a subsequent goal of improving the efficacy of XR tools for archaeology. Both would contribute to a more complete and comprehensive analysis of archaeologists' activities in XR applications.

## FIGURE CREDITS

Figure 2: Workflow Components schema, structured in four blocks: Experiment Settings (purple), Acquisition (green), Processing (blue) and Interpretation (red). Figure from [4].

Figure 3: Three types of anastylosis used in the experiment: (A) complete; (B) with an empty cavity; (C) with a cavity filled wither lighter stone. Figure from [16].

Figure 4: Top Left: Cremation urn. Top, middle and right: slice views from CT scan. Bottom: Volume rendering of the content of the cremation. Figure from [6].

## ACKNOWLEDGEMENTS

This work was partially funded by Equipex+ Continuum ANR-21-ESRE-0030 and CNRS MITI AAP Digital Twin AJAX.

## REFERENCES

[1] F. Criado-Boado, D. Alonso-Pablos, M. J. Blanco, Y. Porto, A. Rodríguez-Paz, E. Cabrejas, E. del Barrio-Álvarez, and L. M. Martínez. Coevolution of visual behaviour, the material world and social complexity, depicted by the eye-tracking of archaeological objects in humans. *Scientific Reports*, 9(1):3985, mar 2019. Number: 1 Publisher: Nature Publishing Group. doi: 10.1038/s41598-019-39661-w

[2] F. Criado-Boado, L. M. Martínez, M. J. Blanco, D. Alonso-Pablos, Y. Porto, and E. d. Barrio-Álvarez. Gazed Pottery: an Archaeometric-Cognitive Approach to Material Culture Visuality. *Journal of Archaeological Science*, 154:105770, jun 2023. doi: 10.1016/j.jas.2023.105770

[3] M. Dumonteil, V. Gouranton, M. J.-M. Macé, T. Nicolas, and R. Gaugne. Do we study an archaeological artifact differently in vr and in reality? In *ICAT-EGVE 2024-joint 34th International Conference on Artificial Reality and Telexistence & the 29th Eurographics Symposium on Virtual Environments*, pp. 1–9, 2024.

[4] M. Dumonteil, M. J.-M. Macé, V. Gouranton, T. Nicolas, and R. Gaugne. A method for standardizing eye-tracking and behavioral data in real and virtual environments. In *Proceedings of VISIGRAPP (1): GRAPP*, 2025.

[5] R. Gaugne, J.-B. Barreau, F. Lécuyer, T. Nicolas, J.-M. Normand, and V. Gouranton. Extended reality for cultural heritage. In *Handbook of Cultural Heritage Analysis*, pp. 1405–1437. Springer, 2022.

[6] R. Gaugne, T. Nicolas, Q. Petit, M. Otsuki, and V. Gouranton. Evaluation of a mixed reality based method for archaeological excavation support. In *ICAT-EGVE 2019*, pp. 1–8, 2019.

[7] M. Hostettler, A. Buhlke, C. Drummer, L. Emmenegger, J. Reich, and C. Stäheli. Digital archaeology between hype and reality: The results of a survey on the use of 3d technologies in archaeology. In *The 3 Dimensions of Digitalised Archaeology: State-of-the-Art, Data Management and Current Challenges in Archaeological 3D-Documentation*, pp. 181–203. Springer International Publishing Cham, 2024.

[8] L. M. McKnight, J. E. Adams, A. Chamberlain, S. D. Atherton-Woolham, and R. Bibb. Application of clinical imaging and 3D printing to the identification of anomalies in an ancient Egyptian animal mummy. *Journal of Archaeological Science: Reports*, 3:328–332, Sept. 2015. doi: 10.1016/j.jasrep.2015.06.028

[9] M. A. Mirzaei, J.-R. Chardonnet, and F. Merienne. Comparing real and virtual object manipulation by physiological signals analysis: A first study. 2018.

[10] B. Molloy and M. Milić. Wonderful things? a consideration of 3d modelling of objects in material culture research. *Open Archaeology*, 4(1):97–113, 2018.

[11] T. Nicolas, R. Gaugne, C. Tavernier, E. Millet, R. Bernadet, and V. Gouranton. Lift the veil of the block samples from the warcq chariot burial with 3d digital technologies. In *International Conference on Virtual Systems & Multimedia (VSMM 2018)*, pp. 1–8. IEEE, 2018.

[12] L. Opgenhaffen. Visualizing archaeologists: a reflexive history of visualization practice in archaeology. *Open Archaeology*, 7(1):353–377, 2021.

[13] E. Pietroni and D. Ferdani. Virtual Restoration and Virtual Reconstruction in Cultural Heritage: Terminology, Methodologies, Visual Representation Techniques and Cognitive Models. *Information*, 12(4):167. Apr. 2021. Number: 4 Publisher: Multidisciplinary Digital Publishing Institute. doi: 10.3390/info12040167

[14] D. Psarros, M. I. Stamatopoulos, and C.-N. Anagnostopoulos. Information technology and archaeological excavations: A brief overview. *Scientific Culture*, 8(2), 2022.

[15] M. Rusnak. 2d and 3d representation of objects in architectural and heritage studies: in search of gaze pattern similarities. *Heritage Science*, 10(1):1–18, 2022.

[16] M. Rusnak, A. Brzozowska-Jawornicka, and Z. Koszewicz. How to enhance perception of reassembled but incomplete works of ancient art? eye-tracking study of virtual anastylosis. In *Arts*, vol. 12, p. 15. MDPI, 2023.

[17] G. Schall, E. Mendez, E. Kruijff, E. Veas, S. Junghanns, B. Retinger, and D. Schmalstieg. Handheld augmented reality for underground infrastructure visualization. *Personal and ubiquitous computing*, 13:281–291, 2009.

[18] S. Shakya et al. Virtual restoration of damaged archeological artifacts obtained from expeditions using 3d visualization. *Journal of Innovative Image Processing (JIIP)*, 1(02):102–110, 2019.

[19] T. Sielhorst, C. Bichlmeier, S. M. Heining, and N. Navab. Depth perception—a major issue in medical ar: evaluation study by twenty surgeons. In *Medical Image Computing and Computer-Assisted Intervention—MICCAI 2006: 9th International Conference, Copenhagen, Denmark, October 1-6, 2006. Proceedings, Part I 9*, pp. 364–372. Springer, 2006.

[20] M. Silva-Gago, F. Ioannidou, A. Fedato, T. Hodgson, and E. Bruner. Visual attention and cognitive archaeology: an eye-tracking study of palaeolithic stone tools. *Perception*, 51(1):3–24, 2022.

[21] N. Takahashi, T. Inamura, Y. Mizuchi, and Y. Choi. Evaluation of the difference of human behavior between vr and real environments in searching and manipulating objects in a domestic environment. In *2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)*, pp. 454–460. IEEE, 2021.

[22] E. Torun and S. Ercan. Two decades of anastylosis experience at sagalassos. *Exempli Gratia. Sagalassos, Marc Waelkens and interdisciplinary archaeology*, Leuven, pp. 27–41, 2013.