# Context-Based Annotation Visualisation in Virtual Reality: A Use Case in Archaeological Data Exploration

Michele de Bonis, Patrick Bourdot, Huyen Nguyen

<figure><img src="image_1.png" /></figure>

## To cite this version:

Michele de Bonis, Patrick Bourdot, Huyen Nguyen. Context-Based Annotation Visualisation in Virtual Reality: A Use Case in Archaeological Data Exploration. EuroXR 21st International Conference 2024, Nov 2024, Athènes, Greece. pp.95-119, 10.1007/978-3-031-78593-1_7. http-04978190

HAL Id: hal-04978190

https://hal.science/hal-04978190v1

Submitted on 5 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Context-based Annotation Visualisation in Virtual Reality: A Use Case in Archaeological Data Exploration

Michele De Bonis¹, Huyen Nguyen¹, and Patrick Bourdot¹

Université Paris-Saclay, CNRS, LISN,
{michele.de-bonis, huyen.nguyen,
patrick.bourdot}@universite-paris-saclay.fr

**Abstract.** In this paper we present a Virtual Reality (VR) tool that facilitates the visualisation and exploration of context-based, multi-level annotations in archaeology. Indeed, thanks to photogrammetry and laser scanning techniques, archaeologists can capture and reconstruct almost faithfully ruins and excavations in 3D. Thus, they can maximise their work performance on the field within their limited excavation time and leave deep, but often slow, analysis tasks at later stages offsite. Using VR, we can enrich 3D captured and reconstructed environments by adding extra information for *ex situ* analysis of archaeological data. This approach can also help the archaeologists speed up their collaboration with other experts or easily share the information with the public. We aim to assist the exploration of numerous annotations and information at disposal that can be densely clustered in a 3D reconstructed world. We collaborated closely with archaeologists at AOROC in the process of designing annotations that convey information with increasing Levels of Detail (LoDs), reflecting the complexity of the contained data. We then developed different interaction techniques that allow users to explore the annotations by switching between LoDs, including active selection by clicking, proximity approach, and the combination of both. Additionally, we proposed and evaluated the effectiveness of annotation grouping mechanism to enhance visual clarity by aggregating annotations in close spatial proximity. The results of our user study with non-experts, aggregated with a qualitative study with the archaeologists, showed user preference for the combination of the interaction techniques, but evidenced some limitations for the grouping of annotations.

**Keywords:** Virtual Reality · Annotation · Information Retrieval · Data Exploration · Selection · Proximity · Archaeology

## 1 Introduction

Extended Reality (XR) has demonstrated its capability in extending our workspace from conventional 2D interfaces to the real world [4, 62]. In immersive data visualisation and analytics, a user does not need to be anchored anymore to their
<figure><img src="image_3.png" /><figcaption>Fig. 1: (A) A user equipped with a VR HMD exploring the reconstructed archaeological site. (B) Overview of the annotation system. Annotations at the first and second level of detail can be seen around a column in the archaeological site.</figcaption></figure>

workstation and be able to be immersed within ‘data worlds’. It facilitates a better understanding of complex data structures and patterns than was previously impossible using conventional desktop-based systems [16]. A variety of display devices exists in different price ranges, from affordable off-the-shelf Virtual Reality (VR) and Augmented Reality (AR) Head-Mounted Displays (HMDs) to high-end multi-touch walls, interactive tabletops, and CAVE-like projection systems. These emerging multi-modal interaction interfaces can be used to support effective analytical reasoning and decision-making [22, 49, 54]. Moreover, the data itself can be either visualised and integrated in a virtual environment [14, 43], or become part of the physical environment as real objects augmented with more information [6, 28], which broadens the horizon of data analysis tasks.

XR technology can substantially enhance data exploration and analysis by overlaying contextual information in 3D space, either through AR with annotations or through VR where virtual scenes can be 3D reconstructions of the real world. A typical example in XR could be to directly add *in situ* annotations to 3D models or objects to specify extra information during the interaction. In the physical reality, this process would typically be conveyed orally or through written notes on paper, thereby diverting the user's attention from the current interaction context to the recorder or written page. In XR applications, adding and working with 3D annotations directly in the scene has the clear benefit of allowing the user to maintain their focus on the visual reference, without interrupting their flow of thoughts and visual connection with the ongoing interaction.

There are two fundamental aspects that we must take into account when designing an XR system dedicated to such applications. The first is how to create and add information in the form of annotations and associate them to an object or area in 3D space. The annotations can also be linked to other comments or remarks that may be useful in the future for the user to revise their work. The second aspect is ensuring the efficient retrieval of the information on annotations, which is generally related to the way information is created and added. Different tools trying to solve both of these aspects have been developed throughout the
years, predominantly on a 2D screen with a web-browser interface. They usually focus on the semantic partition and segmentation of 3D virtual objects to identify the actual parts of their corresponding real objects [44, 63] to annotate. Some of them also aim to incorporate information into the 2D visualisation that links to the elements of 3D models [45, 46].

This paper evidences the strengths and lacks of the current support for the exploration of annotations using VR. We aimed to leverage different possibilities of exploring annotations in a 3D world in a more intuitive and effective way, following the requirements emerged during design sessions with archaeologists from AOROC (Archéologie et Philologie d'Orient et d'Occident) laboratory. Our main contribution was the proposal and evaluation of multi-level, context-based annotations to address the complexity of archaeological annotations. Specifically, we tested three main techniques of exploring multi-level representation of information: proximity based on the distance to the user, active selection, and the combination of both. We also evaluated the advantages that could arise when the annotations are grouped together. The generic design of these techniques makes them applicable to various other applications, as well as different data types. More broadly, these techniques should fit in the overarching perspective of improving the user's familiarisation with XR technology in domain applications when immersive interfaces are likely to be more widespread.

The rest of the paper is organised as follows. Section 2 reviews some existing works on annotations in 2D and more attentively in 3D with examples from VR and AR applications. Section 3 and Section 4 describe our platform and model that aid to explore annotations with several interaction techniques we proposed. Sections 5 presents the experimental study we conducted to assess the performance of those techniques. It will be followed by conclusion and future work in Section 6.

## 2 Related Work

**Annotations in the early days.** In the domain of Human-Computer Interaction (HCI), we can trace the first example of annotations back to Tivoli, an electronic whiteboard application run on Xerox Liveboard screen with pen-based interaction in 1993 [50]. It was designed to support group meetings with writing, drawing, gestural commands, besides other basic features. Since then, many annotation-supported systems have been proposed for the creation and editing of annotations in the form of text or post-its [26, 42, 52], audio recordings [1, 30, 59], drawings or sketches [21, 34], images [1], and photo and/or video [29, 51].

**Annotation Taxonomies, Semantics, and Ontologies.** It is important to have an efficient method of classifying annotations, especially when a high number of them is added to the scene. It can facilitate information retrieval and also the visualisation process during the interaction. Besides the conventional classification approach of annotations based on their nature (i.e. text, audio, image, etc.), Pick *et al.* [51] considered three kinds of annotations: view annotations
capturing a certain perspective onto the scene, point or area markers attached to the annotated object of interest, and action-item annotations as a metadata type for other purposes that do not correlate to the scene or objects in it. In a more advanced approach, there exist ontologies [2] and knowledge models [3, 39] for semantic annotations. The work in [39] shows the advantage of adding semantics to annotations in filtering operation based on different concepts. In AR domain, a general taxonomy of annotations has been recently proposed by Garcia et al. [27]. It identifies several shared characteristics of 3D annotations, namely content, location, temporality, and interaction. This taxonomy can be extended to all 3D annotations in immersive systems, not only in AR applications, with the exception of certain spatial components specific to AR that capture the anchoring location in a reference system connected to the real world, and some other small caveats (cf. Sec. 4.1). An updated taxonomy that takes into account the whole XR spectrum has been proposed by Borhani et al. [10]. To conclude, in most applications, this ontology-based approach may introduce an unnecessary layout of complexity and increase information retrieval time in the interaction process. Moreover, these knowledge models need to be domain-specific and thus limit their range of applications.

**Computer-Vision-based Enhancement and Optimisation.** Recent research in 3D annotations for AR has focused on leveraging computer vision advances for more accurate spatial annotations. Gauglitz et al. [28] developed a mechanism for remote users to add spatial annotations during collaborative tasks on real objects. Bruno et al. [12] implemented a mobile AR system using overlapping 3D models on real objects, capturing annotations' positions from different angles through touch input for information exchange among experts. Cao and Kim in [21] proposed a real-time video annotation system allowing users to draw shapes on live video streams, aiming to facilitate remote collaboration. These systems aim to support collaboration between users, often using tablet or screen-based systems with cameras. When integrating 2D annotations into 3D space, optimizations for interaction and visualization, like Nuernberger et al. [48]'s exploration of 2D gestures on tablets, are considered.

**Change-Tracking Workflow.** In addition to annotation creation and editing techniques, researchers are also interested in the process of keeping track of changes made to annotations during and after collaboration. In IDEA (Interactive Data Exploration and Analysis) framework [1], time-aware annotations can be introduced at any stage of a data analysis process. To keep track of the decision process in collaborative VR for design reviewing, Guerreiro et al. [29] proposed a tree-based structure and issue-based argumentation supporting discussion within annotations and disclosing the provenance of decisions. With regard to complex annotations, Pick et al. [52] evaluated a data annotation workflow that supports annotation creation, review, and modification in a CAVE-like system. They found that other interaction techniques were needed to include VR into factory planning workflow.
**Annotations for Archaeology and Cultural Heritage.** Since our use case is in archaeology-related annotation exploration, we also looked into some existing works on annotations for archaeology and cultural heritage. There are many web-based annotation services that have been proposed using semantic approach such as [33] or [63] for cultural artefacts. In ATON framework for European infrastructures and museum displays [24], the annotations consist of a “sub-portion of a 3D object or scene” linked to related information presented to the users. There exists also AIOLI platform [17] that exploits collaborative annotations for the documentation of heritage assets. The annotation can be drawn on a 2D image and then propagated on its 3D auto-generated model. For VR-based annotation systems, there is Dig@It, an immersive platform visualising the stratigraphical and architectural structures of an archaeological site [40]. Its annotation list is also visualised on a virtual tablet and the data can be searched based on the selection of the strata on the scene. In the AR domain, MAAP (Megalithic Art Analysis Project) Annotate is a tool for the analysis and annotation of megalithic art [6]. The user can create annotations through voice recordings, which are stored on an online database for later use. This latter tool still shows some limits in terms of memory capacity due to the hardware used.

In the approach of annotation creation based on crowd-sourcing (or social tagging) for cultural heritage and archaeology, a research question that arises is how to exploit efficiently this big data of 3D content that is not linked to a specific data structure. There exists a research work from Pittarello who proposed an approach based on a bottom-up, user-driven, classification [53]. Similarly, Hunter and Gerber suggested to use Resource Description Framework (RDF) model and an ontology-directed folksonomy to improve the interoperability across social tagging and annotation systems [33]. While incorporating an authority restriction for closed communities that possess specific knowledge or expertise, a folksonomy-based ontology seems not to be the most suitable and reliable approach for annotations oriented towards experts.

To conclude, existing works on annotations primarily focus on creation and editing functions using VR and AR technologies. Despite efforts in semantics and knowledge modeling, annotations often remain limited to a single Level of Detail (LoD). Some prior works have proposed LoDs for comparing screen-fixed and world-fixed annotations [20] and personalizing the amount of information shown during presentations [55]. However, transitioning between these levels was not actively explored. Köppel et al. [37] used LoDs to address occlusion issues when exploring labeled points of interest on a map, but their approach had limitations with long text and lacked investigation of grouping mechanisms (which goes by the name of super labels). While their method of switching between LoDs was similar to our *Proximity Activation* (Sec. 5.1), it was qualitatively tested only against conditions without LoDs, unlike our *Active Selection* (Sec. 5.1) method which was compared with other interaction techniques. When it comes to information grouping, Tatzgern et al. [57] proposed adaptive hierarchical clustering to enhance information organization, which was clearly preferred by users.
## 3 User Research Study: VR Tool to Edit and Explore Annotations of Archaeological Data

We conducted a comprehensive user research study involving interviews and a workshop with archaeologists at AOROC (Archeology Philologie d'Orient et d'Occident) laboratory to assess their needs both pre- and post-excavation, with regard to offline tasks and workflow enhancements. Five archaeologists were interviewed (and two with a follow-up interview) about the organisation of an excavation campaign, its duration, their respective responsibilities and typical tasks. A pivotal point was the communication between experts in different subfields within the same excavation project such as anthracology, landscape and settlement archaeology, bioarchaeology, etc. In addition, one of the most frequently encountered challenges was the constrained time frame for the excavation as it is often complicated and costly to go back onsite. Moreover, the destructive nature of archaeological ruins over time could lead to loss of important information. Therefore a meticulous documentation of all the activities is needed to facilitate future revisions and reevaluations of previously formulated hypotheses.

The archaeologists have to deal with an heterogeneous mass of data in their daily basis, as emerged from the workshop organised with the same experts in individual interviews. The typical datasets include data in 3D (e.g., scans, models, hypothesised reconstructions), 2D (e.g., photos and drawings of the current and past state of excavation sites, topographical views, orthophotos, stratigraphic layers, measures), and 1D (predominantly text of ancient reports, culturally related literature, context-related and oral information, findings out of context).

Based on the interviews, it appeared that having a feature-rich annotation tool on XR platforms dedicated to archaeology was of major importance. Adopting a design thinking methodology, we identified the necessity for a flexible tool for creating, editing, and interacting with annotations in VR. This tool would be particularly helpful for the archaeologists to share information, even distant in time and space, in a more efficient way. In the first iteration cycle, the archaeologists tested an early-stage prototype, which allowed them to create and visualise annotations on 3D reconstructed excavation data and define categories for the annotation. This produced us their first feedback on how to improve the interface. They also highlighted that the most pressing issue was to be the visualisation and exploration of annotations in virtual environments. The creation of annotations in a VR environment presents another set of open challenges such as text input modality [8] that is being considered in another in-progress work.

## 4 Context-based Approach for Annotation Exploration

Technically, we focus on solving the challenge of how to effectively visualise the annotations in 3D scenes to facilitate the exploration and navigation through a high number of annotations present. Indeed, as mentioned in Section 2, in order to avoid saturating the scene and to help a user find quickly the relevant annotations, we propose an annotation model based on different Levels of Detail (LoDs). This model and the three context-based interaction techniques that exploit it will be detailed in this section.
## 4.1 LoD-based Annotation Model

We used the widely embraced Overview+Detail and Zooming [15, 56] design paradigm in the information visualisation field, which asserts interactive data exploration at multiple levels of detail. By following this paradigm, we aim to enable the effective presentation of both contextual and detailed information without overwhelming the user's cognitive workload [23, 35].

<figure><img src="image_4.png" /><figcaption>Fig. 2: A formal diagram of the content of an annotation. Category information is considered to make our annotation model compatible with ontology-based approaches. Provenance metadata contains the history of the annotation such as author(s), creation time, editing time. Other attributes refer to the other content such as images, photos, recordings, highlighted areas.</figcaption></figure>

In our LoD-based annotation model, we consider an annotation as an object anchored in space (attached to a point, an object, or an area of interest in the scene) (cf. [27]). This concept of annotation which is realised into the scene allows the user to create, edit, visualise, and share with others before, during, and after their task execution and collaboration. An annotation also has the following attributes: category, title, spatial position (which is related to a fixed point, an object, or an area in virtual and/or physical space), description, provenance (e.g., author, creation time, changed time), and others attributes (e.g., images, photos, recordings, etc.) (Fig. 2). The category attribute can be either empty, predefined by the user, or automatically created from an ontology (e.g., [2]) or a knowledge model (e.g., [3, 39]). In archaeology and cultural heritage, there exist several conceptual reference models that can be of use such as CRMarchaeo [18], CIDOC [19], or STAR [58]. With regard to position, an exceptional case that should not be disregarded is an annotation that refers to the entirety of the scene and thus is not anchored to a specific spatial point. This *Location Complexity* (cf. [10]) was not contemplated in [27] because it was focused only on AR environments. Provenance metadata allows the user to update and revise the content of an annotation and trace the modifications back to its prior version(s).

Considering the amount of information that each annotation may have, displaying its full content in the scene all at once can create visual occlusions, cause confusion and high cognitive workload of the user, and lead to an inefficient data exploration process. To overcome this problem, we propose applying the LoD concept, introduced by Clark in 1976 [13], into the management, visualisation, and interaction with annotations.
Despite the extensive research conducted on the LoD concept over the decades
following Clark's foundational paper, only few works specifically focused on the
LoD approach for 3D annotations (e.g., [20, 55]). Furthermore, most of them were
structured according to their application field, such as CAD data structures and
object composition links [3].

<figure><img src="image_5.png" /><figcaption>Fig. 3: Different levels of detail of an annotation visualised in sequence. A) The first level only shows the initial of the category the annotation belongs to. B) The second level shows the title. C) The third level presents the category, title and description. D) Another additional level shows also images and other information of the annotation such as author and time of creation.</figcaption></figure>

In our model, we consider that an annotation has at least the three following
LoDs:

- First level of detail (LoD 1) is the category the annotation belongs to. The category is represented by an initial letter and visually distinguished in the scene by a unique colour.

- Second level of detail (LoD 2) includes the title which gives some hints to the user about the content of the annotation.

- Third level of detail (LoD 3) adds the full description on top of the category and title.

However, considering that an annotation can contain more information than
that of three LoDs, we can provide additional levels (Fig. 3). The content and
number of LoDs that we propose can change according to the design and the
need of each system. In general, we consider that three levels should be enough
to facilitate the filtering of the information, without overwhelming the user with
excessively deep levels during information retrieval. In a more complex case,
based on the volume of the information of an annotation or different types of
multimedia data (images, photos, etc.) it may contain, the LoD structure and its
hierarchy can be flexible. Moreover, a functionality of the system which allows
the user to expose the content all at once can also be considered.

A certain degree of flexibility is highly advisable within an archaeological
context due to its unstructured nature of data, which often refers to incomplete
and partially destroyed elements. This is, for instance, the case when an archaeo-
logical object is composed of several parts, and some are missing or destroyed in
the current data. Each part can also have many historical occurrences. Therefore, many annotations can be created densely at an object or in a small area. In this situation, we define a specific LoD called “group of annotations” in which several annotations at LoD 1 within a certain distance range are grouped together. In our implementation, a such group is visually represented with a small square with a letter G (Group) at the centre. The colour of the square is composed by the colours of annotations that formed the group, disposed in a radial order (Fig. 4). We can apply the same principle for the annotations for a group of objects that are scattered in the scene but are linked due to their historical or semantic association.

<figure><img src="image_6.png" /><figcaption>Fig. 4: Annotation groups of five and four elements (A & B respectively). A group can be expanded and a button to regroup the annotations is shown on top of them (C).</figcaption></figure>

## 4.2 Interaction Techniques with LoD-based Annotations

Having established a hierarchy of annotation representations, the primary concern is how to interact with these LoDs. In traditional LoD systems, activation or LoD changes are based on perception factors like visual focus, distance to the object, and human perceptual thresholds (cf. [38]). Our interactive techniques align with these principles. Proximity triggers activation of an annotation's LoD, while considering perception thresholds for text readability through color differentiation and uppercase characters for category acronyms. This also applies to associated multimedia content like audio and images. The idea of switching between different levels of information according to the distance is also present in the work of Bellendat et al. [5]. However, it was adapted to proxemic interactions, thus more relevant in a multi-user or multi-device scenario.

An important aspect of LoDs of annotations is their interaction with semantic content, necessitating highlighting of category and textual elements often laden with domain-specific concepts and terms. Users should be able to quickly discern relevance even with minimalist annotations (e.g., at group level, LoD 1 or even LoD 2), enabling swift interaction for accessing further details. User judgment on the current visible LoD should also guide LoDs switching, and this leads to active selection as another technique of interaction. Finally, the combination of distance and active selection is also considered in our platform.
# 5 User Study

We conducted a controlled experiment with laymen (Fig. 1) and a qualitative evaluation with archaeological experts to assess the effectiveness of two interaction techniques proposed in Section 4.2. To simplify the user study, we limited the number of annotation LoDs to three. Annotations reflected real content but were simplified in the terms for non-experts’ understanding. We believe our VR tools for visualising and exploring annotations are equally effective for experts and novices. Results, combined with expert feedback, will allow us to refine the platform in subsequent development phases. We detail the laymen experiment first, followed by the qualitative study in Section 5.9, concluding with discussion.

## 5.1 Settings

To evaluate the different interaction techniques, we used as the setting a VR photogrammetric reconstruction of an ancient Hellenistic temple on the isle of Failaka, Kuwait. In the scene, many annotations about its characteristics were added with the help of the archaeologists. The experiment followed a 3×2 within-subjects repeated-measures design with two factors: SELECTION TECHNIQUE and GROUP.

For SELECTION TECHNIQUE, we aim to evaluate different techniques of interaction with the LoD-based annotations. There are three variations considered in this study:

- *Active-Selection:* This is the baseline condition. The user selects an annotation and access its LoDs by actively selecting it using a button on one of the controllers. More details linked to different LoDs of the annotation are revealed one by one by clicking on the same annotation. At the deepest LoD, after selecting it, the annotation goes back to the first LoD.

- *Proximity:* The user can access different LoDs by getting closer to the annotation. The user can move, either physically or by virtual locomotion, to reveal the LoDs. Since the LoD can switch abruptly from one level to another when the user uses teleportation in the virtual scene, which causes distraction and incoherence, we limited virtual navigation technique to “walking” or “flying” metaphors in this study. For this reason, a smoothing visual LoD transition mechanism was introduced (see Fig. 5 Left).

- *Selection-Proximity:* The user can use either one of the techniques to get access to LoD of annotations. By default, the user can open LoDs of an annotation by approaching it. However, the *Active-Selection* technique can be applied to access to an annotation from afar outside of the *Proximity* range, or to close the detailed view of the annotation when the user is too close to it and does not need its high LoD. We believe that this combination will give more options to the user to use whichever appropriate technique depending on the interaction context, and it can potentially lead to low workload.
<figure><img src="image_7.png" /><figcaption>Fig. 5: (Left) Activation by *Proximity* technique. The annotation's content is revealed in different LoDs according to its distance from the user. The threshold to switch between different LoDs was calibrated to ensure that the text was clearly readable at level-switching distance. (Right) A orange panel displaying a hint or question of what to look for. It was attached to the left controller.</figcaption></figure>

We consider *Active-Selection* condition as the baseline due to its default interaction technique in almost all of the XR systems. With regard to *Proximity*, it may cause visual cluttering when several annotations that are close to each other in space are visualised in high LoD at the same time. Targeting an end-user platform, we would expect the *Selection-Proximity* technique to perform better than the two techniques when they are used on their own.

For GROUP, our goal is to provide a more organised setting, especially when a high number of annotations is present in the scene. Therefore, we propose to group some of them together. The interaction with the groups can be triggered by one of the SELECTION TECHNIQUES described above. There are two conditions:

- *No-Grouping*: No annotation is grouped.

- *Grouping*: Annotations that are in close proximity to each other and fall within a radius smaller than a specified threshold are grouped together.

In total, there were six conditions in our user study (Table 1).

<table><caption>Table 1: Six conditions of the user study from two independent factors: SELECTION TECHNIQUE and GROUP.</caption><thead><tr><th></th><th>Selection</th><th>Proximity</th><th>Selection-Proximity</th></tr></thead><tbody><tr><th>No-Grouping</th><td>C1 (SelNoG)</td><td>C2 (ProxNoG)</td><td>C3 (SelProxNoG)</td></tr><tr><th>Grouping</th><td>C4 (SelG)</td><td>C5 (ProxG)</td><td>C6 (SelProxG)</td></tr></tbody></table>

## 5.2 Hypotheses

We assumed that by using position-based assistance mechanisms to help the user retrieve information during the interaction with the archaeological annotations, their task performance would be higher and their workload would be lower, leveraging the presumed rapidity our interaction techniques would provide, and the more organised visual space offered by the groups. We formulated the following hypotheses:
**H1** Users would be expected to move (virtually and physically) less in the conditions with *Active-Selection* compared to the ones with *Proximity*.

**H2** *Active-Selection* combined with *Proximity* techniques would facilitate the exploration and information retrieval of annotations with many LoDs in terms of time and user preference.

**H3** Organising annotation into groups would expedite the process of searching and exploring information in the annotations and therefore, *Grouping* will be preferred by users.

Based on the aforementioned premises, as a consequence of **H2** and **H3**, we would expect a stronger user preference for *Selection-Proximity* with *Grouping*.

## 5.3 Experiment Setup

The study was conducted using a HTC VIVE Cosmos headset with a cable connected to a computer. It enabled the participants to be able to move in a physical space of 3.5×3.5 squared metre. The participants could also use a touch pad for virtual locomotion as an alternative to physical moving to avoid fatigue. The software used for the experiment was Unity Engine version 2021, with Windows Mixed Reality 3 plugin, Unity XR Interaction Toolkit, and VIVE OpenXR that provides the controller visualisation.

## 5.4 Participants

In total, 19 participants from our Department and University, recruited by email and word of mouth, took part in the experiment. One participant had to stop the experiment due to cybersickness. At the end, we used the data collected from 18 participants. Those who completed the experiment aged between 22 and 44 ($\mu$ = 28.66, $\sigma$ = 7.34), including 13 men and five women.

## 5.5 Experimental Task

The participants explored a VR scene of a 3D reconstructed archaeological ruins and looked for information described in annotations. A hint or a question was provided to guide them in the process of finding its target annotation. For each condition, there were in total 24 annotations with three LoDs each that could be explored using three interaction techniques described in Section 5.1. For each task, four of them were predefined by an archaeologist, which helped to make the tasks plausible to novice users. They were also the target annotations to find during the task. The remaining 20 were automatically generated, but designed to resemble the real ones to maintain a similar appearance.

The task was designed to be performed by non-archaeological experts, provided that they sought well enough into the details of the annotations. Some annotations could be easier to find than others given the additional clue with regard to category from the hints. However, in all the conditions, the participants had to look around and carefully read the text before being able to answer the
questions. We also randomised the set of annotations used for each condition to avoid learning effect and bias created by the difficulty of the hints or questions.

We chose five different categories of annotations: archaeological elements, walls, columns, measures, materials. Intuitively, there could be some annotations that could belong to different categories, but the indication in the hints or questions were deemed disambiguating enough. Table 2 provides some exemplary hints or questions for each category that were used in the experiment.

<table><caption>Table 2: Some exemplary hints and questions given to the participants to find the annotations, arranged by category.</caption><thead><tr><th>Category</th><th>Hint or Question</th></tr></thead><tbody><tr><td><strong>Archaeological</strong><br><strong>Elements</strong></td><td>Find the element that consists of a palmette with eight leaves.<br>Find hypotheses of the reconstruction of the door.</td></tr><tr><td><strong>Walls</strong></td><td>What is the geographical origin of the wall blocks?</td></tr><tr><td><strong>Columns</strong></td><td>Which element of the column originally had a torus?</td></tr><tr><td><strong>Measures</strong></td><td>Find the length of the stylobate.<br>Find the size of the podium step.</td></tr><tr><td><strong>Materials</strong></td><td>What is the element composed partly of clay?</td></tr></tbody></table>

In dealing with categories, it was established that annotations had to be divided into categories to simulate the use of an ontology. While several ontologies already exist in archaeology domain (e.g., CIDOC CRM [19]), the aim of our study was to have a reasonably small number of categories so the users could easily orient themselves in the environment. In addition, we avoided using complex archaeological concepts, that would have been exceeding for the experimental tasks focusing on the interaction techniques. For the same reason, we did not integrate a hierarchy of architectural parts of historical objects, although they are frequently used in excavation and their hierarchical organisation could be useful both for annotation management, and, more generally, for archaeology reconstitution activities. A database was built to store all the annotations created in this experiment.

The annotation grouping was done based on spatial relationship using the Algorithm 1. The position of the group's icon (Fig. 4) was the mean of the annotations' position that composed it.

The hint or question for the search task was shown in an orange panel, attached to the left controller (Fig. 5 Right) so that it could be consulted quickly at all time, but did not encumber the visual field of the user in the scene. Both left- and right-handed participants found the placement of this panel acceptable. It was preferred over the mirrored placement near the right arm, which was initially assumed to be more suitable for left-handed participants (16% of the total number of participants, based on feedback during the tutorial phase).

## 5.6 Procedure

Each experimental session lasted approximately 45 minutes. Each participant was welcomed by the experimenter, signed a consent form, and received an in-
<table><caption>**Algorithm 1:** Annotation grouping algorithm</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>Input:</td><td>annoList, radius</td></tr><tr><td>Output:</td><td>annoGroupList</td></tr><tr><td>1</td><td>Function GroupAnnotations (annoList, radius)</td></tr><tr><td>2</td><td>minDist ← FindShortestDistance (annoList);</td></tr><tr><td>3</td><td>while minDist &lt; radius do</td></tr><tr><td>4</td><td>closestAnnos ← FindTwoClosestAnnotations (annoList);</td></tr><tr><td>5</td><td>midpoint ← ComputeCenter (closestAnnos);</td></tr><tr><td>6</td><td>annosWithinRadius ← FindUngroupedWithinRadius (annoList, midpoint, radius);</td></tr><tr><td>7</td><td>groupCenter ← ComputeCenter (annosWithinRadius);</td></tr><tr><td>8</td><td>annoGroupList.append(groupCenter);</td></tr><tr><td>9</td><td>RemoveGroupedAnnotations (annoList, annosWithinRadius)</td></tr><tr><td>10</td><td>/* Update the minDist */</td></tr><tr><td>11</td><td>minDist ← FindShortestDistance (annoList);</td></tr><tr><td>12</td><td>return annoGroupList;</td></tr></tbody></table>

introduction of the system at the beginning. They also filled out a pre-experimental questionnaire on demographic information.

Once they were equipped with the VR headset, we launched a training session so they could familiarise themselves with the use of the headset and the two controllers, as well as the system with its functions and different interactions. Next, the participant started the evaluation in one of the two sets of conditions (*No-Grouping* or *Grouping*) in a counter-balanced order. Within each set (C1-C2-C3 or C4-C5-C6), the participant performed three tasks in a random order following the three variations of *SELECTION TECHNIQUE*.

Within each condition Ci, the participant had to complete a task of finding four annotations based on the hints. After the task was completed, they answered two questionnaires (NASA-TLX and SUS), which were integrated directly in the virtual scene based on [25]. They were also prompted to take a break if needed. After having finished all the six conditions, the participant rated them according to the 7-point Likert scale.

## 5.7 Data Collection

We registered 432 trials from 3 *SELECTION TECHNIQUE* × 2 *GROUP* × 4 repetitions × 18 participants. For each trial, we collected the following data:

- *Task Completion Time (TCT)*: The time elapsed between the moments when a hint was given and when its corresponding target annotation was found.

- *Errors*: The number of errors that the participant made when identifying the wrong annotation from the given hint.

- *Movements*: The position of the participant in the virtual scene during the task.

At the end of each condition, we collected the answers from two subjective questionnaires: NASA-TLX [31] and SUS [41]. The NASA-TLX questionnaire's items were rated on a 20-point scale and were converted to a 100-point score.
At the end of each experiment, we also recorded user preference with the
rating of all the conditions using 7-point Likert scale.

5.8 Statistical Results

<figure><img src="image_8.png" /><figcaption>Fig. 6: Mean **TCT** by **SELECTION TECHNIQUE × GROUP** (left). Mean **TCT** in seconds aggregated by **SELECTION TECHNIQUE** (center) and **GROUP** (right). Error bars show 95% CI.</figcaption></figure>

**Task Completion Time (TCT) and Errors** The TCT to find four annotations for each condition was measured and analysed. QQ-plots and Shapiro-Wilk tests revealed a non-normality in the data, so log transformation was used (cf. [36] p.316). Using two-way repeated-measures ANOVA statistical analysis, we found no significant interaction effect between the two factors **SELECTION TECHNIQUE × GROUP** on the completion time ($F_{2,34} = 0.77, p = 0.469$) (Fig. 6-left). Therefore, we run the test for the main effect of those two factors and found a significant independent effect from both of them on TCT: **SELECTION TECHNIQUE** ($F_{2,34} = 5.703, p = 0.007$) and **GROUP** ($F_{1,17} = 7.890, p = 0.012$).

Reported means include their standard deviations. For **SELECTION TECHNIQUE**, post-hoc Tukey HSD tests revealed that *Proximity* (M = 48.54s±27.11) was significantly faster compared to *Active-Selection* (M = 62.39s±30.13) (p = 0.03). However, there were no significant differences between *Proximity* and *Selection-Proximity* (M = 60.14s±31.17) (p = 0.09), and *Active-Selection* and *Selection-Proximity* (p = 0.90) (Fig. 6-left).

For **GROUP**, post-hoc Tukey HSD tests indicated that the TCT is significantly shorter in *No-Grouping* (M = 49.99s±23.23) than *Grouping* (M = 64.06s±34.07) (p = 0.01) (Fig. 6-right).

The number of **errors** for every condition was very low ($M = 0.91, σ = 0.32$). A Friedman test on the errors revealed no significant difference between the conditions ($χ^2(2) = 9.10, p = 0.1$).

**Movements** Fig. 7-left shows some exemplary paths traced by different participants while searching for the annotations in six conditions. In the *Active-Selection* conditions, the participants tended to move a bit less, compared to the *Proximity* conditions. In the *Selection-Proximity* conditions, we can observe some mixed effects. In the *Grouping* conditions, the participants moved
<figure><img src="image_9.png" /><figcaption>Fig. 7: **Left:** Examples of the path per condition traced over time by different participants while searching for annotations. For each condition, four little triangles represent the position of the target annotations. The triangles have various colours representing different categories. The starting position was located at the top-left corner on the map **Right:** Average percentage of virtual locomotion (using joystick) in respect of overall movement. Error bars show 95% CI.</figcaption></figure>

a bit more, compared to the *No-Grouping* conditions. However, using two-way repeated-measures ANOVA analysis, we found no significant interaction effect between the two factors *SELECTION TECHNIQUE* × *GROUP* on the total covered distance (*F*<sub>2,34</sub> = 1.735, *p* = 0.192). The results of the test for the main effect of the two factors also showed no significantly independent effect: *SELECTION TECHNIQUE* (*F*<sub>2,34</sub> = 1.308, *p* = 0.284) and *GROUP* (*F*<sub>1,17</sub> = 2.616, *p* = 0.192).

**Questionnaires** The results of NASA-TLX showed no significant difference between conditions when being tested with Friedman test (χ²(2) = 5.038, *p* = 0.411). The highest value for each condition was *Performance* (M = 65.82/100).

With regard to SUS results, some combination of techniques obtained a score higher than 68 (considered to be acceptable, as being above the average [41]) for the following conditions: C4 *SelG* (68.33), C5 *ProxG* (71.52), C6 *SelProxG* (71.38), and C3 *SelProxNoG* (78.75). However, Friedman test revealed no significant difference between conditions (χ²(2) = 10.115, *p* = 0.072).

**User Preference** 12 out of 18 participants rated the C6 *SelProxG* as the better technique, followed by C3 *SelProxNoG* chosen by four participants.

A Wilcoxon signed-rank test [61] showed significant difference aggregating by *SELECTION TECHNIQUE*, between *Active-Selection* and *Selection-Proximity* (*Z* = 44.0, *p* < 0.001), and between *Proximity* and *Selection-Proximity* (*Z* = 43.5, *p* < 0.001). However, no significant difference was found between *Active-Selection* and *Proximity* (*Z* = 255.0, *p* = 0.647). In addition, no significant difference was found aggregating by *GROUP* between *Grouping* and *No-Grouping* (*Z* = 409.0, *p* = 0.309).

## 5.9 Qualitative Study with Archaeological Experts

Following the controlled experiment with laymen, we further validated the platform with archaeological experts. Due to their limited availability, the validation
took the form of a qualitative study. We invited a total of five archaeologists with different expertise. Two of them were part of the project and were involved in the design of the system. The other three were unaware of the goals of this research collaboration.

The evaluation phase spanned two sessions, each lasting one day. The ar-
chaeologists were first invited to examine multiple annotations created for the
controlled experiment using the various selection techniques and groups dis-
cussed above in the cognitive walk-through methodology [60]. As an example,
they were required to examine the annotations to identify the specific annotation
corresponding to a particular section. This task closely resembles the one car-
ried out with non-experts but was executed in a more smooth and streamlined
manner. Thinking-aloud methodology [9] was also employed, with each subject
gently prompted to express their thoughts during the evaluation. Specifically,
after each step, they discussed any details that were bothersome or about areas
where improvements could expedite the process. The second session involved in-
depth discussions when the experts provided comments and feedback to further
improve the system's overall effectiveness and functionality. The reflection with
experts shared also some remarks similar to the result of the controlled exper-
iment with non-experts. The quantitative and qualitative studies thus allowed
us to evaluate our system from two substantially different perspectives, which
can be combined to have a broader overview on the tool for the annotations. We
will detail in the upcoming discussion the results from the controlled experiment
with laymen and when pertinent, those from this qualitative study.

**5.10 Discussion**

With regard to user movement, **H1** cannot be fully confirmed. The users tended to move slightly more in the *Proximity* conditions compared to the *Selection*-based ones. When using the *Proximity* technique, the act of moving is preferred over the act of selection, which hastens the annotation search time, but increases the covered distance. This would also explain why in the conditions with the *Selection-Proximity*, the distance covered was longer than the *Active-Selection* conditions. We could thus come to the conclusion that avoiding the selection task is preferred by the users. However, during the tests, we observed that the users navigating in *Proximity* conditions had to walk around an annotation in order to read it clearly when other annotations were present nearby and occluded it.

It is also interesting to notice an evident preference for virtual navigation in-
stead of physical walking. This may be due to the fact that the limited size of the
space prevented the users walking the totality of the VR scene. However, there
was probably a tendency of them to perceive virtual locomotion by joystick as
easier or simpler than physical walking. It could be observed that they preferred
the virtual navigation when they had to reach far annotations, but tended to
walk to an annotation when they were closer to it. Video-games habits could
also have introduced this bias as half of the participants reported playing video
games at least once per week (cf. [7, 47]).
Based on the user preference result, we can confirm partly **H2** because the *Selection-Proximity* technique, with or without *Grouping*, was highly rated by the non-expert participants. In the qualitative study, the archaeologists also evidenced that the *Proximity* technique presented some occlusion problems, but *Selection-Proximity* was almost unanimously indicated as the favourite selection technique. They especially highlighted the need to prevent that the inspected annotations cover the visible space, particularly when it is the focus of an annotation.

In terms of task completion time, the *Proximity* technique was significantly faster than *Active-Selection* and slightly faster than *Selection-Proximity*. This result is not really surprising because the participants must be more active when they performed selections, and thus spent more time doing that. However, according to the participants’ feedback, the *Selection-Proximity* has the benefit of allowing them to close off detailed annotations when they are nearby, to avoid the distraction caused by unrelated information.

From the results of the SUS questionnaire, the best emerged technique was the *Selection-Proximity* with *No-Grouping* (i.e. *SelProxNoG*), followed by the *Selection-Proximity* with *Grouping* (i.e. *selProxG*). Nonetheless, most of the participants (12) expressed their preference for the version with grouping *SelProxG*, despite this being ranked second in the SUS questionnaire.

From the subjective questionnaires and user preference rating, we cannot confirm **H3** about the *Grouping* conditions. The benefits of spatial grouping were not verified either in terms of usability and task completion time. The problem may lie in the fact that ungrouping the annotations that belong to different categories demands an intermediate interaction step from the participants. This extra step might distract them from the task, although the visual space was more organised. Grouping would likely shine with an even higher number of annotations concentrated in a even small area, such as around a column, where it could effectively manage increased complexity and reduce visual clutter. One participant commented: “*I like the combination of proximity and selection. I don’t like grouping because you always have to unfold to see the inside.*” Discussing with the archaeologists, additional enhancements for grouping were also suggested, including the ability to focus on a single annotation within an opened group. Filters were also put forward to allow the users to selectively display one or a few categories, enabling concentrated attention on specific categories throughout the workflow.

The hints for the annotations seemed to be clear for the participants with no expertise in archaeology. We did not find any technique that caused a higher number of errors from all the conditions. Compared to non-specialist participants, the experts accomplished the task of finding annotations much faster. This efficiency can be attributed to two main factors: their ability to interpret the excavation site and orient themselves, coupled with a clear understanding of the relevant terminology. What proved instantly advantageous was the seamless integration into the thinking workflow, offering an immersive experience that is easily incorporated into daily work. The archaeologists could immediately con-
centrate on annotation content rather than grappling with how to use the LoD
system annotations. They could swiftly identify inaccuracies or areas requiring
further archaeological investigation.

Another suggestion from the archaeologist was that LoD 3 and above, which provide an extended description of the annotation, should only be accessible through *Active-Selection* and not via *Proximity* criteria. They also suggested that the first LoD can be a more condensed representation of an annotation such as a point in 3D space rather than a colored box with the initial letter of the corresponding category.

6 Conclusion and Future Work

3D annotations supporting systems are a challenging issue in many immersive
 analytics applications using Extended Reality (XR) technologies. When appli-
cation fields focus on structured data (e.g., computer-aided design data for ar-
chitecture and building [3], molecular 3D representations in biology [32], etc.),
the structure of the data is frequently adapted in real time to the visual repre-
sentation of 3D annotations. In the archaeology domain, data is not necessarily
fully unstructured because many archaeological objects can be identified accord-
ing to some domain taxonomies. However, a number of objects analysed during
excavations can be spatially found in scattered locations due to geophysical phe-
nomena or human activities over time. Starting from these premises, the first
contribution of this paper has been to propose a 3D annotation model for ar-
chaeologists, which allows adaptive visualisation of 3D annotations, regardless
of the underlying domain structure of the data. This 3D annotation framework
especially proposes three basic Levels of Detail (LoDs) - taxonomy-based label,
annotation title, and annotation description. These LoDs can be enlarged with
additional levels such as “Group of annotations”. Additionally, we proposed three
activation techniques of these LoDs: the first is based on the distance of the user
from the annotation (proximity), the second is the active selection of LoDs, and
the last is the combination of the two.

Thanks to this framework, our second contribution has been to conduct a
controlled experiment with non-experts and a qualitative study with experts to
compare the advantages and drawbacks of the LoD activation techniques, and
the possible added value of less detailed LoDs (i.e. grouping of annotations). In
the controlled experiment, only the three basic LoDs have been deployed be-
sides the grouping. It helped the participants focus on a sufficient content in 3D
annotations. It made it feasible for them to perform exploration and informa-
tion retrieval tasks for a large number of 3D annotations. In addition, we could
compare and complement the results with a qualitative study involving experts
in archaeology. This study enabled us to gain more insights on the effectiveness
of our platform, and enquire how we could further expand and integrate it as a
professional tool for experts.

From the controlled experiment, we found that annotations of different cat-
egories grouped together based on spatial relationship could help decluttering
the scene. However, since spatial grouping did not necessarily aggregate similar
information, a semantic approach could be more relevant, especially with the use of ontologies and knowledge models. This method could lead to automatic information extraction of the annotations directly from 3D reconstruction phase.

Concerning the activation techniques, proximity approach was very effective in terms of time efficiency, and its combination with the active selection was proved to be successful and gained the approval of archaeologists. Nonetheless, it could be perfected by incorporating aid-based approach such as eye-dwelling technique to isolate the information from the annotation that is sought after, and thus avoiding cluttering in dense annotation areas.

Moreover, despite the fact that the technique with active selection and proximity without grouping emerged as the best from the System Usability Scale questionnaire, the users subjectively preferred the technique with grouping, corroborating the positive effect that grouping could produce. In future work, the idea of grouping should be rethought and integrated with semantic categories and hierarchically organised data rather than spatial dependencies as previously discussed. Similarly, for different LoDs, it would be interesting to automatically manage them depending on their actual content. For instance, we could use natural language processing and machine learning to summarise the detailed information for better visibility. Additionally, multimedia information such as images, audio recordings, etc. could be visualised in a more dynamic layout if there are many of them.

We also obtained other valuable feedback from the non-expert participants and archaeologists. The proximity technique was very appreciated for not having to actively select an annotation to open its LoDs. This could compensate for the fatigue and frustration from the users when they often unsuccessfully select an object due to Heisenberg effect of spatial interaction [11]. We could also design the representation of annotations to slightly increase its size when pointed at, as remarked by one of the participants. Yet there is some limit to this method, especially when there are too many annotations present in the scene. Overall, the participants found the experiment tasks to be fun and engaging. The archaeologists also thought it to be useful and suitable to be integrated into their daily workflow. In the future, we would like to evaluate the full system, for a next iteration, extended and improved on the basis of the presented results with a higher number of archaeologists and experts in this domain. In addition, we would need to adapt such a platform to support collaboration between multiple users, either in remote or co-located context. Finally, our platform could also be suitable for Augmented Reality applications with some minor adaptations.

**Acknowledgments.** This work was supported by French government funding managed by the National Research Agency under the Investments for the Future program (PIA) with the grant ANR-21-ESRE-0030 (CONTINUUM project) and the grant ANR-23-CE33-0008-01 (MuseoXR project), and for the first author, by PhD grants from Région Ile-de-France and Noovae company. We also wanted to express warm thanks to Jean-Baptiste Houal and Frédérique Marchand-Beaulieu from AOROC.
## References

1. Assenmacher, I., Hentschel, B., Ni, C., Kuhlen, T., Bischof, C.: Interactive Data Annotation in Virtual Environments. In: Eurographics Symposium on Virtual Environments. pp. 119–126. The Eurographics Association, (2006)

2. Attene, M., Robbiano, F., Spagnuolo, M., Falcidieno, B.: Characterization of 3D Shape Parts for Semantic Annotation. Computer-Aided Design 41(10), 756–763 (2009)

3. Aubry, S., Thouvenin, I., Lenne, D., Olive, J.: A Knowledge Model to Read 3D Annotations on a Virtual Mock-Up for Collaborative Design. In: 2007 11th International Conference on Computer Supported Cooperative Work in Design. pp. 669–674 (Apr 2007)

4. Bach, B., Sicat, R., Beyer, J., Cordeil, M., Pfister, H.: The Hologram in My Hand: How Effective is Interactive Exploration of 3D Visualizations in Immersive Tangible Augmented Reality? IEEE Transactions on Visualization and Computer Graphics 24(1), 457–467 (2018)

5. Ballendat, T., Marquardt, N., Greenberg, S.: Proxemic interaction: designing for a proximity and orientation-aware environment. In: ACM International Conference on Interactive Tabletops and Surfaces. p. 121–130. ITS ’10, Association for Computing Machinery, New York, NY, USA (2010). https://doi.org/10.1145/1936652.1936676, https://doi.org/10.1145/1936652.1936676

6. Barbier, J., Kenny, P., Young, J., Normand, J.M., Keane, M.T., O’Sullivan, M., Ventresque, A.: MAAP Annotate: When archaeology meets augmented reality for annotation of megalithic art. In: 2017 23rd International Conference on Virtual System and Multimedia (VSMM). pp. 1–8 (2017)

7. Boletsis, C., Cedergren, J.: VR Locomotion in the New Era of Virtual Reality: An Empirical Comparison of Prevalent Techniques. Advances in Human-Computer Interaction 2019, 1–15 (Apr 2019)

8. Boletsis, C., Kongsvik, S.: Controller-based text-input techniques for virtual reality: An empirical comparison. International Journal of Virtual Reality 19 (10 2019). https://doi.org/10.20870/IJVR.2019.19.3.2917

9. Boren, T., Ramey, J.: Thinking aloud: Reconciling theory and practice. Professional Communication, IEEE Transactions on 43, 261 – 278 (10 2000). https://doi.org/10.1109/47.867942

10. Borhani, Z., Sharma, P., Ortega, F.R.: Survey of annotations in extended reality systems. IEEE Transactions on Visualization and Computer Graphics pp. 1–20 (2023). https://doi.org/10.1109/TVCG.2023.3288869

11. Bowman, D., Wingrave, C., Campbell, J., Ly, V.: Using Pinch Gloves for both Natural and Abstract Interaction Techniques in Virtual Environments. Proceedings of HCI International pp. 1–5 (Jan 2001)

12. Bruno, F., Barbieri, L., Marino, E., Muzzupappa, M., D’Oriano, L., Colacino, B.: An Augmented Reality Tool to Detect and Annotate Design Variations in an Industry 4.0 Approach. The International Journal of Advanced Manufacturing Technology 105, 1–13 (Nov 2019)

13. Clark, J.H.: Hierarchical Geometric Models for Visible Surface Algorithms. Communications of the ACM 19(10), 547–554 (1976)

14. Clergeaud, D., Guitton, P.: Design of an Annotation System for Taking Notes in Virtual Reality. In: 2017 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON). pp. 1–4 (2017)
15. Cockburn, A., Karlson, A., Bederson, B.B.: A Review of Overview+detail, Zooming, and Focus+context Interfaces. ACM Computing Surveys 41(1), 1–31 (Jan 2009)

16. Cordeil, M., Cunningham, A., Dwyer, T., Thomas, B.H., Marriott, K.: ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation. In: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. p. 71–83. UIST ’17 (2017)

17. De Luca, L.: Aïoli, une plateforme d'annotation sémantique 3D pour la documentation collaborative d'objets patrimoniaux. Culture et recherche (May 2018)

18. Doerr, M., Felicetti, A., Hermon, S., Hiebel, G., Kritsotaki, A., Masur, A., May, K., Ronzino, P., Schmidle, W., Theodoridou, M., et al.: Definition of the CRMarchaeo. An Extension of CIDOC CRM to Support the Archaeological Excavation Process (2018)

19. Doerr, M., Ore, C.E., Stead, S.: The CIDOC Conceptual Reference Model-A new Standard for Knowledge Sharing ER2007 tutorial. In: 26th International Conference on Conceptual Modeling (ER 2007) (2007)

20. Dominic, J., Robb, A.: Exploring Effects of Screen-Fixed and World-Fixed Annotation on Navigation in Virtual Reality. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 607–615 (Mar 2020)

21. Dongxing, C., Sangwook, K.: Augmented Reality Annotation for Real-Time Collaboration System. Journal of Korea Multimedia Society 23(3), 483–489 (Mar 2020)

22. Du, J., Zou, Z., Shi, Y., Zhao, D.: Zero latency: Real-time synchronization of BIM data in virtual reality for collaborative decision-making. Automation in Construction 85, 51–64 (2018). https://doi.org/10.1016/j.autcon.2017.10.009, https://www.sciencedirect.com/science/article/pii/S0926580517309172

23. Engelke, U., Hutson, H., Nguyen, H., de Souza, P.: MelissAR: Towards Augmented Visual Analytics of Honey Bee Behaviour. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. p. 2057–2063. CHI EA ’16 (2016)

24. Fanini, B., Ferdani, D., Demetrescu, E., Berto, S., d'Annibale, E.: ATON: An Open-Source Framework for Creating Immersive, Collaborative and Liquid Web-Apps for Cultural Heritage. Applied Sciences 11(22) (2021). https://doi.org/10.3390/app112211062, https://www.mdpi.com/2076-3417/11/22/11062

25. Feick, M., Kleer, N., Tang, A., Krüger, A.: The Virtual Reality Questionnaire Toolkit. UIST ’20 Adjunct, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3379350.3416188, https://doi.org/10.1145/3379350.3416188

26. Frécon, E., Nöu, A.A.: Building Distributed Virtual Environments to Support Collaborative Work. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology. p. 105–113. VRST ’98 (1998)

27. García-Pereira, I., Gimeno, J., Morillo Tena, P., Casanova-Salas, P.: A Taxonomy of Augmented Reality Annotations. pp. 412–419 (Jan 2020). https://doi.org/10.5220/0009193404120419

28. Gauglitz, S., Nuernberger, B., Turk, M., Höllerer, T.: World-Stabilized Annotations and Virtual Scene Navigation for Remote Collaboration. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software
and Technology. p. 449–459. UIST ’14, Association for Computing Machin-
ery, New York, NY, USA (2014). https://doi.org/10.1145/2642918.2647372,
https://doi.org/10.1145/2642918.2647372

29. Guerreiro, J., Medeiros, D., Mendes, D., Sousa, M., Jorge, J., Raposo, A., Santos, I.: Beyond Post-It: Structured Multimedia Annotations for Collaborative VEs (Jan 2014)

30. Harmon, R., Patterson, W., Ribarsky, W., Bolter, J.: The virtual annotation sys-
tem. In: Proceedings of the IEEE 1996 Virtual Reality Annual International Sym-
posium. pp. 239–245 (1996)

31. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, Advances in Psychology, vol. 52, pp. 139–183. North-Holland, (1988). https://doi.org/https://doi.org/10.1016/S0166-4115(08)62386-9, https://www.sciencedirect.com/science/article/pii/S0166411508623869

32. Hoksza, D., Gawron, P., Ostaszewski, M., Schneider, R.: MolArt: A molecular structure annotation and visualization tool. Bioinformatics 34(23), 4127–4128 (Jun 2018). https://doi.org/10.1093/bioinformatics/bty489

33. Hunter, J., Gerber, A.: Harvesting community annotations on 3D models of mu-
seum artefacts to enhance knowledge, discovery and re-use. Journal of Cultural
Heritage 11 (Mar 2010). https://doi.org/10.1016/j.culher.2009.04.004

34. Jung, T., Gross, M.D., Do, E.Y.L.: Annotating and Sketching on 3D Web Models. In: Proceedings of the 7th International Conference on Intelligent User Interfaces. p. 95–102. IUI ’02 (2002)

35. Kalkofen, D., Sandor, C., White, S., Schmalstieg, D.: Visualization Techniques for Augmented Reality, pp. 65–98. Springer, New York (2011)

36. Kaptein, M., Robertson, J.: Modern Statistical Methods for HCI (Apr 2016).
https://doi.org/10.1007/978-3-319-26633-6

37. Koppel, T., Eduard Groller, M., Wu, H.Y.: Context-responsive labeling in aug-
mented reality. In: 2021 IEEE 14th Pacific Visualization Symposium (Paci-
ficVis). IEEE, (Apr 2021). https://doi.org/10.1109/pacificvis52677.2021.00020,
http://dx.doi.org/10.1109/PacificVis52677.2021.00020

38. Krus, M., Bourdot, P., Guisnel, F., Thibault, G.: Levels of Detail Polygonal Simplification. XRDS 3(4), 13–19 (May 1997). https://doi.org/10.1145/270955.270966, https://doi.org/10.1145/270955.270966

39. Lenne, D., Thouvenin, I., Aubry, S.: Supporting design with 3D-annotations in a collaborative virtual environment. Research in Engineering Design 20, 149–155 (Sep 2009). https://doi.org/10.1007/s00163-009-0071-8

40. Lercari, N., Shiferaw, E., Forte, M., Kopper, R.: Immersive Visualization and Cu-
ration of Archaeological Heritage Data: Çatalhöyük and the Dig@IT App. vol. 25
(Apr 2018). https://doi.org/10.1007/s10816-017-9340-4

41. Lewis, J.R.: The System Usability Scale: Past, Present, and Fu-
ture. International Journal of Human-Computer Interaction 34(7),
577-590 (2018). https://doi.org/10.1080/10447318.2018.1455307,
https://doi.org/10.1080/10447318.2018.1455307

42. Loughlin, M., Hughes, J.: An annotation system for 3D fluid flow isualization. In: Proceedings Visualization ’94. pp. 273–279 (1994)

43. Luo, W., Lehmann, A., Widengren, H., Dachselt, R.: Where Should We Put It? Layout and Placement Strategies of Documents in Augmented Reality for Collaborative Sensemaking. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. CHI ’22, Association for Computing Ma-
chinery, New York, NY, USA (2022). https://doi.org/10.1145/3491102.3501946
https://doi.org/10.1145/3491102.3501946

44. Manferdini, A.M., Remondino, F.: Reality-Based 3D Modeling, Segmentation and Web-Based Visualization. pp. 110–124 (Nov 2010). https://doi.org/10.1007/978-3-642-16873-4_9

45. Manuel, A., M'Darhri, A., Abergel, V., Rozar, F., Luca, L.: A semi-automatic 2D/3D annotation framework for the geometric analysis of heritage artefacts. In: 3rd Digital Heritage International Congress (DigitalHERITAGE) held jointly with 2018 24th International Conference on Virtual Systems and Multimedia (VSMM 2018). pp. 1–7 (Oct 2018). https://doi.org/10.1109/DigitalHeritage.2018.8810114

46. Manuel, A., Véron, P., De Luca, L.: 2D/3D Semantic Annotation of Spatialized Images for the Documentation and Analysis of Cultural Heritage. In: Catalano, C.E., Luca, L.D. (eds.) Eurographics Workshop on Graphics and Cultural Heritage. The Eurographics Association, (Oct 2016). https://doi.org/10.2312/gch.20161391

47. Nabiouni, M., Saktheeswaran, A., Bowman, D., Karanth, A.: Comparing the Performance of Natural, Semi-Natural, and Non-Natural Locomotion Techniques in Virtual Reality (Mar 2015). https://doi.org/10.1109/3DUI.2015.7131717

48. Nuernberger, B., Lien, K.C., Höllerer, T., Turk, M.: Interpreting 2D gesture annotations in 3D augmented reality. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI). pp. 149–158 (2016). https://doi.org/10.1109/3DUI.2016.7460046

49. Pagé, C., Bernier, P.M., Trempe, M.: Using video simulations and virtual reality to improve decision-making skills in basketball. Journal of Sports Sciences 37(21), 2403–2410 (2019). https://doi.org/10.1080/02640414.2019.1638193, https://doi.org/10.1080/02640414.2019.1638193, pMID: 31280685

50. Pedersen, E., McCall, K., Moran, T., Halasz, F.: Tivoli: An Electronic Whiteboard for Informal Workgroup Meetings. pp. 391–398 (Jan 1993). https://doi.org/10.1145/169059.169309

51. Pick, S., Gebhardt, S., Weyers, B., Hentschel, B., Kuhlen, T.: A 3D collaborative virtual environment to integrate immersive virtual reality into factory planning processes. In: 2014 International Workshop on Collaborative Virtual Environments (3DCVE). pp. 1–6 (2014)

52. Pick, S., Weyers, B., Hentschel, B., Kuhlen, T.W.: Design and Evaluation of Data Annotation Workflows for CAVE-like Virtual Environments. IEEE transactions on visualization and computer graphics 22(4), 1452–1461 (Apr 2016)

53. Pittarello, F.: An Architecture and a Visual Interface for Tagging the 3D Web. In: Proceedings of the International Conference on Advanced Visual Interfaces. p. 311–315. AVI ’10, Association for Computing Machinery, New York, NY, USA (2010). https://doi.org/10.1145/1842993.1843049, https://doi.org/10.1145/1842993.1843049

54. Rehring, K., Greulich, M., Bredenfeld, L., Ahlemann, F.: Let's Get in Touch - Decision Making about Enterprise Architecture Using 3D Visualization in Augmented Reality. In: Proceedings of the 52nd Annual Hawaii International Conference on System Sciences. pp. 8–11 (Jan 2019). https://doi.org/10.24251/HICSS.2019.215

55. Rijnsburger, W., Kratz, S.: Personalized presentation annotations using optical HMDs. Multimedia Tools and Applications 76 (Feb 2017). https://doi.org/10.1007/s11042-016-4064-0

56. Shneiderman, B.: The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings 1996 IEEE Symposium on Visual Languages. pp. 336–343 (1996)
57. Tatzgern, M., Orso, V., Kalkofen, D., Jacucci, G., Gamberini, L., Schmalstieg, D.: Adaptive information density for augmented reality displays. In: 2016 IEEE Virtual Reality (VR). pp. 83–92 (2016). https://doi.org/10.1109/VR.2016.7504691

58. Tudhope, D., Binding, C., Jeffrey, S., May, K., Vlachidis, A.: A STELLAR role for knowledge organization systems in digital archaeology. Bulletin of the American Society for Information Science and Technology 37(4), 15–18 (2011)

59. Verlinden, J., Bolter, J., Mast, C.: Virtual Annotation: Verbal Communication in Virtual Reality. NASA STI/Recon Technical Report N (Jan 1993)

60. Wharton, C., Rieman, J., Lewis, C., Polson, P.: The cognitive walkthrough method: a practitioner's guide, p. 105–140. John Wiley & Sons, Inc., USA (1994)

61. Wilcoxon, F.: Individual comparisons by ranking methods. In: Breakthroughs in Statistics: Methodology and Distribution, pp. 196–202. Springer, (1992)

62. Xi, N., Chen, J., Gama, F., Riar, M., Hamari, J.: The challenges of entering the metaverse: An experiment on the effect of extended reality on workload. Information Systems Frontiers (Feb 2022). https://doi.org/10.1007/s10796-022-10244-x

63. Yu, C.H., Hunter, J.: Documenting and Sharing Comparative Analyses of 3D Digital Museum Artifacts through Semantic Web Annotations. vol. 6. Association for Computing Machinery, New York, NY, USA (Dec 2013). https://doi.org/10.1145/2532630.2532634; https://doi.org/10.1145/2532630.2532634