# Augmented urban models: Investigating how data layout and link to the physical referent impact visual search in augmented reality

Mohamed Reda Benkhelifa, Emmanuel Dubois, Louis-Pierre Bergé, Marcos Serrano

<figure><img src="image_1.png" /></figure>

## To cite this version:

Mohamed Reda Benkhelifa, Emmanuel Dubois, Louis-Pierre Bergé, Marcos Serrano. Augmented urban models: Investigating how data layout and link to the physical referent impact visual search in augmented reality. International Journal of Human-Computer Studies, 2025, 203, pp.103556. 10.1016/j.ijhcs.2025.103556. hal-05514978

HAL Id: hal-05514978

https://hal.science/hal-05514978v1

Submitted on 17 Feb 2026

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Augmented urban models: Investigating how data layout and link to the physical referent impact visual search in augmented reality

Mohamed Reda Benkhelifa<sup>a,∗</sup> <sup>IDI</sup>, Emmanuel Dubois<sup>a,†</sup>, Louis-Pierre Bergé<sup>b,‡</sup>, Marcos Serrano<sup>a</sup>

<figure><img src="image_3.png" /></figure>

## ARTICLE INFO

**Keywords:**

Augmented reality

Physical referent

Visual search

## ABSTRACT

Recent studies in situated augmented reality have explored interactive urban physical 3D models providing a physical anchor and context to data analysis. See-through Head-Mounted Displays (HMDs) enhance these models by displaying virtual data around the buildings. However, the optimal positioning of multiple information relative to the physical referents is crucial yet has barely been explored. In this paper, we focus on arranging multiple data windows around a district model and present a design space describing the window layouts and linking techniques between the virtual data and its physical referent. We conduct three studies to evaluate these design dimensions for two fundamental visual search tasks: finding a data window for a given physical referent, and vice-versa. Our results reveal novel findings on the best combinations of window layout and link in terms of performance, mental workload and user preference. Finally, we propose a set of design guidelines for presenting situated immersive data around a physical urban model.

CCS CONCEPTS • Human-centered computing~Human computer interaction (HCI)~Interaction paradigms~AR

## 1. Introduction

The vision of Smart Cities involves many spatially situated sensors generating multiple and heterogeneous data of interest for various stakeholders with different expertise (Rhazal and Tomader, 2019). In addition, exploring the data in its geographical context reinforces the links between the spatial environment and the digital data (Maquil, May 2016), enhancing the user's awareness of the data's physical surroundings and facilitating memorization and proprioception (Taher et al., 2015). This type of data exploration can be performed on interactive urban physical 3D models, which have been demonstrated to be valuable tools (Alonso et al., 2018; Ens et al., 2021 Feb). Such models consist of 3D-printed buildings, usually at the scale of a district, combined with projected or augmented reality (AR) visualizations. Buildings serve as physical referents to access their related data in various contexts, such as urban planning, energy consumption monitoring, or crisis management. The model provides an essential global perspective on urban context and allows situated access to the associated data.

Previous works have explored various methods to implement physical urban models to support data browsing. Using a video projector can render the data directly on the model (Cityscope), but it is challenging

because of the freeform projection surface and potential occlusion by the user's hands. An alternative approach involves placing the 3D-printed buildings on a tabletop and displaying the data around them (Walker et al., 2017). However, this becomes impractical when the model comprises multiple buildings, as they occlude the screen and limit the visualization space. In UpLift (Ens et al., 2021 Feb), buildings can be grasped as tokens to access their data in AR. While AR seems the most promising visualization approach for augmented models, grasping the buildings is impractical for large models. Hence, combining AR with static buildings is a promising solution. However, it raises the challenge of where to place the multiple virtual windows around the physical model while preserving the link between the virtual information and its physical referent (Ens et al., 2021).

In this paper, we explore how to arrange 2D data windows in AR around a physical urban model and link them to the relevant elements of the urban model, i.e., their physical referent (see Fig. 1). We focus on the fundamental task of visually searching for digital data linked to a physical referent (or vice versa), which precedes any data analysis on the urban model (Zhou and Feiner, 1998). As demonstrated in a recent literature survey (Kraus et al., 2022), we decided to explore a visual search task as it is one of the most frequent analysis tasks used in

<footnote><sup>a</sup> University of Toulouse, IRIT, Toulouse, France</footnote>

<footnote><sup>b</sup> CEA, CEA Tech Occitanie, F-31670 Labège, France</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: mohamed.benkhelifa@irit.fr (M.R. Benkhelifa).</footnote>
immersive analytics. Kraus et al. (Kraus et al., 2022) stated that visual search is a “generic way to explore data in order to build new hypotheses”. It is also a task used frequently in dense and occluded environments, such as network visualization.

We first introduce a set of design dimensions focusing on two key aspects: the data layout and the link. The data layout comprises two dimensions: the location of the windows with respect to the model (distant, intermediate, or close) and the alignment of the virtual windows (grid or map-based). The link involves two dimensions: its form (direct, such as a line; indirect, such as a radar-view; or ID-based only) and its availability (permanent or on-demand).

To explore these numerous combinations of data layout and link, we conducted three studies to understand better the factors that influence visual search in such context. The first two studies were centered on visually searching for digital data, e.g., a graph, corresponding to a specific building. In the first study, we examined only distant layouts (i. e., the data windows are located behind the model), commonly considered in the literature (Marco Cavallo et al., 2019; Perelman et al., 2022). In a second study, we compared the best distant layouts with the two other window locations (intermediate or close). The third study tackled the opposite task, i.e., visually searching for the physical building corresponding to a specific digital graph. In this final study, we explored only the best combinations of data layout and link, guided by the results of the first two studies.

The results reveal novel findings, such as the superiority of a map-based alignment of virtual windows over the traditional grid alignment. Additionally, users appreciate the option to display a visual link between the referent and the associated window on-demand. However, simply displaying an ID next to the referent and window yields better performance with map-based window alignment. Our studies also emphasize the need to find a balance between window location and the potential occlusion of the model.

Our contributions include (1) a set of design dimensions for Situated Immersive Visualization around a physical model, (2) three studies investigating these dimensions for two tasks: visually searching for a graph related to a given building and vice-versa, and (3) a set of design guidelines for situated visualizations around physical urban models.

## 2. Related work

This section highlights the benefits and limitations of situated visualization. Namely, situated visualization displays information near their physical referent, improving users' understanding and analysis of related data. However, existing recommendations were studied in situations involving few referents only, all in the user's reachable space. In contexts such as crisis management or urban planning, situations may

scale up to the size of an urban city model representing an entire district, where each building becomes a physical referent: displaying information near each referent may then cause visual clutter. However, there are no appropriate guidelines on where and how to arrange situated visualizations around multiple physical referents. To address this research challenge, our work focuses on data arrangement around urban models and the methods to link the data to its referent.

### 2.1. From situated to prox situated visualization

The definitions of situated visualization provided by Willet et al. (Willet et al., 2016) and White and Feiner (White and Feiner, 2009) are among the most widely used in the literature. White and Feiner (White and Feiner, 2009) define situated visualization as a visualization that “is related to and displayed in its environment”. Willet et al. (Willet et al., 2016) expand this definition by introducing physical data referent to describe the physical entity associated with the data. Situated visualizations enhance contextual understanding (Kadek A. Satriadi et al., 2023; Willett et al., 2016) by enabling viewers to explore and extract additional information from the environment that may not be present in the dataset.

Various technologies have been employed for rendering situated visualization, including mobile devices (Alallah et al., 2020; Valkanova et al., 2015; Whitlock et al., 2019; Zollmann et al., 2016), surrounding screens (Wannamaker et al., 2021), and see-through Head-Mounted Displays (HMDs) (Lin et al., 2021; White et al., 2006; White and Feiner, 2009). Among these, HMDs that offer augmented reality (AR) are the dominant technology in situated visualization (Bressa et al., Jan. 2022). AR-HMDs enable seamless integration of virtual information into the physical environment by displaying it directly on or near its physical referent (Lin et al., 2021).

Alternatively, instead of presenting information in close proximity to the actual physical referent, proxies, such as physical models, can be employed to represent the real-world referent and display information at that level. This approach is called “ProxSituated visualization”, as proposed by Satriadi et al. (Kadek A. Satriadi et al., 2023).

### 2.2. Augmented physical models

While the term “ProxSituated Visualization” has recently emerged, combining augmented models with situated visualization has been the subject of prior research to gain insights into urban morphology and datasets (Mitasova et al., 2006). Augmenting a model can be achieved through various methods. One approach involves directly integrating sensors and/or lighting into buildings for interaction and visualization (Chan et al., 2012). However, this method adds complexity to model

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** We study an interactive physical district model for data monitoring, where buildings are physical referents associated with data windows displayed in AR. Although such situated visualizations are generally arranged in a grid (left), our studies show that a map-based alignment performs better (center). We study various layouts with different referent-data links(right).</figcaption></figure>
fabrication and may not be practical for large models. Another method involves using video projection to overlay data directly onto the model (Alonso et al., 2018), but this approach is challenged by potential occlusion from the user's body and arms. Tabletop displays offer an alternative but are constrained by limited display space around the buildings (Walker et al., 2017). In contrast, HMDs emerge as an optimal solution to display data around a building, as demonstrated in the UpLift system (Ens et al., 2021 Feb). Using HMDs avoids modifying the model and can display information anywhere on or around it.

Regarding the interaction with the model and its physical proxies, Satriadi et al. (Kadek Ananta Satriadi et al., 2023) distinguish between active proxies, where users can manipulate the physical scale model, and passive proxies, where the scale model remains static. They explored the use of active proxies in physical models, allowing the user to move or dwell on a building to interact with its visualization (Kadek Ananta Satriadi et al., 2023). However, active proxies are impractical in large urban models involving multiple buildings that may be out of reach. For this reason, our work focuses on augmenting models composed solely of passive proxies.

## 2.3. Virtual data arrangement in virtual and augmented reality

To further inform our work that aims to find the best arrangement of multiple virtual 2D windows around a 3D physical model, we reviewed prior work on data arrangement in virtual (VR) and augmented reality (AR). Ens et al. (Barrett Ens et al., 2014) introduced a conceptual framework for 2D information spaces within AR environments, which is centered on the perspective of the content (body-based or world-based coordinates) and the mobility of the content (moving or stationary). Frequently, an egocentric perspective is coupled with mobile content, such as in Personal Cockpit (Barrett M. Ens et al., 2014), Maps around me (Satriadi et al., 2020), or MultiFi systems (Grubert et al., 2015). Conversely, exocentric interfaces often involve world-fixed content placement, as seen in scenarios like AR virtual monitors around physical screens (Pavanatto et al., 2021), document organization for collaborative sensemaking (Luo et al., 2022), analysis of human motion data in situ (Luo et al., 2023) or the arrangement of small multiples visualizations (Liu et al., 2020). In addition, Liu et al. (Liu et al., 2020) highlighted the value of small multiples visualization in immersive environments: it benefits from the large display space to present a large amount of data simultaneously, facilitating quick data comparisons and fostering a more comprehensive understanding. Consequently, our research focuses on fixed, world-based small multiples visualizations, centered on the physical model. Considering this context, the following question is where to display the data around the model.

Recent VR studies on virtual data arrangement in front of users found that curved layouts perform similarly to flat layouts but are preferred by users because the data is easily accessible within arm's reach (Liu et al., 2020; Liu et al., 2022; Liu et al., 2023; Kadek A. Satriadi et al., 2023). However, whether these findings would still hold with a visualization centered on a physical model with a large amount of data remains unclear.

Among previous studies exploring an object-centered data arrangement in AR, Cavallo et al. (Marco Cavallo et al., 2019; M. Cavallo et al., 2019) developed a system with virtual screens positioned around a circular tabletop to display additional content. SnapToReality implemented an AR system that automatically aligns virtual content on the edges and surfaces of the real world (Nuernberger et al., 2016). In their literature review on augmented reality for digital twins, Künz et al. (Künz et al., 2022) established that in most cases, the data is superimposed on physical referent; they also underlined that little attention has been paid so far to the usability of such systems. Lee et al. (Lee et al., 2024) identified several design dimensions of situated visualization. Three of them are related to the spatial relationship between referent and visualization (named Situatedness): View Placement (situated or embedded), Spatial coordinate frame (shared, independent), and View

Coordinate Frame (referent, view, or camera). In our paper, we focus on situated, independent, and view-frame situatedness. Indeed, in our case, an embedded view placement would not be appropriate given the limited size of the referents; according to the authors, an independent frame is the traditional information visualization mode; finally, our approach displays the data in one fixed area, which is independent of the referent-frame and the camera-frame. The authors define another dimension called Cardinality, defining the numerical relationship between referents and views (One-to-one, One-to-many, Many-to-one, or Many-to-many): in our work, we focus on the One-to-one cardinality, the most basic form of situated visualization as stated by the authors, as a first step towards understanding such situations. We discuss the other options in the future work section. Wen et al. explored different automatic layout adaptation methods to position the data around the referent (Wen et al., Jan. 2023). However, their study only involved four to six virtual referents instead of physical referents.

Satriadi et al. explored the visualization of multivariate data around scale models (Satriadi et al., 2022). The experiment involved only five physical scale buildings and compared various multivariate data arrangements and display locations: behind the model, side-ways just above the model, and on the tabletop around the model. The data arrangement did not impact performance when comparing simple charts across buildings, but displaying them on the tabletop was the least preferred location. Perelman et al. (Perelman et al., 2022) investigated the impact of transitioning between a tabletop and the virtual information displayed around it. They demonstrated that displaying information at the back of the tabletop is more advantageous than on the sides. Applying and scaling these results to our context, a wider urban model, suggests that data should be displayed at the back of the model, relatively far away from its referent. This approach would require explicitly showing the link between the data and its referent.

## 2.4. Linking virtual data with its physical referent

When dealing with multiple situated data, additional visual cues become essential for aiding users in grasping the relationships between data and its referent. In immersive environments, various methods have been suggested to address this challenge. The most common approach involves using lines to link objects together (Prouzeau et al., 2019; Li et al., 2017; Cheng et al., 2021; Wieland et al., 2022), already demonstrated in the context of augmented physical models (Satriadi et al., 2022). While these links usually consist of straight lines (Hubenschmid et al., 2021), some works have explored different design options for these lines (Li et al., 2018), such as the length of the leader line, developed algorithms to adjust virtual object properties (e.g., position, size, transparency) and ensure their visibility when used in AR (Bell et al., 2001) and measured their positive impact on the user's efficiency (Wang et al., 2023). Others proposed more sophisticated curved links (Prouzeau et al., 2019; Wieland et al., 2022), albeit with limitations related to stability and scalability in their algorithms. A recent and extensive review of existing works of the literature on labeling techniques synthesizes different aspects, models, and algorithms for automatically placing external labels (Bekos et al., 2019): it highlights the lack of methods and measures to compare each approach and the lack of guidelines. Color has also been explored to establish links, as Vock et al. (Vock et al., 2021) demonstrated, where colored tabs on a smartphone correspond to colored windows displayed in an HMD. However, it is important to note that using color to encode links should be limited to seven or fewer distinct colors (Silva et al., 2011).

Another method to retrieve information involves using a radar view like in TDome (Saidi et al., 2017), which provides the user with the spatial positions of surrounding screens to facilitate their location. In immersive environments, Ihara et al. (Ihara and Kawaguchi, 2023) propose to use a miniature model of the environment in which various virtual or physical objects can be represented. The challenge across all these linking methods lies in their scalability when dealing with multiple
virtual or physical objects that need to be connected.

# 3. Augmented urban model: concept and design dimensions

## 3.1. Concept of the augmented urban model

As presented in the previous section, situated visualization enhances data exploration, but its scalability is still an open problem. Our work focuses on visualizing data from an urban physical 3D model, using passive proxies (model buildings), in a ProxSituated data visualization centered on the model. Data related to the referent may come from real-time sensors or simulations (Cureton and Dunn, 2021). To explore this data, users will refer to specific buildings (e.g., the school, the hospital, etc.) and their related data, one after the other. Hence, they will frequently transition from the physical referents to the digital information. We chose to explore the scenario where one data window is associated with one referent: each window can display several data, limiting the number of windows and reducing the visual clutter around the referent. We use HMDs, a prominent AR technology in this context, and propose small multiple visualizations displayed around the model to effectively present the extensive data. Finally, we integrate links connecting the data with the model to strengthen data-referent correlation.

Given the lack of guidelines in the literature on situated immersive visualization around multiple physical referents, our work explores the open research questions of the optimal arrangement of data windows around the model and the most effective presentation of links to help users locate the data window for a specific building, and vice-versa.

## 3.2. Design dimensions

We derived four design dimensions through an iterative design thinking process conducted by three senior and one junior researchers in immersive visualization, HCI, and mixed reality. Two dimensions describe the data layout (window location and alignment), while the other two describe the link between the referent and its data (form and availability), as illustrated in Fig. 2.

### 3.2.1. Window location

The window location dimension determines the distance between

the virtual window and its physical referent in the urban model, i.e., a single building. We considered three window locations: close, distant, and intermediate. For the sake of simplicity, we will refer to these as close/distant/intermediate layouts. With **close** layouts, the window is positioned very near its physical referent, reducing the time to access the data. However, this layout can lead to occlusion and visual clutter when dealing with multiple windows and referents. With **distant** layouts, each data is displayed away from its referent, but they are all grouped around the physical model. Prior work (Perelman et al., 2022) on combining a tabletop and mixed reality suggests that the optimal approach is to use the vertical plane facing the user behind the physical model. Finally, **intermediate** layouts consist of varying the window position, between close and distant, according to the position of their referent. The goal is to minimize the distance between the referent and its data window while maximizing its visibility.

### 3.2.2. Window alignment

We consider two types of window alignments: grid and map. **Grid alignment** organizes data windows into rows and columns, potentially resulting in a mismatch between the window position and the referent's location on the urban model. Consequently, a grid layout necessitates a thoughtful ordering of the windows, which could be based on factors like the building's name, ID, or other filtering criteria (e.g., data type or value). **Map alignment** positions data windows to accurately mirror the geographical position of their referents on the model. This design dimension is bounded by constraints such as map size, window dimensions, and the number of buildings.

### 3.2.3. Form of link

We envision three forms of links between windows and their referents: direct, indirect, or ID-only. The **direct link** explicitly connects a digital window to its referent. A line is the prevalent type of direct link in literature (Hubenschmid et al., 2021). The **indirect link** is a graphical representation designed to help the user spatially locate the data window for a given referent without explicitly linking them. A common example in literature is the use of a radar view (Saidi et al., 2017), representing the general area where the digital window is located. Finally, the minimal link between the referent and its window is usually based solely on their IDs, i.e., an ID displayed next to each referent and

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Design Dimensions explored in this paper. Window location characterizes the distance between the physical referent and its data window. Window alignment structures the way data windows are laid out. Form of link describes how the link representation connects a physical referent and its data window. Availability characterizes the persistence of the link representation.</figcaption></figure>
window (also present for direct and indirect links). In this case, the user must visually scan the visualization space to find a window with a specific ID.

### 3.2.4. Link availability

We consider links that are permanent or only available on demand. **Permanent links** are always displayed for all referent-window pairs, ensuring that users can always locate the 2D window corresponding to a given referent. However, in a physical model with multiple buildings, displaying all the links may result in visual clutter, making it difficult to see or follow each link. **On-demand links** are only displayed in response to an explicit user action, such as touching or looking at a building. While this solution mitigates the visual clutter associated with permanent links, it requires an explicit user action, which can consume time. Moreover, depending on the size and density of the physical model, selecting distant buildings may be difficult.

## 4. Experimental exploration of our design dimensions

We used an Augmented Urban Model and these design dimensions to drive our experimental exploration through three incremental experiments. Our approach was to progressively and systematically explore the design dimensions on the urban physical model, using the results of each experiment to refine and reduce the number of conditions explored in the subsequent experiments. We hereafter present the physical model, the experimental tasks, and an overview of our experimental approach.

### 4.1. Construction of the urban model

We constructed the urban physical model using Lego bricks (see Fig. 3-left). The entire model measured 110 cm x 80 cm and contained 48 identical rectangular buildings, each made of 9 Lego bricks, eliminating potential biases associated with building shapes. We used a large number of buildings representing a relatively large district or university campus to ensure the external validity of our study. To evenly distribute the buildings across the model, we divided the model into two halves (top and bottom), each containing 24 buildings.

We built a digital twin of the physical model to precisely position each building and to implement the user interaction. We used the Metabricks website to create a 3D Lego model of a building, imported the model into Blender, replicated it 24 times, and used Blender's randomization tool to position these 24 buildings in one-half of the physical model. We repeated this process two times to fill the two halves of the model.

Each building had a unique number ID, permanently displayed **above it and on its associated data window**.

### 4.2. Visual search tasks

We decided to explore a visual search task as it is one of the most frequent analysis tasks used in immersive analytics, as demonstrated in a recent literature survey (Kraus et al., 2022). In their survey, Kraus et al. underline that visual search supports data exploration and hypothesis

generation and is useful for interactive visualization in complex environments. We considered two different visual search tasks in the context of our augmented urban model:

- **The first task** consists in visually *finding the virtual window* associated with a given physical referent from the urban model (see Fig. 3-center). We used this task in the first two experiments as it is a common task in situated analytics. In our context, this task corresponds to finding the virtual window for a building.

- **The second task** is the opposite and consists in visually *finding the physical referent* for a given digital data window (see Fig. 3-right). This task is also interesting in our augmented model: inspecting the small-multiples visualization (i.e., all the data windows) can visually reveal anomalous data values on a window. In this case, the data analyst needs to locate the associated building on the physical map to gain contextual insight into the data (i.e., a peak in the flood sensor readings could be explained by the building's proximity to a river or a flood-prone area). We used this task in our last experiment.

### 4.3. Dimensions exploration across our three studies

We explored our design dimensions in three consecutive studies, as illustrated in Fig. 4:

**Study 1 (task: finding a virtual window):** the goal of the first study is to focus only on distant layouts and investigate the impact of the form of the link, link availability, and window alignment. We decided to focus only on distant layouts as they are the most frequent layouts in the literature (Marco Cavallo et al., 2019; Perelman et al., 2022). Therefore, we considered twelve conditions resulting from the combination of the forms of links, link availabilities, and window alignments.

**Study 2 (task: finding a virtual window):** the goal of the second study is to explore the different window locations (close, intermediate, distant) using the two best data-referent links identified in Study 1. To this end, we studied two distant layouts, three intermediate layouts, and one close layout. This results in the comparisons of 12 different conditions.

**Study 3 (task: finding a physical referent):** the goal of the last study is to extend the results of our previous studies to the opposite visual search task. We perform this study on a subset of seven conditions among those considered in study 2, i.e., the best-performing conditions for the first task.

## 5. Protocol of our studies

### 5.1. Task execution

*Finding a virtual window task (studies 1 and 2)*: As described above, in the first two studies, the user's task was to locate the virtual window for a given building within the layout. With the Grid layout, to avoid a bias in memorizing the position of the windows in the Grid across conditions, we decided to randomly affect a number ID between 1 and 200 for each

<figure><img src="image_6.png" /><figcaption>**Fig. 3.** Our urban physical model contained 48 buildings (left). We explored two visual search tasks (illustrated with OnDemand-Direct link): finding the virtual window associated with a given physical building (center) and finding the physical building for a given data window (right).</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>**Fig. 4.** Overview of our experimental approach for exploring our design dimensions, highlighting the dimensions explored in each of the three studies.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Task 2:** Retrieving the referent related to one window</figcaption></figure>

building, and we changed the buildings' ID between conditions. Concerning the second layout, the Map layout, since it mirrors the urban model, randomizing the ID would not add any bias in memorizing the position.

Participants were instructed to find a balance between task completion speed and comfort by completing the task as quickly as possible without compromising their comfort. For both OnDemand and Permanent conditions, to start a trial, the participant had to gaze at a "start trial" virtual button displayed at the bottom of the tabletop and, while looking at it, press a clicker (Fig. 5- left). Then, all the graphics were displayed: the ID of each building was displayed above them, and the *target building* was highlighted. The feedback for the rest of the trial differed for OnDemand and Permanent links.

For OnDemand links, the task is a two-step process. First, the user had to locate the trial's target *building*, which was highlighted in blue (Fig. 5-OD1). Once gazing at this *building*, the user could validate the selection with a clicker (Fig. 5-OD2). Following a correct selection in this first step, the *building* turned green (Fig. 5-OD3); otherwise, it turned red. Furthermore, the *link* to the *associated window* was displayed, and the *target window* was highlighted in green (Fig. 5-OD3). Once gazing at the *target window*, the participant could use the clicker to validate the selection (Fig. 5-OD4).

For Permanent links, the *target building* of the trial was directly highlighted in green after the start button was pressed, and all the *links* for each *building/graph* pair were displayed (Fig. 5-P1). The user had to locate the *building* to find the *associated window* but did not need to gaze and click at it, as with OnDemand links. The user could only rely on the ID and the form of the link to locate the *associated window*, which was not highlighted. The rationale for not highlighting the window is that this condition represents a visual search without explicit validation or interaction; hence, the target window is not known in advance and cannot be highlighted. When a *window* was gazed at, the participant could use the clicker to validate the selection (Fig. 5-P2).

With both forms of links, if the participant selected the wrong

window, the window turned red, and the task continued until *s/he* selected the right window to end the trial.

*Finding a physical referent task (study 3)*: In the third study, we explored the opposite task, i.e., finding a building on the physical model for a given window. Therefore, the task instructions, procedure and feedback were the same as in previous studies but inversing the window and referent. The numbering of the buildings and windows was, however, adapted: the buildings on the model were labeled in ascending order starting from the top-left to bottom-right of the physical model of the urban district, and the numbering of the windows in the grid layout was randomized to avoid a too close resemblance between Distant-Map and Distant-Grid.

## 5.2. Apparatus and setup

We used a HoloLens2 HMD (2048 × 1080, with a diagonal field of view of 52 degrees) with a wireless connection to a laptop (CPU: Intel i7; RAM: 16 G). We conducted an ocular calibration on the Hololens2 for each participant before the study began to ensure good eye tracking. We used a mini-mouse held in the user's hand as a clicker to start the trials and validate the target. To adjust the physical model with its virtual counterpart, we placed on the physical model a QR code that was scanned by the HoloLens2.

Since our task does not involve data analysis, we used randomly generated dummy data for each graphic/building pair and decided to represent them with line charts.

Participants were standing in a fixed position next to the model. We chose this static setup to control the user position and avoid any bias on the results due to different movements across participants. Besides, in application contexts such as crisis management or urban redevelopment, many stakeholders may be standing around the model, effectively constraining the movements of users. Finally, our visual representations are static, and the question of how they would dynamically adapt is out of the scope of this paper.

<figure><img src="image_9.png" /><figcaption>**Fig. 5.** Step-by-step description of the experimental task for On-Demand (top) and Permanent (bottom) conditions.</figcaption></figure>
## 5.3. Participants and procedure

The participants of our studies are all research students in the local computer science lab, right-handed, and have normal or corrected-to-normal vision. All the participants had to fill out an informed consent form before participating in the studies, and their collected data were anonymized. Each study began with a briefing on the session goal and procedures. Participants could take a break between each condition of the study.

## 5.4. Choice of metrics

For each of these studies, we chose to collect and analyze objective and subjective data, as in most works focusing on visual search and AR interaction techniques (Cheng et al., 2021; Prouzeau et al., 2019; Whitlock et al., 2019). Objective data are useful to assess the overall performance of users with the different considered techniques, while subjective data allow for exploring user perceptions of the techniques:

* Objective data only includes completion time as the task cannot be terminated without selecting the targeted window (studies 1 and 2) or the physical referent (study 3), i.e., errors are impossible.

* Subjective data includes the ranking of each technique to identify the solution that the participants preferred to achieve the task and the user's informal feedback collected during post-experiment interviews. We also chose to rely on the NASA-TLX questionnaire (Hart, 2006) to provide a multi-dimensional assessment of the workload felt by the participant during the task.

Combining these two forms of assessment increases the applicability of our results. Indeed, in highly demanding and real-time situations, completion time must be minimized: this is the case in crisis management, for example, where decisions must be taken and transmitted rapidly. Conversely, in the context of simulation or data monitoring, some tasks will be repeatedly performed throughout the analysis; in such cases, completion time is no longer the primary focus. Instead, comfortable and preferred techniques can be favored.

## 5.5. Data collection and analysis

We collected the completion time for all trials, i.e. the time between the click on the virtual button to start the trial and the click to select the right window target or physical referent. After finishing the trials for each condition, each participant was asked to fill out a NASA-TLX form (Hart, 2006). We used the unweighted version of the NASA-TLX form, which is as reliable but faster to complete (Moroney et al., May 1992). Participants were finally invited to provide informal feedback and a ranking of the conditions.

For the data analysis, we chose to rely on estimation techniques with 95 % confidence intervals and ratio analysis as recommended by the APA (VandenBos, 2009). Using CIs rather than p-values allows for a more nuanced analysis of the direction and magnitude of the effect instead of a dichotomous inference based on p-values. Ratio analysis is an intra-subject measurement that expresses the effect size (pair-wise comparison) and is computed between each arithmetic means. It also allows us to better quantify the differences between participants and can be used to refine the analysis when appropriate. Many research works adopted this approach over the recent years (Cabric et al., 2019; Jansen et al., 2019; Keddisseh et al., 2021). All CIs are 95 % Bca bootstrap confidence intervals with 10,000 replicates. Our bootstrap method uses a deterministic approach. For the reader more used to interpreting the p-values, a parallel might be drawn with results obtained through the estimation technique and confidence interval reports (see Fig. 3 in (Krzywinski and Altman, 2013) and discussion in annex B).

# 6. User study 1: distant layouts

As detailed in Section 4.3, the goal of the first study is to focus only on distant layouts and investigate the impact of the form of the link, link availability, and window alignment. We focus on the visual search task where the user needs to find, for a given building, a virtual window among the multiple windows.

## 6.1. Studied layout and link dimensions

As explained earlier, we study only visualizations with a Distant value on the **Location dimension**. We explore all the other dimensions and values. We study the two **window alignments**: grid and map. The windows are sorted in ascending order of ID in the grid layout. We compare the three forms of **links**: direct link (a line between the referent and the virtual window), indirect link (a radar view indicating in which quadrant of the windows' plane the digital window can be found), and **ID** only. Finally, regarding **link availability**, we compare permanent links with links on-demand, where the user needs to select the referent first. As a result, for each Window alignment, we combined the three forms of links and two link availability to create **six Links**, illustrated in Fig. 6: *Permanent-direct*, *Permanent-indirect*, *Permanent-ID*, *OnDemand-Direct*, *OnDemand-Indirect*, and *OnDemand-ID*.

## 6.2. Design of the experiment and participants

We followed a 2 × 3 × 2 within-subjects design, with **Window alignment** (grid, map), **Form of link** (direct, indirect, ID-only), and **Link availability** (permanent, on-demand) as factors. The participants were divided into two groups, and we counterbalanced the order of the Window alignment between each group. For each Window alignment, we combined the two forms of links and three values of link availability to create **six Links** (*Permanent-direct*, *Permanent-indirect*, *Permanent-ID*, *OnDemand-Direct*, *OnDemand-Indirect*, and *OnDemand-ID*), presented in a randomized order. For each of the 12 conditions, participants performed two training and four experimental trials (two per model half-part). In addition, when starting each Window alignment, participants had two training trials with the ID-only condition to familiarize themselves with the alignment.

In total, each participant performed 2 Window alignments x 3 Forms of links x 2 Link availability x (2 training + 4 trials) + 2 alignments x 2 trainings = 76 trials (28 training trials and 48 experimental trials). On average, the experiment lasted approximately 40 min per participant.

We recruited twelve participants (6 self-reported as male and six as female), aged 19 to 27 years (mean=23.17, SD=2.62), with little or no experience of AR with see-through HMDs (mean= 2.0 on a 5-point Likert scale).

## 6.3. Results

We collected a total of 576 experimental trials. Outliers were detected considering data points that fell outside +3 standard deviations from the mean. The total number of outliers identified in the dataset was 6 (1.05 %). Consequently, the dataset used for analysis comprised 570 trials. Participants only performed six wrong target selections: 3 with Permanent-Direct, 2 with Permanent-Indirect, and 1 with OnDemand-Indirect.

### 6.3.1. Completion time per window alignment for each link

We compared Grid and Map window alignments for each of the six Links. The mean time with CIs (Fig. 7- left) clearly establishes that participants required more time to complete the task using a Grid alignment than using a Map alignment for four out of the six links: Permanent-Direct, Permanent-Indirect, Permanent-ID, and OnDemand-ID. For these links, the intra-subject ratio analysis (Fig. 7- right) reveals that using the Grid takes between 29 % and 52 % more time than
<figure><img src="image_10.png" /><figcaption>**Fig. 6.** In study 1, we evaluated Grid and Map alignments for a Distant Layout under six Links resulting from the combination of Link availability (permanent, on-demand) with the forms of links (direct, indirect, and ID).</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>**Fig. 7.** Left: Average completion time in s per link and layout (95 % CIs). Right: Time ratio grid/map per link (95 % CIs).</figcaption></figure>

using the Map. However, we found no clear difference between the Grid and Map alignments when using an OnDemand-Direct or OnDemand-Indirect links. For clarity, we do not detail all the time values, as they are discussed in the following sections, illustrated in Fig. 7 and provided in Table A1 and A2 (in the Annexes).

We believe that, in the first four links, the spatial aspect of the map alignment is more helpful in finding the window than the link itself, hence explaining the superiority of Map over Grid. For the two other links, we see two different reasons for not finding a difference between Map and Grid: with OnDemand-Direct, the user only needs to follow the line and does not need to rely on the spatial position of the window; with OnDemand-Indirect, the radar-view provides a spatial hint to find the window, hence removing the difference between Map and Grid alignments.

### 6.3.2. Completion time per link for the grid alignment

Regarding the conditions involving the Grid alignment (gridded bars in Fig. 7-left), and based on the analysis of the average completion time and the time ratio (Table A1, Table A3, and Table A4), results establish that participants are more efficient when using the OnDemand-Direct [5.24 s, CI: 4.71, 6.06] and Permanent-ID [5.24 s, CI:4.40, 6.61] links compared to all others (OnDemand-ID: [6.39 s, CI: 5.75, 7.11], OnDemand-Indirect: [6.89 s, CI:5.65, 9.32], Permanent-Direct: [6.13s, CI:5.29, 7.02], Permanent-Indirect: [6.57s, CI:5.74, 7.95]). No clear difference can be established between these two conditions or between the four remaining ones. The intra-subject ratios, detailed in Table A3 and Table A4, further establish that compared to the OnDemand-Direct link, participants take 20 % to 29 % more time with the four other links, and compared to Permanent-ID 25 % to 33 % more time.

### 6.3.3. Completion time per link for the map alignment

Regarding Map alignment conditions (non-gridded bars in Fig. 7-left), Permanent-ID [3.82 s, CI: 3.21, 4.52] is the most efficient link. No clear distinction could be established between the four remaining links

(Permanent-Direct [4.80 s, CI:4.18, 5.66], Permanent-Indirect [4.62, CI:4.05, 5.35], OnDemand-ID [5.04, CI:4.51, 5.66], OnDemand-Direct [4.87, CI:4.56, 5.18]). The intra-subject ratio analysis strongly confirms this result and shows that compared to Permanent-ID, participants take between 29 % and 59 % more time with the other conditions (see Table A5). Results also show that OnDemand-Indirect [5.77 s, CI:5.06, 6.79] takes more time than all other links. Ratio analysis confirms this result, especially for Permanent-ID and Permanent-Indirect in comparison to which OnDemand-Direct requires >30 % more time (see Table A6).

### 6.3.4. Completion time: best condition overall

Overall, the best combination of window alignment and link among the 12 studied conditions is the Map with Permanent-ID [3.82 s, CI: 3.21, 4.52], compared to which, the other conditions take between 29 % and 87 % more time (see Table A7).

Although on-demand techniques are more time-consuming than permanent ones (because they require action on the user's part), this result is surprising given the reduced visibility under this condition. Despite the longer interaction time, we hypothesized that On-Demand techniques would be more effective than Permanent-ID, as there is no visual link between the referents and the windows. This result indicates that it is more efficient for users to rely on the ID and spatial position of the windows in the Map layout to find them.

### 6.3.5. NASA-TLX

We first discuss the results for the conditions with Grid alignment and then Map alignment (all values in Table A8).

For the Grid alignment, the NASA-TLX results (Fig. 8) show that in terms of Mental Demand, both Permanent-Direct [57.50, CI: 43.75, 69.58] and Permanent-Indirect [45.42, CI: 36.67, 52.92] links are more mentally demanding than On-demand conditions (which range from 25.00 to 27.50) and than Permanent-ID [35.00, CI: 27.92, 44.58]. For the Performance score, participants felt they performed better when
<figure><img src="image_12.png" /><figcaption>Fig. 8. NASA-TLX results for the six Links with a Grid window alignment (95 % CIs).</figcaption></figure>

using the OnDemand-Indirect link [17.50, CI: 12.50, 21.25] than using Permanent-Direct [32.50, CI: 25.42, 40.42], Permanent-Indirect [29.17, CI: 20.83, 37.50] and OnDemand-ID [29.58, 19.58, 42.50]. In terms of Effort, Permanent-Direct [56.25, CI: 40.00, 68.33] yields higher scores than the other conditions (ranging from 26.67 to 39.17). Permanent-Direct (50.00, CI: 36.67, 65.83) is also deemed more Frustrating than others (ranging from 20.00 to 25.83). The scores across the other NASA-TLX indexes do not show any differences.

Concerning the Map alignment (Fig. 9), in terms of Mental Demand, we observed a difference between OnDemand-Direct [17.92, CI: 12.50, 22.92] and the other conditions (ranging from 29.58 to 42.50) except with OnDemand-ID [24.17, CI: 16.67, 39.07], with which there is no difference. For the Performance score, participants felt they performed better when using Permanent-ID [17.50, CI: 13.33, 23.33] than Permanent-Indirect [27.50, CI: 19.58, 36.25] and Permanent-Direct [35.00, CI: 25.42, 47.92]. Participants also felt a better Performance with OnDemand conditions (ranging from 19.17 to 22.08) than with Permanent-Direct. In terms of Effort, Permanent-Direct [43.75, CI: 32.50, 54.17] is more demanding than Permanent-ID [27.92, CI: 21.67, 40.00] and OnDemand-ID [25.42, CI: 17.50, 43.33]. Concerning Frustration, Permanent-Direct [40.42, CI: 27.92, 55.83] obtains a higher score than OnDemand-Indirect [25.00, CI: 15.42, 38.33], but no other clear differences were observed. The scores across the other NASA-TLX indexes do not show any differences.

### 6.3.6. Links ranking per window alignment

Participants were asked to rank their favorite Link for each Window alignment (Grid and Map).

OnDemand-Direct is the most appreciated among the six Links for both types of Window alignments, with an average rank of 1.17 [CI: 1.00, 1.33] for Map and 1.42 [CI: 1.08, 1.75] for Grid. The second favorite links for Map are OnDemand-ID [2.75, CI: 2.00, 3.50] and Permanent-ID [3.00, CI: 2.58, 3.50]. The second favorite link For Grid is OnDemand-Indirect [2.17, CI: 1.67, 2.67].

The least popular link is Permanent-Direct, both on Map [5.83, CI: 5.00, 6.00] and on Grid, where it was always ranked last [6.00, CI: 6.00, 6.00]. Seven over twelve participants reported that permanent direct and indirect links induced visual clutter; hence, they relied on the building ID to retrieve the corresponding window even though this strategy led to a high cognitive load for memorizing and finding the ID on the window layout, particularly for the grid.

### 6.3.7. Preferred window alignment and feedback

Considering their preferred Links for the Map and Grid Window

alignments, participants had to choose which of the two Window alignments they preferred. Ten participants voted for Map and only 2 for Grid. As P1 shared, “Map alignment is better than Grid because when looking for information related to a building, I do not want to search it by ascending order, as I have to think using the Grid. While in Map, it is easy to search”.

### 6.3.8. Study 1 summary

In this first study, we aimed to compare the time required to locate a virtual window for a given physical building using different links within a grid or map window alignment. Participants took less time to find a window on a Map alignment, the preferred Window alignment, than on a Grid alignment for four out of the six links. For the Map alignment, the most efficient link to find a window was the Permanent-ID link. Interestingly, the most popular link for this layout was OnDemand-Direct, as participants appreciated the explicit interaction for displaying the visual link. For the Grid alignment, the most efficient links were Permanent-ID and OnDemand-Direct, the latter being also the most popular. Overall, the best condition among the 12 combinations of link and layout is Map alignment with Permanent-ID link. Permanent-Direct and Permanent-Indirect links received the worst scores on the NASA-TLX questionnaire (the lowest the score, the better) despite not being the slowest techniques, likely due to the visual overload and the occlusion problems stemming from the simultaneous and permanent display of all links.

## 7. User study 2: close, intermediate and distant layouts

As detailed in Section 4.3, the second study's goal is to explore the different window locations (close, intermediate, distant) using the two best data-referent links identified in study 1. We focus on the same visual search task as in study 1, i.e., the user must find a virtual window for a given building.

### 7.1. Studied layout and link dimensions

Our design dimensions considered three values for the **Windows Location dimension**: Distant, Intermediate, and Close. We already explored the Distant value in study 1. Close layouts consist of displaying the window just above its physical referent. We decided to display the windows vertically above their referent to be consistent with the Distant conditions. Intermediate layouts consist of varying the distance of the window according to the position of their referent on the urban model. We designed three solutions to maximize windows' visibility while minimizing the distance to their referents. In the first one, we positioned

<figure><img src="image_13.png" /><figcaption>Fig. 9. NASA-TLX results for the six links when displaying the windows with a Map alignment (95 % CIs).</figcaption></figure>
the windows on an *ascending* 45° plane (i.e., intersecting the front of the model, see Intermediate-Maps Ascending in Fig. 10-bottom, left); in this case, the windows partially occlude the buildings, and the largest distance between a window and its referent is on the back of the urban model. In the second one, we positioned the windows on a *descending* 45° plane above the model (i.e., intersecting the back of the model, see Intermediate-Maps Descending in Fig. 10-bottom, center): the windows do not occlude the buildings, but there is a large visual distance between the referents and their window in the front of the model, thus maximizing the visual distance between referents and windows compared to the first solution. In the third solution, we positioned the windows on a *curved* surface around the model (see Intermediate-Maps Curved in Fig. 10-bottom, right) so that when looking at a building, the user just needs to raise the head in the same direction to see the associated window, i.e., minimizing the visual distance between window and referent.

Regarding the **Window alignment (Grid and Map)**, we decided to explore the Grid alignment only for the Distant layout since Close and Intermediate layouts result in a strong mismatch between the buildings and the windows under the Grid alignment. We explore the Map alignment for all the values of the Window location.

Finally, regarding the **Links**, the first study's results revealed that OnDemand-Direct was the preferred link and Permanent-ID the condition that minimized the completion time for both Grid and Map alignments. Hence, we only considered these two links.

To sum up, we study six layout conditions in total (see Fig. 10), one of them with a Grid alignment (Distant-Grid), and the five others with a Map alignment: Distant-Map; Intermediate Curved-Map, Intermediate Ascending-Map, and Intermediate Descending-Map (for the sake of concision, we will omit the term Intermediate in the name of the conditions); and Close-Map. These six layouts were combined with two links (OnDemand-Direct and Permanent-ID).

## 7.2. Design of the experiment and participants

Our study followed a 6 × 2 within-subject design with Layout (6 layout conditions cited above) and Links (OnDemand-Direct, Permanent-ID) as factors. The participants were divided into six groups counterbalanced by the order of the layouts. For each of the 12 conditions (layout x link), participants performed three training trials and two blocks of four experimental trials (two per model half part).

In total, each participant performed 132 trials (6 layouts*2 links*3 trials = 36 training trials, and 6 layouts*2 links*2 blocks*4 trials = 96 experimental trials). The experiment lasted around 50 min for each participant.

## 7.3. Results

We collected a total of 1152 experimental trials. As in Study 1, we computed the total number of outliers in the dataset, resulting in 18 outliers (1.5 %). Consequently, the dataset used for analysis comprised 1134 trials. Participants only performed six wrong target selections: 1 with Ascending-Map and 5 with Close-Map.

### 7.3.1. Completion time per link for each layout

We compared the two links (OnDemand-Direct, Permanent-ID) for each of the six layouts. The mean time with CIs (Fig. 11-left) clearly establishes that using the OnDemand-Direct link takes more time than the Permanent-ID link for four out of the six layouts: Ascending-Map, Descending-Map, Distant-Map and Close-Map (Fig. 11-left). The intra-subject ratio (Fig. 11-right) confirms this result and also highlights that OnDemand-Direct takes more time than the Permanent-ID link for Curved-Map. It establishes that OnDemand-Direct takes between 17 % and 98 % more time than Permanent-ID for these five layouts. For Distant-Grid, results did not reveal a difference between the two forms of links. For clarity, we do not detail all the time values, as they are discussed in the following sections, illustrated in Fig. 11, and provided in Table A9 and Table A10. The results for the distant layouts (Distant-Map and Distant-Grid) align with our study 1 results.

### 7.3.2. Completion time per layout with ondemand-direct links

When using OnDemand-Direct (stripped bars in Fig. 11-left), Close-Map [7.0 s, CI: 6.23, 8.58] is the least efficient layout. The intra-subject ratio, detailed in Table A11, further establishes that participants take 25 % to 48 % more time with Close-Map than with the other layouts.

Among the five other layouts, the average completion time of Ascending-Map [5.66 s, CI: 5.12, 6.88] is higher than Descending-Map [4.88 s, CI: 4.47, 5.27] and Distant-Grid [4.88 s, CI: 4.48, 5.50]. However, in this group of five layouts, no clear difference could be highlighted regarding the two remaining layouts (Distant-Map [4.88 s, CI: 4.37, 5.67], Curved-Map [5.27 s, CI: 4.61, 6.18]).

We believe that Close-Map leads to longer completion time because the windows occlude the urban model, requiring users to make efforts to locate the building and select it using gaze. This occlusion problem could also explain the long completion time observed for the Ascending-Map layout.

### 7.3.3. Completion time per layouts with permanent-id links

When using Permanent-ID (non-stripped bars in Fig. 11-left), results show that it takes less time to find a window with Descending-Map

<figure><img src="image_14.png" /><figcaption>**Fig. 10.** In study 2, we evaluated six layout conditions resulting from the combination of different Layout Locations and Window Alignments. For each of them, we studied two links: Permanent-ID and OnDemand-Direct.</figcaption></figure>
<figure><img src="image_15.png" /><figcaption>Fig. 11. Left: Mean completion time per layout and link in seconds. Right: Ratio between OnDemand-Direct and Permanent-ID links for each layout (95 % CIs).</figcaption></figure>

[3.59s, CI: 3.22, 4.19] and Close-Map [3.63s, CI: 3.16, 4.28] layouts
than with Distant-Grid [4.57s, CI: 4.21, 5.17] and Curved-Map [4.63s,
CI: 3.95, 5.71]. In addition, the intra-subject ratio analysis reveals that
they also perform better than Ascending-Map [4.27s, CI:3.52, 5.32] and
Distant-Map [4.14s, CI:3.64, 4.69]: the ratio analysis indicates that the
latter four layouts take between 17 % and 30 % more time than
Descending-Map (Fig. 12- left), and between 18 % and 32 % more time
than Close-Map (Fig. 12- center).

Among the four remaining layouts, the ratio analysis also reveals that
compared to the quickest of them, Distant-Map (Fig. 12- right), it takes
14 % more time to use Distant-Grid and 11 % more time to use Curved-
Map, but there is no difference with Ascending-Map. All ratio values are
detailed in Table A11-A13.

7.3.4. Completion time: best condition overall

Overall, among the 12 studied conditions (link x layout), two con-
ditions perform better than the others: Descending-Map with
Permanent-ID [3.59 s, CI: 3.22, 4.19] and Close-Map with Permanent-ID
[3.63 s, CI: 3.16, 4.28].

7.3.5. NASA-TLX

We first discuss the results for the conditions with OnDemand-Direct,
then Permanent-ID (all values in Table A15).

For the OnDemand-Direct link (see Fig. 13), the Mental Demand is
higher with the Close-Map [34.58, CI: 26.25, 45.42] layout compared to
other layouts (ranging from 16.25 to 24.17) except for Distant-Map
[23.75, CI: 16.25, 36.08]. In addition, the Mental Demand is lower
with Descending-Map [16.25, CI: 13.33, 19.58] than other layouts
except for Distant-Grid [17.08, CI: 12.50, 22.08]. Concerning Physical
Demand, Close-Map [30.00, CI: 21.67, 46.67] layout is again more
demanding than other layouts (ranging from 16.25 to 21.25) except for
Curved-Map [21.25, CI: 14.58, 31.25]. Regarding the other indexes
 Temporal Demand, Performance, Effort, Frustration), Close-Map layout
is always worse than the others, but there are no significant differences
between the other layouts.

Regarding the Permanent-ID link (see Fig. 14), the Mental Demand is
lower with Descending-Map [18.33, CI: 14.58, 22.92] than with the
other layout (ranging from 25.42 to 35.42), among which no differences

can be highlighted. In terms of Effort, Descending-Map [22.50, CI:
18.33, 27.50] obtains a lower score compared to the other layouts
(ranging from 30.42 to 35.00) except for Ascending-Map [27.92, CI:
18.33, 27.50], with which there is no difference. Finally, for the Frus-
tration index, results do not highlight differences between the layouts
except for Descending-Map [17.50, CI: 11.67, 27.92] being less frus-
trating than Close-Map [31.25, CI: 21.25, 42.08]. Regarding Physical
Demand, Temporal Demand, and Performance, results do not reveal any
difference among conditions.

7.3.6. Preferred link per layout

Concerning the link preference for each layout, the majority of par-
ticipants preferred using the OnDemand-Direct link rather than the
Permanent-ID link for every layout except for Close-Map. Among the 12
participants, the number of those voting for OnDemand-Direct is 12 for
Distant-Grid, 7 for Distant-Map, 8 for Descending-Map, 8 for Curved-
Map, and 7 for Ascending-Map. For the Close-Map layout, 11 out of 12
participants preferred using Permanent-ID link.

7.3.7. Layouts ranking

Considering their preferred link, participants were asked to rank the
layouts in order of preference. The analysis of the average preference
with CIs does not reveal any distinction of preferences between Distant-
Map [3.42, CI: 2.5, 4.17], Ascending-Map [3.5, CI: 2.67, 4.17],
Descending-Map [2.91, CI: 2.00, 3.67], and Curved-Map [2.41, CI: 1.58,
3.50] layouts. However, this analysis shows that participants preferred
these four layouts to Distant-Grid [4.5, CI: 3.67, 5.17]. The average and
CIs analysis also reveals that Close-Map [4.25, CI: 3.08, 5.17] is less
appreciated than Descending-Map and Curved-Map, but no distinction
can be made between Close-Map and the other layouts (Distant-Grid,
Distant-Map, and Ascending-Map).

This analysis is confirmed by subjective feedback collected at the end
of the experiment concerning the layouts, such as P2, who shared that
“Curved-Map layout is more interesting than Distant-Map since the
layout is larger and the graphs are further apart”. Three participants said
that they would need more training to master it. However, other par-
ticipants noticed this layout forces them to maintain a fixed position at
the center of curvature to ensure the alignment between graphs and

<figure><img src="image_16.png" /><figcaption>Fig. 12. Ratio analyses between layouts with Permanent-ID links. Left: Ratio over Descending-Map (DescM). Center: ratio over Close-Map (CM). Right: ratio over Distant-Map (DistM). 95 % CIs.</figcaption></figure>
<figure><img src="image_17.png" /><figcaption>Fig. 13. NASA-TLX scores per layout with OnDemand-Direct links (95 % CIs).</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>Fig. 14. NASA-TLX scores per layout with Permanent-ID links (95 % CIs).</figcaption></figure>

buildings. Many participants shared that they liked the Descending-Map layout because the windows do not occlude the model, unlike with Ascending-Map. However, raising the head to see the upper windows was sometimes reported as uncomfortable.

Concerning Close-Map layout, P2 shared that “Windows are in the way”, a comment repeated by other participants such as P8, who shared that “It’s good for the buildings in front, but for those behind, it’s annoying because the model is occluded and you have to move around.”.

Thanks to the feedback we received from participants and their ranking of the layouts, we can consider Curved-Map and Descending-Map as participants' favorite layouts and Close-Map and Distant-Grid as the least preferred layouts.

### 7.3.8. Summary

This second study evaluated distant, intermediate, and close layouts with two links. As in the first study, the condition using Permanent-ID links is equally or more efficient than those using OnDemand-Direct links. Indeed, for five out of our six layouts (Distant-Map, Descending-Map, Ascending-Map, Curved-Map, and Close-Map), participants performed better with a Permanent-ID link than with an OnDemand-Direct link. When looking at all the combinations of link and layout, the best performing solutions are Descending-Map and Close-Map, both with a Permanent-ID link. Interestingly, some of these performance values deviate from user preference: participants preferred using an OnDemand-Direct link for all layouts except Close-Map, and Curved-Map and Descending-Map were the favorite layouts. We further discuss the implications of these results in our Discussion section.

## 8. User study 3: finding the physical referent for a given window

As detailed in Section 4.3, the goal of Study 3 is to extend the results of our previous studies to the opposite visual search task. We evaluate the best combinations of layouts (window alignment x location, see Fig. 15) and links identified in the previous user studies to assess their usability for the opposite visual search task, i.e., finding a physical referent corresponding to a given virtual window.

### 8.1. Studied layout and link dimensions

Regarding the **layout conditions**, as in study 2, we excluded the combination of a Grid alignment with Intermediate and Close locations. Following the results of study 2, we also decided to discard two Intermediate layouts due to their poor performance: Curved-Map and Ascending-Map. As a result, we considered the following four layouts: Distant-Map (best layout from study 1 with Permanent ID), Distant-Grid (to keep the same baseline across all studies), Descending-Map and Close-Map (best layouts from study 2 with Permanent ID link).

Regarding the **links**, studies 1 and 2 revealed that OnDemand-Direct is the preferred link for all layouts, except for Close-Map, but Permanent-ID is the most efficient. Therefore, in study 3, we decided to combine the previously mentioned layouts with these two links (OnDemand-Direct and Permanent-ID), except for Close-Map, which we only considered with a Permanent-ID link.

To sum up, in this study, we investigated seven conditions: three layouts (Distant-Map, Distant-Grid, and Descending-Map) with the two links and Close-Map with a Permanent-ID link.

### 8.2. Design of the experiment and participants

The study followed a 4 × 2 within-subject design with layout (Distant-Map, Descending-Map, Close-Map, and Distant-Grid) and Link (Permanent-ID and OnDemand-Direct) as factors. However, as explained earlier, we did not study the combination of Close-Map with OnDemand-Direct; hence, in total, we studied seven conditions resulting from the combination of layouts with links. The participants were divided into four groups counterbalanced by the order of the layouts. For each of the seven conditions (layout x link), participants performed three training trials and four blocks of four experiment trials (two per model half part).

In total, each participant performed 133 trials (7 conditions*3 trials = 21 training trials and 7 conditions*4 blocks*4 trials = 112 experimental trials). The experiment lasted around 45 min per participant.

We recruited twelve participants (8 self-reported as male and 4 as

<figure><img src="image_19.png" /><figcaption>Fig. 15. In the third study, we focus on the four illustrated layouts.</figcaption></figure>
female) aged 19 to 27 years (mean=23.17, SD=2.79), with little or no
experience of AR using HMDs (mean= 2.08). Half of them took part in
the previous studies.

8.3. Results

We collected a total of 1344 experimental trials. As in previous
studies, we computed the number of outliers in the dataset, resulting in
24 outliers (1.8 %). Consequently, the dataset used for analysis com-
prises 1320 trials. Participants did not perform any wrong target
building selection.

8.3.1. *Completion time per link for each layout*

We compared the two links (OnDemand-Direct, Permanent-ID) for
each of the three concerned layouts (Distant-Grid, Distant-Map, and
Descending-Map). The mean time and CIs (Fig. 16 -left) reveal that
OnDemand-Direct takes more time than Permanent-ID on a Descending-
Map layout, and the intra-subject ratio analysis (Fig. 16 -right) estab-
lishes that this is also true on a Distant-Map layout. Using OnDemand-
Direct takes 32 % more time than Permanent-ID on a Descending-Map
and 13 % more on a Distant-Map. These results are in line with previ-
ous studies for the opposite task. For clarity, we do not detail all the time
values here, as they are discussed 7 in the following sections, illustrated
in Fig. 16 and provided in Table A16 and Table A17.

Conversely, Permanent-ID takes more time than OnDemand-Direct on a Distant-Grid layout (8 % more, as per the ratio analysis). This result is surprising and differs from previous studies, where we found no difference between both links on a Distant-Grid layout. In those studies, OnDemand-Direct was never more effective than Permanent-ID for any layout. This discrepancy may be because, when using Permanent-ID on a Distant-Grid, the user relies solely on the ID to find the building on the model, without any spatial hint as in map-based layouts. Although the buildings were labeled in ascending order, they were not aligned with each other, making the task of finding the building with the right ID in the model rather tedious.

8.3.2. *Completion time per layout with ondemand-direct links*

When using OnDemand-Direct links (stripped bars in Fig. 16-left), the analysis of the average completion time with CIs shows that Descending-Map [4.82 s, CI: 4.57, 5.19] takes more time than Distant-Map [4.31 s, CI: 4.08, 4.7]. The intra-subject ratio, detailed in Table A18, confirms this result and further establishes that Descending-Map takes 12 % more time than Distant-Map. Ratio analysis also weakly reveals that Descending-Map takes 7 % more time than Distant-Grid [4.53 s, CI: 4.30, 4.96]. Descending map is thus the worst-performing layout when using OnDemand-Direct link. The ratio analysis does not clearly establish a difference between Distant-Grid and Distant-Map (see Table A18).

In study 2, all three layouts had similar completion times. This discrepancy can be attributed to an increased difficulty to localize a window with Descending-Map than with Distant-Map, as windows are visually more widespread.

8.3.3. *Completion time per layout with permanent-ids links*

When using Permanent-ID links (non-stripped bars in Fig. 16-left), the analysis of the average completion time with CIs reveals that Distant-Grid [5.01 s, CI:4.64, 5.52] takes more time than the other layouts: Distant-Map [3.91 s, CI:3.59, 4.33], Descending Map [3.75 s, CI:3.42, 4.14] and Close-Map [3.69 s, CI:3.31, 4.33]). The ratio analysis established that using Distant-Grid takes respectively 31 %, 35 %, and 40 % more time than these layouts (see Table A19). We found no difference among the three map-based layouts. These results align with those of study 2, where Distant-Grid was also slower than these map-based layouts for the opposite task.

8.3.4. *Completion time: best condition overall*

Overall, the best conditions in terms of performance for this study are
those involving a map-based layout with a Permanent-ID link (Close-
Map, Distant-Map, and Descending-Map). Descending-Map and Close-
Map with Permanent-ID link were also the best conditions in study 2
for the opposite task.

8.3.5. NASA-TLX

We first discuss the results for the conditions with OnDemand-Direct, then Permanent-ID (all values in Table A20).

When using the OnDemand-Direct link (Fig. 17-left), the Descending-Map layout [23.75, 17.50, 32.50] requires more Physical Demand than the Distant-Grid [15.00, 10.42, 20.00] and Distant-Map layouts [15.00, 10.83, 21.67], which are similar. This difference may be attributed to the graphs closest to the users in Descending-Map, which require them to raise their heads, inducing neck fatigue. In terms of Frustration, Descending-Map [17.50, CI: 11.25, 26.35] is more prone to Frustration than Distant-Map [10.42, 7.08, 15.83]. There are no significant differences in the other NASA-TLX indexes between the layouts.

When using the Permanent-ID link (Fig. 17-right), Distant-Grid
[34.17, CI: 27.50, 40.83] is the layout that requires the highest Mental
Demand (others ranging from 19.58 to 24.58). As explained in 6.5.1, this
is probably due to the difficulty of locating the building on the model
only by its ID, without any spatial hint as in map-based layouts.
Regarding Physical Demand, Close-Map [29.58, CI: 20.83, 41.25] and
Descending-Map [22.08, CI: 17.50, 26.67] layouts received worse scores
than Distant-Grid [15.83, CI: 11.67, 20.83] and Distant-Map [17.08, CI:
14.17, 20.42] layouts. With Close-Map, we noticed several participants
had to move slightly to the side to see the building, hidden by windows.
With Descending-Map, as explained earlier, participants had to raise
their heads to see the closest windows, inducing neck fatigue. In terms of
Performance, Close-Map received worse scores [29.17, CI: 17.50, 42.92]
than all other layouts (ranging from 13.33 to 17.08), which contradicts
the measured completion time results, where it performed similarly to
the other map-based layouts. The level of Frustration is also higher for
Close-Map [34.17, CI: 18.33, 51.67] compared to the other layouts
(ranging from 14.17 to 16.67). This result is partially in line with Study
2, where Close-Map also received worse scores on Frustration than
Descending-Map. These negative Performance and Frustration percep-
tions of Close-Map might be explained by the visual clutter of windows
occluding the buildings. There are no significant differences in the two
remaining indexes (Temporal demand and effort) between the layouts.

<figure><img src="image_20.png" /><figcaption>**Fig. 16.** Left: Mean completion time per layout and link in seconds. Right: ratio OnDemand-Direct over Permanent-ID for each layout (95 % CIs).</figcaption></figure>
<figure><img src="image_21.png" /><figcaption>Fig. 17. NASA-TLX scores per layout with OnDemand-Direct link (left) and Permanent-ID link (right).</figcaption></figure>

### 8.3.6. Preferred link

Concerning the link preference for each layout, as in previous studies, the majority of participants preferred using OnDemand-Direct link rather than Permanent-ID link with Distant-Map (8 out of 12) and Distant-Grid (10 out of 12). For Descending-Map, opinions are balanced (6 out of 12).

### 8.3.7. Layouts ranking

Considering their preferred link for each layout, participants were asked to rank the layouts in order of preference. The analysis of the average preference with Cls does not reveal any distinction of preferences between Distant-Grid [2.47, 1.75, 2.83], Distant-Map [2.17, 1.58, 2.75] and Descending-Map [2.00, 1.50, 2.50] layouts. However, this analysis shows that participants preferred these three layouts to Close-Map [2.47, 1.75, 2.83], which is clearly the least favorite layout. Indeed, it was ranked in last position 8 times out of 12, which aligns with NASA-TLX results.

### 8.3.8. Summary

This third study evaluated the best combination of layouts and links from the second study for the opposite visual search task, i.e., finding the building for a given window. With OnDemand-Direct link, Descending-Map is the worst-performing layout, with weak evidence for Distant-Grid. With Permanent-ID link, Distant-Grid is the worst-performing layout. Overall, considering all the combinations of link and layout, the best conditions are the three map-based layouts (Close-Map, Distant-Map, and Descending-Map) with a Permanent-ID link. As in previous studies, some of these results do not align with the NASA-TLX indexes and the participants' preferences, who clearly dislike Close-Map.

## 9. Discussion

### 9.1. Design guidelines

#### 9.1.1. G1. data windows alignment should mirror the position of referents in the real world

In all our studies, map-based layouts (mirroring the position of the referent on the model) have shown their superiority in terms of efficiency over grid-based layouts on a visual search task. This result seems to contradict the traditional presentation of small multiple visualizations in immersive environments using a grid (Liu et al., 2020), as well as previous work showing that grid alignment reduces visual exploration time over freeform arrangements (Simon et al., 2019). However, these works did not evaluate the grid on a situated visualization, which introduces a mismatch between the position of the windows and the physical referents. Our studies show that with a map-based layout, participants heavily rely on the spatial information to the extent that they do not need another form of link between referent and window. Interestingly, these map-based layouts bear similarities to 2D geographical visualizations, such as Cartograms (Nusrat et al., 2020) and especially TileGridMaps (McNeill and Hale, 2017), which display geographic data as a grid of identical tiles. Prior work (McNeill and Hale, 2017) has proposed algorithms for optimizing the position of each tile,

which could potentially be applied to automate and refine our map-based layouts.

#### 9.1.2. G2. window location: "When close is too close."

The location of the windows with respect to their referent has yielded some surprising results. Prior works often depict Close layouts with data displayed close to its physical referent (Satriadi et al., 2022) or with text labels displayed in AR next to the object of interest (Li et al., 2017). However, our results show that while Close layouts are among the best performing conditions for the two investigated visual search tasks, they systematically receive negative scores on the NASA-TLX indexes (e.g., Frustration or Performance depending on the task) and are the least preferred layout. Hence, as they induce visual clutter and referent occlusion, we can say that "Close is too close" for these layouts.

Distant layouts eliminate any occlusion between windows and referents but induce a longer time to locate a window than intermediate or close layouts, as evidenced in our study 2.

Intermediate layouts emerge as the best overall window location for both visual search tasks, but they require careful design to prevent model occlusion. We explored three versions of the intermediate layouts: Ascending-Map, Descending-Map, and Curved-Map. Descending performed the best overall. However, if the front windows are positioned too high above the model, it can lead to neck fatigue, especially with our current heavy HMDs. Thus, the layout angle above the model (45° in our work) should probably be adapted to the user's height. Previous work has already explored content positioning above the user and also noted neck strain problems (Satkowski et al., 2022). They suggested that less important content be placed there. But to our knowledge, no prior work has proposed or evaluated a Descending-Map visualization, making the advantages of such a layout a key finding of our work.

#### 9.1.3. G3. links and visual clutter: avoid permanent spatial links

Among the three visual links we studied (Direct, Indirect, and IDs only), the first two had a strong spatial component: direct is a line spatially connecting the referent to its window, and indirect is a radar view indicating the spatial area where the window is located. When combined with a Permanent link availability, our initial assumption was that displaying all lines would result in visual clutter, which was confirmed by Study 1. Interestingly, many visualizations in AR use this permanent display of lines (Prouzeau et al., 2019). While some techniques have been proposed to enhance the visibility of the lines, such as curving them, we believe they would not improve this condition in a model with so many buildings.

With the indirect link, we were surprised to discover that permanently displaying all radar views did not help locate the windows either. Participants seemed to rely on the ID or the referent position on the map, rather than on our radar view, to perform the visual search task.

#### 9.1.4. G4. "Can I see your ID?" permanent IDs are sufficient

Another surprising result of our work is that the Permanent-ID link performed better than an OnDemand-Direct link. While we anticipated that OnDemand would add time for the explicit interaction (i.e., to select
the initial referent/window), we believed it would be compensated by facilitating the second part of the task (i.e., following the line between the referent and window). While this is true on a Distant-Grid, where both links perform similarly (study 1), the Permanent-ID was sufficient on map-based layouts, and performed better. The reason is that the spatial component of the map-based layout is a sufficient hint to locate the window/referent.

### 9.1.5. G5. link availability: offer on-demand interaction

Although Permanent-ID performs better for map-based layouts, the OnDemand-Direct link tends to reduce the mental effort (study 1) and is preferred by participants (all studies). This result indicates that while the situated visualization should always be based on permanently displaying all IDs, it should also allow the user to display direct links on-demand. Give this opportunity to the user would be in line with the general principle behind Schneiderman's mantra, "Overview First, Details on Demand" (Shneiderman, 1996). Besides, when using a Distant-grid layout (which we do not recommend - see G1), OnDemand-Direct performs as well as Permanent-ID.

### 9.1.6. Practical application of the guidelines to future system design

As reported in the three studies, preferences expressed by participants are not always in line with the results in terms of performance. It is an interesting result that can be explained by different subjective criteria, such as occlusion and visual overload, as underlined in study 1, or higher mental effort noted in study 2 and study 3. Consequently, according to the context of use of the augmented urban model, designers will have to choose the most appropriate layouts and links among our previously presented recommendations.

In crisis management, decisions will have to be taken rapidly based on discussions that can be motivated by the most appropriate data on the physical model. Designing systems for such situations should thus increase the efficiency of the stakeholders and rely on our completion time-based recommendations: rely on map-based layouts instead of grid layouts (G1); use either Close Layout or Intermediate Descending Map as they are the best performing (G2); and use a Permanent-ID link (G4).

Conversely, in urban planning, stakeholders must visualize the outcomes of multiple simulations to determine the optimal configuration for a district (e.g., land use, infrastructure, and transportation). The physical model should, therefore, favor interaction experience over performance, as the user may spend a long time interacting with the model without strong time constraints. Concretely, the Close layout should be discarded (G2); the Permanent-ID link should be used along the OnDemand-Direct, as it reduces mental effort even though it takes more time (G5).

Beyond crisis management and urban planning, digital twin applications may also benefit from such interactive situations. According to Vainiopää et al. (Vainionpää et al., 2022), one of the challenges of the domain is to reflect "the dynamic interaction between the physical and the digital": any changes in the physical or digital part of the Digital Twin should lead to displaying a link between the physical and digital parts to notify the user. In such contexts, map-based layouts should thus be favored to support the use of OnDemand links (G5).

## 9.2. Similarity and divergence from past work

In this section, we discuss our experimental results compared to previous experiments from our related work on data arrangement in AR and VR (Section 2.3) and on virtual-physical links (Section 2.4).

Regarding data arrangement, many studies have focused on comparing flat and curved layouts in VR (Barrett M. Ens et al., 2014; Liu et al., 2020; Satriadi et al., 2020). Our results show that intermediate curved layouts are preferred by users over distant layouts, although no difference in performance was found compared to distant layouts. These results confirm previous studies on curved layouts in VR, which have been shown to perform similarly to flat layouts and be preferred by

users. However, our study provides novel results regarding curved layouts: they perform worse than a Descending-Map layout around a physical referent in AR.

Regarding situated data arrangement, our results confirm some of the guidelines of Lee et al. (Lee et al., 2024) on designing situated visualizations in AR: they suggested that in case of overlapping views, the views can be placed away from the referents. Our results on the Descending-Map layout confirm that placing the windows away from the referents can be a better solution than having the windows next to the referent. Among the results presented by Wen et al. (Wen et al., Jan. 2023), those related to situatedness align with ours: grid layout is less efficient than intermediate or close layouts, and the close layout tends to be the most efficient. However, our results expand these findings in two ways: we confirmed these results are still valid for a large set of physical referents (48 instead of only 6), and we assessed some subjective aspects of the interaction that refine these results.

Finally, we can compare our results to previous work on multivariate data presentation around physical spaces. Satriadi et al. (Satriadi et al., 2022) found that the data arrangement did not impact performance when comparing simple charts across physical building referents. The difference between their work and ours is that their physical model included only five buildings; hence, our results complement their exploration by showing that data arrangement has an important impact when considering multiple referents and windows. Perelman et al. (Perelman et al., 2022) showed that displaying AR information at the back of a tabletop is more advantageous than on the sides. Our results complement their study by demonstrating that displaying virtual data over the surface containing the physical referents is better than displaying it at the back.

Finally, compared to previous works, our work further informs the use of straight lines, the most common approach for linking virtual data with a physical referent (Prouzeau et al., 2019; Satriadi et al., 2022). Our results confirm that permanently displaying direct links is not an appropriate approach when having multiple objects to link due to visual clutter. In addition, our results further contribute by showing that it is an interesting approach if the user can display the link on-demand. Our results also confirm previous studies on the limitations of Indirect links, such as color, which are only useful for a limited number of links.

## 9.3. From visual search task to data analysis

In this paper, we explored two visual search tasks: finding the data associated to a physical referent (task 1) and finding the referent for a given window in the small multiples visualization (task 2). These are two fundamental tasks that will precede any other data analysis tasks, such as filtering. However, reordering the data (i.e., re-arranging the windows according to their value), another frequent task in small multiples visualization, would pose some challenges with the map-based layouts. For the Distant-Map layout, the position of the windows on the vertical plane is fixed, as it mirrors the position of the buildings on the model. An option to encode such ordering would be to use the depth position of the window (e.g., closer or further away from the user). For the intermediate layouts, the window height over the building could encode its value, but the resulting layout would probably induce occlusion on the model.

Furthermore, the need to activate more data analysis commands will raise the question of how to invoke them. Previous work has, for instance, proposed to use smartphone-based gestures around the HMD FoV to execute commands (Saidi et al., 2021). The smartphone could then be used to invoke and refine the filtering too (Perelman et al., 2019).

Map-based layout happens to be useful in analysis tasks where the location of the data has an impact on the analysis and decision-making (e.g., flood crisis management). Mirroring data positioning can enable users to quickly get a geographical understanding of the situation and compare different areas with each other.
## 9.4. Generalization to egocentric and mobile situations

In this section we discuss how our findings would generalize to contexts where the user would move around the model, as well as to the case of windows displayed in an egocentric reference frame.

**Mobility and user placement:** In our studies, the user stood in a static position, and the windows were spatially fixed. If a user wants to move around the model, the layout of the windows would need to dynamically adapt their position or orientation to preserve the benefits observed in our studies. For distant layouts, the data are always placed on the opposite side of the participants: when using a Map alignment, every time the user moves to a different side of the model, the windows would relocate to the opposite side, and the position of the windows change according to the user's perspective on the map. For intermediate locations, when using a Map Alignment, the window's height would need to adapt to the user's position to ensure that all windows are still visible, and the windows would have to rotate to face the user. This vertical adjustment of the windows can be performed dynamically, hence continuously preserving the benefits of this type of layout. For close layouts, moving around the model would only require rotating the windows to face the user. However, this condition would still suffer from the limitations observed in our studies, i.e., visual clutter and referent occlusion.

**Egocentric reference frame:** In our studies, our goal was to investigate small multiple visualizations, which facilitate quick data comparisons. This type of visualization requires a large display space; hence, it is best suited with a fixed world-based position. As explained in the related work section, another approach is to display the windows in an egocentric reference frame, i.e., the window is attached to the user or the HMD FoV. While this approach would be less suited to display multiple windows, it could be interesting if the user moves around the model and wishes to focus on one or two data graphs. The challenge, then, is to link these egocentric windows and the world-fixed buildings. Given that the visual search task would involve only one or two windows and referents, according to our results, the best link for this situation would be a direct link. A deeper analysis will, however, be required as it raises questions about the readability of the windows: the background may hinder the user's ability to easily perceive the data.

## 9.5. Limitations and future work

In this project, we performed a first exploration of the design of situated visualization around a scale model. As such, we considered a fundamental but limited set of conditions and cases.

First, our studies only considered the fundamental visual search task on a simplistic model (to avoid any confounding effect). Further studies are needed to validate our results for a more ecological model and data analysis task, where the user will need to perform multiple back-and-forth movements between the data and the model.

Second, our study involved only one user in front of the model, although multiple users may stand around the model in a real use case. It can impact the optimal placement of the data windows, which can interfere with inter-personal communication. A dynamic window placement (Niyazov et al., 2021; Niyazov et al., 2023) could help to position the data around other users instead of in front of them. This will open new challenges regarding interaction with visualizations situated close to other participants, as previous work on collocated immersive visualization has shown that users follow social protocols and do not interact with visualizations outside of their workspace (Lee et al., Feb. 2021).

Third, each physical building is a referent for a single data window in our work. In the design patterns for situated visualization (Lee et al., 2024), this condition corresponds to situated, independent, and view-frame situatedness and a one-to-one cardinality. However, multi-variate data might be related to one building, as exemplified in (Satriadi et al., 2019) (one-to-many cardinality), or different parts of the referent

might be related to different data (many-to-many cardinality), such as energy monitoring data for various floors. Similarly, embedding the data on the physical referent can be an interesting view placement for large buildings. Future studies will explore these other cardinalities and view placements with expert users to address their specific data analysis needs.

Fourth, we chose to consider each interaction technique holistically, that is, including all the interaction steps necessary to make them usable in real-life scenarios, to ensure the ecological validity of our studies. To further analyze and understand the impact of the different design dimensions (cf. Section 3.2) on users' mental effort while using the techniques, it would be necessary to isolate and compare each step of the interaction techniques: target search, target validation, data window search, and data window validation. Such studies would further inform the design of data layout and link between a physical referent and its data for visual search tasks in AR.

Finally, we used a Hololens2, which has a bounded FoV (Cabric et al., 2021). While we believe this has limited impact on our results, as our studied layouts occupy similar visual space, our results may need to be reassessed if see-through HMDs with larger FOVs become available in the future. However, in line with authors of prior work in similar contexts (Perelman et al., 2022; Satriadi et al., 2022), we used this device for its stability, robustness, and ecological validity, given its widespread adoption on the market.

## 10. Conclusion

In conclusion, this work has explored the design of techniques to support visual search in augmented urban physical 3D models. These models, comprising 3D-printed building representations seamlessly integrated with augmented reality, facilitate access to data situated in the urban environment. We focus on the visual search task in such context as a fundamental step preceding any data exploration. We have examined the design aspects of presenting 2D data windows in augmented reality around physical urban models. The results reveal that a map-based positioning of the virtual windows works better than the traditional grid alignment. While users appreciate showing a visual link between the referent and the associated window on-demand, simply showing an ID next to the referent and window yields a good performance. Adopting a layout that reduces the distance between the window and referent performs better as long as it does not occlude the model. Finally, whether considering the visual search of a window corresponding to a given building or the opposite task, the best performing conditions in one task also appropriately support the other task. Our contributions include a set of design dimensions and empirical insights, serving as a guide for designers of situated visualizations around physical urban models. As sensor proliferation continues, these results hold promise for enhancing urban data interaction and analysis.

### CRediT authorship contribution statement

**Mohamed Reda Benkhelifa:** Writing – review & editing, Writing – original draft, Visualization, Software, Conceptualization.
**Emmanuel Dubois:** Writing – review & editing, Writing – original draft, Supervision, Project administration, Methodology, Conceptualization.
**Louis-Pierre Bergé:** Writing – original draft, Supervision, Project administration, Methodology, Conceptualization.
**Marcos Serrano:** Writing – review & editing, Writing – original draft, Validation, Supervision, Project administration, Methodology, Conceptualization.

### Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
## Acknowledgments

This work was lead as part of the Vilagil-AmUrb and ANR-INTERPLAY projects. Vilagil-AmUrb is supported by the French Government in the framework of the major investment plan (Territoires d'Innovation), by Toulouse Métropole and the neoCampus Scientific Interest Group of the University of Toulouse III Paul Sabatier. INTERPLAY is supported by the french research agency ANR (grant ANR-21-CE33-0022).

## Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.ijhcs.2025.103556.

## Data availability

Data will be made available on request.

## References

Alallah, Fouad, Sakamoto, Yumiko, Irani, Pourang, 2020. Exploring the need and design for situated video analytics. In: Proceedings of the 2020 ACM Symposium on Spatial User Interaction (SUI '20). New York, NY, USA, 15. Association for Computing Machinery, pp. 1-11. https://doi.org/10.1145/3385959.3418458. Article.

Alonso, Luis, Zhang, Yan Ryan, Grignard, Arnaud, Noyman, Ariel, Sakai, Yasushi, ElKatsha, Markus, Doorley, Ronan, Larson, Kent, 2018. CityScope: a data-driven interactive simulation tool for urban design. Use case volpe. In: Morales, A., Gershenson, C., Braha, D., Minai, A., Bar-Yam, Y. (Eds.), Unifying Themes in Complex Systems IX. ICCS 2018. Springer, Cham. https://doi.org/10.1007/978-3-319-96661-8 27. Springer Proceedings in Complexity.

Bekos, M., Niedermann, B., Nöllenburg, M., 2019. External labeling techniques: a taxonomy and survey. Comput. Graph. Forum 38 (3), 833-860. https://doi.org/10.1111/cgf.13729. Online publication date: 1 June 2019. https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.13729.

Bell, Blaine, Feiner, Steven, Höllerer, Tobias, 2001. View management for virtual and augmented reality. In: Proceedings of the 14th annual ACM symposium on User interface software and technology (UIST '01). New York, NY, USA. Association for Computing Machinery, pp. 101-110. https://doi.org/10.1145/502348.502363.

Bressa, Nathalie, Korsgaard, Henrik, Tabard, Aurélien, Houben, Steven, Vermeulen, Jo, Jan. 2022. What's the situation with situated visualization? A survey and perspectives on situatedness. IEEE Trans. Vis. Comput. Graph. 28 (1), 107-117. https://doi.org/10.1109/TVCG.2021.3114835.

Cabris, Florent, Dubois, Emmanuel, Serrano, Marcos, 2021. A predictive performance model for immersive interactions in mixed reality. In: 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Bari, Italy, pp. 202-210. https://doi.org/10.1109/ISMAR52148.2021.00035.

Cabris, Florent, Dubois, Emmanuel, Irani, Pourang, Serrano, Marcos, 2019. TouchGlass: raycasting from a glass surface to point at physical objects in public exhibits. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 249-269. https://doi.org/10.1007/978-3-030-29387-1 15.

Cavallo, Marco, Dolakia, Mishal, Havlena, Matous, Ocheltree, Kenneth, Podlaseck, Mark, 2019a. Immersive Insights: a hybrid Analytics system for collaborative exploratory Data analysis. In: Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology (VRST '19). New York, NY, USA, 9. Association for Computing Machinery, pp. 1-12. https://doi.org/10.1145/3359996.3364242. Article.

Cavallo, M., Dholakia, M., Havlena, M., Ocheltree, K., Podlaseck, M., 2019b. Dataspace: a reconfigurable hybrid reality environment for collaborative information analysis. In: 2019 IEEE conference on virtual reality and 3D user interfaces (VR). IEEE, pp. 145-153.

Chan, Liwei, Müller, Stefanie, Roudaut, Anne, Baudisch, Patrick, 2012. CapStones and ZebraWidgets: sensing stacks of building blocks, dials, and sliders on capacitive touch screens. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). New York, NY, USA. Association for Computing Machinery, pp. 2189-2192. https://doi.org/10.1145/2207676.2208371.

Cheng, Yifei, Yan, Yukang, Yi, Xin, Shi, Yuanchun, Lindlbauer, David, 2021. SemanticAdapt: optimization-based adaptation of mixed reality layouts leveraging virtual-physical semantic connections. In: The 34th annual ACM Symposium on User Interface Software and Technology (UIST '21). New York, NY, USA. Association for Computing Machinery, pp. 282-297. https://doi.org/10.1145/3472749.3474750.

Cityscope: https://www.media.mit.edu/projects/cityscope/overview/.

Cureton, Paul, Dunn, Nick, 2021. Digital twins of cities and evasive futures. In: Aurigi, Alessandro, Odendaal, Nancy (Eds.), Shaping Smart for Better Cities. Academic Press, Cambridge, Massachusetts, pp. 267-282. https://doi.org/10.1016/B978-0-12-818636-7 0.00017-2.

Rhazal, Oumaima El, Tomader, Mazri, 2019. Study of smart city data: categories and quality challenges. In: Proceedings of the 4th International Conference on Smart City

Applications (SCA '19). New York, NY, USA, 4. Association for Computing Machinery, pp. 1-7. https://doi.org/10.1145/3368756.3368965. Article.

Ens, Barrett, David Hincapié-Ramos, Juan, Irani, Pourang, 2014a. Ethereal planes: a design framework for 2D information space in 3D mixed reality environments. In: Proceedings of the 2nd ACM symposium on Spatial user interaction (SUI '14). New York, NY, USA. Association for Computing Machinery, pp. 2-12. https://doi.org/10.1145/2659766.2659769.

Ens, Barrett M., Finnegan, Rory, Irani, Pourang P., 2014b. The personal cockpit: a spatial interface for effective task switching on head-worn displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI '14). New York, NY, USA. Association for Computing Machinery, pp. 3171-3180. https://doi.org/10.1145/2556288.2557058.

Ens, Barrett, Bach, Benjamin, Cordeil, Maxime, Engelke, Ulrich, Serrano, Marcos, Willett, Wesley, Prouzeau, Arnaud, Anthes, Christoph, Büschel, Wolfgang, Dunne, Cody, Dwyer, Tim, Grubert, Jens, Haga, Jason H., Kirshenbaum, Nurit, Kobayashi, Dylan, Lin, Tica, Olaosebikan, Monsurat, Pointecker, Fabian, Saffo, David, Saquib, Nazmus, Schmalstieg, Dieter, Szafir, Danielle Albers, Whitlock, Matt, Yang, Yalong, 2021. Grand challenges in immersive analytics. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. New York, NY, USA, 459. Association for Computing Machinery, pp. 1-17. https://doi.org/10.1145/3411764.3446866. Article.

Ens, Barrett, Goodwin, Sarah, Prouzeau, Arnaud, Anderson, Fraser, Wang, Florence Y., Gratz, Samuel, Lucarell, Zac, Moyle, Brendan, Smiley, Jim, Dwyer, Tim, 2021 Feb. Uplift: a tangible and immersive tabletop system for casual Collaborative visual analytics. IEEE Trans. Vis. Comput. Graph. 27 (2), 1193-1203. https://doi.org/10.1109/TVCG.2020.3030334.

Grubert, Jens, Heinisch, Matthias, Quigley, Aaron, Schmalstieg, Dieter, 2015. MultiFi: multi-fidelity interaction with displays on and around the body. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI '15). New York, NY, USA. Association for Computing Machinery, pp. 3933-3942. https://doi.org/10.1145/2702123.2702331.

Hart, Sandra G., 2006. NASA-task load index (NASA-TLX); 20 years later. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 50, pp. 904-908. https://doi.org/10.1177/154193120605000909.

Hubenschmid, S., Zagermann, J., Butscher, S., Reiterer, H., 2021. STREAM: exploring the combination of spatially-aware tablets with augmented reality head-mounted displays for immersive analytics. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). New York, NY, USA, 469. Association for Computing Machinery, pp. 1-14. https://doi.org/10.1145/3411764.3445298. Article.

Jansen, Yvonne, Schjerlund, Jonas, Hornbæk, Kasper, 2019. Effects of locomotion and visual overview on spatial memory when interacting with wall displays. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). New York, NY, USA, 291. Association for Computing Machinery, pp. 1-12. https://doi.org/10.1145/3290605.3300521. Paper.

Keddisseh, Elio, Serrano, Marcos, Dubois, Emmanuel, 2021. KeyTch: combining the keyboard with a touchscreen for rapid command selection on toolbars. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). New York, NY, USA, 191. Association for Computing Machinery, pp. 1-13. https://doi.org/10.1145/3411764.3445288. Article.

Ihara, Keiichi, Kawaguchi, Ikkaku, 2023. Virtual object placement in MR space using a 3D miniature model of a room. In: Asian HCI Symposium '22 (Asian HCI Symposium '22). New York, NY, USA. Association for Computing Machinery, pp. 23-29. https://doi.org/10.1145/3516492.3558789.

Kraus, M., Fuchs, J., Sommer, B., Klein, K., Engelke, U., Keim, D., Schreiber, F., 2022. Immersive Analytics with abstract 3D visualizations: a survey. Comput. Graph. Forum. 41, 201-229. https://doi.org/10.1111/cgf.14430.

Krywinski, M., Altman, N., 2013. Error bars. Nat. Method. 10, 921-922. https://doi.org/10.1038/nmeth.2659.

Künz, Andreas, Rosmann, Sabrina, Loria, Enrica, Pirker, Johanna, 2022. The potential of augmented reality for digital twins: a literature review. In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, pp. 389-398. https://doi.org/10.1109/VR51125.2022.00058.

Lee, B., Sedlmaier, M., Schmalstieg, D., 2024. Design patterns for situated visualization in augmented reality. IEEE Trans. Vis. Comput. Graph. 30 (01), 1324-1335. https://doi.org/10.1109/TVCG.2023.3327398.

Lee, B., Hu, X., Cordeil, M., Prouzeau, A., Jenny, B., Dwyer, T., Feb. 2021. Shared surfaces and spaces: collaborative data visualisation in a Co-located immersive environment. IEEE Trans. Vis. Comput. Graph. 27 (2), 1171-1181. https://doi.org/10.1109/TVCG.2020.3030450.

Li, Gang, Liu, Yue, Wang, Yongtian, Xu, Zhaoji, 2017. Evaluation of labeling layout method for image-driven view management in augmented reality. In: Proceedings of the 29th Australian Conference on Computer-Human Interaction (OzCHI '17). New York, NY, USA. Association for Computing Machinery, pp. 266-274. https://doi.org/10.1145/3152771.3152800.

Li, Gang, Liu, Yue, Wang, Yongtian, 2018. An empirical evaluation of labelling method in augmented reality. In: Proceedings of the 16th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI '18). New York, NY, USA, 7. Association for Computing Machinery, pp. 1-9. https://doi.org/10.1145/3284398.3284422. Article.

Lin, Tica, Singh, Rishi, Yang, Yalong, Nobre, Carolina, Beyer, Johanna, Smith, Maurice A., Pfister, Hanspeter, 2021. Towards an understanding of situated AR visualization for basketball free-throw training. In: Proceedings of the 21st CHI Conference on Human Factors in Computing Systems (CHI '21). New York, NY, USA, 461. Association for Computing Machinery, pp. 1-13. https://doi.org/10.1145/3411764.3445649. Article.
Jiazhou Liu, Arnaud Prouzeau, Barrett Ens, and Tim Dwyer. 2020. Design and evaluation of interactive multi-scales data visualisation in immersive spaces. 588-597. doi: 10.1109/VR46266.2020.00081.

Liu, Jiazhou, Prouzeau, Arnaud, Ens, Barrett, Dwyer, Tim, 2022. Effects of display layout on spatial memory for immersive environments. In: Proc. ACM Hum.-Comput. Interact., 576, p. 21. https://doi.org/10.1145/3567729, 6, ISS, Articlepages.

Liu, Jiazhou, Ens, Barrett, Prouzeau, Arnaud, Smiley, Jim, Kara Nixon, Isobel, Goodwin, Sarah, Dwyer, Tim, 2023. DataDancing: an exploration of the design space for visualisation view management for 3D surfaces and spaces. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). New York, NY, USA, 379. Association for Computing Machinery, pp. 1-17. https://doi.org/10.1145/3544548.3580827. Article.

Luo, Weizhou, Lehmann, Anke, Widengren, Hjalmar, Dachselt, Raimund, 2022. Where should we put it? Layout and placement strategies of documents in augmented reality for collaborative sensemaking. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22). New York, NY, USA, 627. Association for Computing Machinery, p. 16. https://doi.org/10.1145/3491102.3501946. Articlepages.

Luo, Weizhou, Yu, Zhongyuan, Rzayev, Rufat, Satkowski, Marc, Gumhold, Stefan, McGinity, Matthew, Dachselt, Raimund, 2023. Pearl: physical environment based augmented reality lenses for In-situ Human movement analysis. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). New York, NY, USA, 381. Association for Computing Machinery, pp. 1-15. https://doi.org/10.1145/3544548.3580715. Article.

Maquil, V., May 2016. Towards understanding the design space of tangible user interfaces for collaborative Urban planning. Interact. Comput. 28 (3), 332-351. https://doi.org/10.1093/iwc/iwv005.

McNeill, Graham, Hale, Scott A., 2017. Generating tile maps. Comput. Graph. Forum 3 (June 2017), 435-445. https://doi.org/10.1111/cgf.13200. 36.

Mitasova, H., Mitas, L., Ratti, C., Ishii, H., Alonso, J., Harmon, R.S., 2006. Real-time landscape model interaction using a tangible geospatial modeling environment. IEEE Computer. Graph. Appl. 26 (4), 55-63. https://doi.org/10.1109/MCG.2006.87. July-Aug.

Moroney, W.F., Biers, D.W., Eggemeier, F.T., Mitchell, J.A., May 1992. A comparison of two scoring procedures with the NASA task load index in a simulated flight task. In: Proceedings of the IEEE 1992 National Aerospace and Electronics Conference, 2, pp. 734-740 pages.

Niyazov, Aziz, Mellado, Nicolas, Barthe, Loic, Serrano, Marcos, 2021. Dynamic decals: pervasive freeform interfaces using constrained deformable graphical elements. In: Proc. ACM Hum.-Comput. Interact., 493, p. 27. https://doi.org/10.1145/3488538, 5, ISS, Articlepages.

Niyazov, Aziz, Ens, Barrett, Satriadi, Kadek Ananta, Mellado, Nicolas, Barthe, Loic, Dwyer, Tim, Serrano, Marcos, 2023. User-driven constraints for layout optimisation in augmented reality. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). New York, NY, USA, 35. Association for Computing Machinery, pp. 1-16. https://doi.org/10.1145/3544548.3580873. Article.

Nuernberger, B., Ofek, E., Benko, H., Wilson, A., 2016. SnapToReality: aligning augmented reality to the real world. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). New York, NY, USA. Association for Computing Machinery, pp. 1233-1244. https://doi.org/10.1145/2858036.2858250.

Nusrat, Sabrina, Alam, Jawaherul, Kobourov, Stephen, 2020. Recognition and recall of geographic data in cartograms. In: Proceedings of the International Conference on Advanced Visual Interfaces (AVI '20), 29. Association for Computing Machinery, New York, NY, USA, Article, pp. 1-9. https://doi.org/10.1145/3399715.3399873.

Pavanatto, L., North, C., Bowman, D., Badea, C., Stoakley, R., 2021. Do we still need physical monitors? An evaluation of the usability of AR virtual monitors for productivity work. 2021 IEEE Virtual Reality and 3D User Interfaces (VR), pp. 759-767. https://doi.org/10.1109/VR50410.2021.00103.

Perelman, Gary, Serrano, M., Bortolaso, C., Picard, C., Derras, M., Dubois, E., 2019. Combining tablets with smartphones for data analytics. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., Zaphiris, P. (Eds.), Human-Computer Interaction - INTERACT 2019. INTERACT 2019, Human-Computer Interaction - INTERACT 2019. INTERACT 2019, 11749. Springer, Cham. https://doi.org/10.1007/978-3-030-29390-1_24. Lecture Notes in Computer Science().

Perelman, Gary, Dubois, Emmanuel, Probst, Alice, Serrano, Marcos, 2022. Visual transitions around tabletops in Mixed reality: study on a Visual acquisition task between vertical virtual displays and horizontal tabletops. In: Proc. ACM Hum.-Comput. Interact., 585, p. 20. https://doi.org/10.1145/3567738, 6, ISS, Articlepages.

Prouzeau, Arnaud, Lhuillier, Antoine, Ens, Barrett, Weiskopf, Daniel, Dwyer, Tim, 2019. Visual link routing in immersive visualisations. In: Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces (ISS '19). New York, NY, USA. Association for Computing Machinery, pp. 241-253. https://doi.org/10.1145/3343055.3359709.

Saidi, Houssem, Serrano, Marcos, Irani, Pourang, Dubois, Emmanuel, 2017. Tdome: a touch-enabled 6DOF interactive device for multi-display environments. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). New York, NY, USA. Association for Computing Machinery, pp. 5892-5904. https://doi.org/10.1145/3025453.3025661.

Saidi, Houssem, Dubois, Emmanuel, Serrano, Marcos, 2021. HoloBar: rapid command execution for head-worn AR exploiting around the field-of-view interaction. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). New York, NY, USA, 745. Association for Computing Machinery, pp. 1-17. https://doi.org/10.1145/3411764.3445255. Article.

Satkowski, M., Rzayev, R., Goebel, E., Dachselt, R., 2022. ABOVE & BELOW: investigating ceiling and floor for augmented reality content placement. In: 2022

IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Singapore, Singapore, pp. 518-527. https://doi.org/10.1109/ISMAR55827.2022.00068.

Satriadi, K.A., Ens, B., Cordeil, M., Jenny, B., Czauderna, T., Willett, W., 2019. Augmented reality map navigation with freehand gestures. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Osaka, Japan, pp. 593-603. https://doi.org/10.1109/VR.2019.8798340.

Satriadi, K.A., Ens, B., Cordeil, M., Czauderna, T., Jenny, Bernhard, 2020. Maps around me: 3D multiview layouts in immersive spaces. In: Proc. ACM Hum.-Comput. Interact., 201, p. 20. https://doi.org/10.1145/3427329, 4, ISS, Articlepages.

Satriadi, Kadek Ananta, Cunningham, Andrew, Thomas, Bruce H., Drogemuller, Adam, Odi, Antoine, Patel, Niki, Aston, Cathlyn, Smith, Ross T., 2022. Augmented scale Models: presenting multivariate data around physical scale models in Augmented reality. In: 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 54-63. https://doi.org/10.1109/ISMAR55827.2022.00019.

Satriadi, Kadek A., Cunningham, Andrew, Smith, Ross T., Dwyer, Tim, Drogemuller, Adam, Thomas, Bruce H., 2023a. ProxSituated visualization: an extended model of situated visualization using proxies for physical referents. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). New York, NY, USA, 382. Association for Computing Machinery, pp. 1-20. https://doi.org/10.1145/3544548.3580952. Article.

Satriadi, Kadek Ananta, Ens, Barrett, Goodwin, Sarah, Dwyer, Tim, 2023b. Active proxy dashboard: binding physical referents and abstract data representations in situated visualization through tangible interaction. In: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23). New York, NY, USA, 23. Association for Computing Machinery, pp. 1-7. https://doi.org/10.1145/3544549.3585797. Article.

Shneiderman, B., 1996. The eyes have it: a task by data type taxonomy for information visualizations. In: Proc. of the 1996 IEEE Symposium on Visual Languages, 96. IEEE Computer Society, pp. 336-343. VL'.

Silva, Samuel, Sousa Santos, Beatriz, Madeira, Joaquim, 2011. Using color in visualization: a survey. Comput. Graph. 35 (2), 320-333. https://doi.org/10.1016/j.cag.2010.11.015. ISSN 0097-8493.

Simon, Florine, Roudaut, Anne, Irani, Pourang, Serrano, Marcos, 2019. Finding information on non-rectangular interfaces. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). New York, NY, USA, 102. Association for Computing Machinery, pp. 1-8. https://doi.org/10.1145/3290655.3300332. Paper.

Taher, Faisal, Hardy, John, Kárník, Abhijit, Weichkel, Christian, Jansen, Yvonne, Hornbæk, Kasper, Alexander, Jason, 2015. Exploring interactions with physically dynamic bar charts. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). New York, NY, USA. Association for Computing Machinery, pp. 3237-3246. https://doi.org/10.1145/2702123.2702604.

Vainionpäi, Fanny, Kinnula, Marianne, Kinnula, Atté, Kuutti, Kari, Hosio, Simo, 2022. HCI and Digital twins - A critical look: a literature review. In: Proceedings of the 25th International Academic Mindtrend Conference (Academic Mindtrend '22). New York, NY, USA. Association for Computing Machinery, pp. 75-88. https://doi.org/10.1145/3569219.3569376.

Valkanova, Nina, Jorda, Sergi, Moore, Andrew Vande, 2015. Public visualization displays of citizen data: design, impact and implications. Int. J. Hum. Comput. Stud. 81, 4-16. https://doi.org/10.1016/j.ijhcs.2015.02.005.

VandenBos, G.R., 2009. Publication Manual of the American Psychological Association, 6th edn. American Psychological Association, Washington, DC. https://doi.org/10.1037/0000165-000.

Vock, Katja, Hubenschmid, Sebastian, Zagermann, Johannes, Butscher, Simon, Reiterer, Harald, 2021. IDIAR: augmented reality dashboards to supervise mobile intervention studies. In: Proceedings of Mensch and Computer 2021 (MuC '21). New York, NY, USA. Association for Computing Machinery, pp. 248-259. https://doi.org/10.1145/3473856.3473876.

Walker, Will, Sung, Hyungie, Ong, Chris Kevin, Casalegno, Federico, 2017. Exploring spatial meaning with a tangible map. In: Proceedings of the 6th ACM International Symposium on Pervasive Displays (PerDis '17). New York, NY, USA, 6. Association for Computing Machinery, pp. 1-8. https://doi.org/10.1145/3078810.3078826. Article.

Wang, Fengyusheng, Chang, Chia-Ming, Igarashi, Takeo, 2023. Flying names: manipulating name tags in XR Social environments for target selection tasks. In: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23). New York, NY, USA, 139. Association for Computing Machinery, pp. 1-8. https://doi.org/10.1145/3544549.3585677. Article.

Wannamaker, Kendra A., Kollannur, Sandeep Zechariah George, Dörk, Marian, Willett, Wesley, 2021. I/O bits: user-driven, situated, and dedicated self-tracking. In: Proceedings of the 2021 ACM Designing Interactive Systems Conference (DIS '21). New York, NY, USA. Association for Computing Machinery, pp. 523-537. https://doi.org/10.1145/3461778.3462138.

Wen, Z., Zeng, W., Weng, L., Liu, Y., Xu, M., Chen, W., Jan. 2023. Effects of view layout on situated analytics for multiple-view representations in immersive visualization. IEEE Trans. Vis. Comput. Graph. 29 (1), 440-450. https://doi.org/10.1109/TVCG.2022.3209475.

Sean White, Stephen Feiner, &J. Kopylec. (2006). Virtual vouchers: prototyping a mobile augmented reality user interface for botanical species identification. 119-126. 10.1109/TRIDUI.2006.1618281.

White, Sean, Feiner, Steven, 2009. SiteLens: situated visualization techniques for urban site visits. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). New York, NY, USA. Association for Computing Machinery, pp. 1117-1120. https://doi.org/10.1145/1518701.1518871.
Whitlock, Matt, Wu, Keke, Szafir, Danielle Albers, 2019. Designing for mobile and immersive visual analytics in the field. IEEE Trans. Vis. Comput. Graph. https://doi.org/10.1109/TVCG.2019.2934282.

Wieland, J., García, R.C.H., Reiterer, H., Feuchtner, T., 2022. Arrow, bézier curve, or halos? - comparing 3D out-of-view object visualization techniques for handheld augmented reality. In: 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Singapore, Singapore, pp. 797-806. https://doi.org/10.1109/ISMAR55827.2022.00098.

Willett, Wesley, Jansen, Yvonne, Dragicevic, Pierre, 2016. Embedded data representations. IEEE Trans. Vis. Comput. Graph. 23, 1. https://doi.org/10.1109/TVCG.2016.2598608. -1.

Zhou, Michelle X., Feiner, Steven K., 1998. Visual task characterization for automated visual discourse synthesis. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '98). USA. ACM Press/Addison-Wesley Publishing Co., pp. 392-399. https://doi.org/10.1145/274644.274698

Zollmann, Stefanie, Poglitsch, Christian, Ventura, Jonathan, 2016. VISGIS: dynamic situated visualization for geographic information systems. In: 2016 International Conference on Image and Vision Computing New Zealand (IVCNZ). Palmerston North, New Zealand, pp. 1-6. https://doi.org/10.1109/IVCNZ.2016.7804440.