# Understanding affordances in XR interactions through a design space

Victor Vieira, Eder Oliveira, Aline Menin, Esteban Clua, Marco Winckler, Daniela Trevisan

► To cite this version:

Victor Vieira, Eder Oliveira, Aline Menin, Esteban Clua, Marco Winckler, et al. Understanding affordances in XR interactions through a design space. IHC 2024 - XXIII Brazilian Symposium on Human Factors in Computing Systems, Oct 2024, Brasília DF Brazil, France. pp.1 - 14, 10.1145/3702038.3702112. hal-04875209

HAL Id: hal-04875209

https://inria.hal.science/hal-04875209v1

Submitted on 8 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Understanding affordances in XR interactions through a design space

Victor Vieira

victor.vieira@ifac.edu.br

Instituto Federal do Acre

Rio Branco, Brazil

Eder Oliveira

eder_oliveira@id.uff.br

Universidade Federal Fluminense

Niterói, Brazil

Aline Menin

aline.menin@inria.fr

Université Côte d'Azur

Sophia Antipolis, France

Esteban Clua

esteban@ic.uff.br

Universidade Federal Fluminense

Niterói, Brazil

Marco Winckler

marco.winckler@inria.fr

Université Côte d'Azur

Sophia Antipolis, France

Daniela Trevisan

daniela@ic.uff.br

Universidade Federal Fluminense

Niterói, Brazil

## ABSTRACT

This paper presents a qualitative analysis of a literature review on affordances in XR environments. Affordances refer to the way how the system communicates to the user how to perform a task. Preliminary studies showed that there was no consensus and neither a standard taxonomy for describing affordances in XR. The goal of this study was to investigate how affordances were used as a strategy to promote discovery and usage in XR environments. We collected and analyzed works that describe affordance concepts and carry out experiments in 3D environments describing the applied affordances. We mapped the affordances usage in 8 dimensions: kind of environment, modality, location, temporal aspect, learning aspect, category of tasks, domain tasks and kind of affordance. Additionally, findings were categorized according to VR tasks and Bloom's learning existing taxonomies. Finally, the relationship between the dimensions was analyzed and discussed as a tentative reflection and guidance on the future design and development of affordance taxonomy for XR environments.

## CCS CONCEPTS

* Human-centered computing → Mixed / augmented reality; Virtual reality.

## KEYWORDS

affordances, XR environments, discovery, strategies, taxonomy

**ACM Reference Format:**

Victor Vieira, Eder Oliveira, Aline Menin, Esteban Clua, Marco Winckler, and Daniela Trevisan. 2024. Understanding affordances in XR interactions through a design space. In *XXIII Simpósio Brasileiro de Fatores Humanos em Sistemas Computacionais (IHC 2024)*, 7 a 11 de outubro de 2024, Brasília - DF, Brasil. ACM, New York, NY, USA, 14 pages. https://doi.org/10.1145/3702038.3702112

## 1 INTRODUCTION

This paper investigates the use of affordances in three-dimensional (3D) virtual environments and more specifically, 3D environments that rely on the use of Head Mounted Displays (HMDs) to create applications in Virtual Reality (VR), Augmented Reality (AR), and/or Mixed Reality (MR), which make up Extended Reality (XR) [20, 45, 46, 50]. XR environments offer unique opportunities for creating immersive and interactive experiences that can be applied in diverse areas, from education to entertainment [8, 14]. However, the effectiveness of these experiences largely depends on the user's ability to understand and interact effectively with the virtual environment [27]. In this context, the concept of affordance emerges as a fundamental piece to improve the user experience (UX) [4, 39].

The concept of affordance, derived from psychology and popularized by James Gibson [17], refers to the clues or suggestions that the environment offers to an observer about how it can be used. In Human-Computer Interaction (HCI), the term began to be used when Donald Norman published the book *The Design of Everyday Things*, conceptualizing affordance as the quality of an object or environment that suggests how it should be used [33]. However, despite its potential importance, the concept of affordance in XR is not yet well established. There is a diversity of interpretations and approaches [5, 21, 38, 43, 49], which made it crucial to carry out a Systematic Literature Review (SLR) to investigate how researchers have explored this concept in different contexts.

As we did not find other works on affordances in XR with a similar objective to ours, we carried out this research to identify current trends in the scientific community. Our objective was to investigate what is being researched and used in terms of affordance. In this sense, we believe that this work, as a literature review, is valid.

When discussing the use of affordances as design strategies, we refer to the way designers apply them to create immersive environments. There is a vast literature describing the use of affordances. Nonetheless, the review of the literature presented hereafter is mainly focused on articles describing XR systems that employ affordances to create user experiences of *discovery*, which refer to the ability to learn how to use the system to complete the proposed tasks. Based on an in-depth analysis selected papers from the literature review, we could identify eight dimensions that characterize affordances used in learning tasks (which are related to the concept of discovery in XR environments).

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only. Request permissions from owner/author(s).

IHC 2024, 7 de outubro a 11 de outubro de 2024, Brasília - DF, Brasil

© 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.

ACM ISBN 979-8-4007-1224-1/24/10

https://doi.org/10.1145/3702038.3702112
The results of the analysis of the literature provides a description on how researchers are exploring affordances in XR. Our analysis allows to understand how affordances can be explored as effective strategies to improve discovery in 3D virtual environments. This included investigating how different types of affordances can be applied in different contexts and how the use of these affordances affected the UX and the usage learning process. Finally, this paper prompts to emerging trends in the research in the field by analyzing how established dimensions interact and correlate. The data were analyzed quantitatively, through diagrams and graphs, and qualitatively, through grounded theory approach including thematic analysis [37, 47, 52].

The rest of the text is organized as follows: Section 2 presents a concept of affordance in XR is suggested based on literature insights; Section 3 presents a SLR of affordances in XR, describing the planning, conduction and data analysis; Section 4 presents the design space of the research, describing the dimensions and classification of the articles; Section 5 provides a discussion of the results; Section 6 presents insights into affordances in XR environments that emerged from SLR, such as opportunities and challenges; Finally, Section 7 addresses conclusions, research limitations and recommendations for future work.

## 2 DEFINITION OF AFFORDANCES IN IMMERSIVE ENVRIONEMENTS

In this section, we explore the concept of affordance in XR environments, highlighting its origins in psychology, its evolution in the area of Human-Computer Interaction (HCI) and its specific application in XR. Furthermore, we analyzed works that described affordances in XR to identify insights that contributed to a definition of the concept.

### 2.1 Origins: psychology and perception

The concept of affordance has its roots in psychology in the 70s, having been proposed and popularized by James Gibson [17]. The term is derived from the English "afford", which means "to provide" [1]. It intrinsically relates to the perception, cognition and interaction of human beings with the environment around them [16].

At its core, affordance refers to the clues or suggestions that the environment offers to an observer about how it can be used [17]. In the field of psychology, can still be described as "a resource or support that the environment offers" [51] and "factors of perception and action that make interfaces easy to learn and use" [16].

To understand affordances, human beings use their senses as sensors, interpreting information through vision, hearing, touch, smell and taste, to perceive and understand the possibilities of action in the world around them.

### 2.2 Affordance in HCI

In the area of HCI, the concept of affordance gained notoriety with the publications of Donald Norman [33, 34]. He conceptualized affordance as the quality of an object or environment that suggests how it should be used. A classic example brought from the real world by Norman is that of a door with a handle: the handle suggests being turned, indicating the action necessary to open the door. This

approach highlights the importance of affordances in the UX and usability of information systems, as design elements that favor interaction are implemented.

Other HCI authors define affordance as "the relationship between the properties of a person and the perceptual properties of an object in the environment" [41] and "the attribute of an object that allows people to know how to use it" [42]. Rogers' [41] definition emphasizes the bidirectional relationship between the user and the object, highlighting that affordances are influenced by the characteristics of the object and the user's perceptual capabilities. Sharp's definition [42] reinforces the idea that affordances are closely linked to the user's perception and understanding of the environment and objects. One factor highlighted is the use of reality. The authors mention that the use of reality in the interface can make beginners more comfortable when learning to use an application, highlighting the importance of affordances in facilitating learning.

Affordance concept in HCI highlight the relationship between perceptual elements and user actions. They emphasize the importance of clear and intuitive perceptual cues to guide the user in interacting with objects and environments[33], but they also bring additional nuances, such as bidirectionality [41] and the ability of affordances to provide comfort and facilitate learning [42].

### 2.3 Insights About Affordance Concepts in XR

Although widely established in HCI for interfacing traditional systems, the concept of affordance is still in development when applied to immersive environments.

We found few works that explicitly describe affordance concepts [5, 9, 15, 21, 38, 40, 49]. This suggests that, although most works address affordances in some way, a clear definition of this concept is still a gap in the literature. Among the explicit concepts, a keyword analysis highlighted the following terms: "object", "possible", "learned", "user", "target", "actions", "through", "use" and "perception".

The explicit concepts found in the literature provide valuable insights into the definition of affordance in XR. They highlight the emphasis on the user's ability to understand objects and relationships in the 3D environment, the influence of affordances on the user's actions and the potential to stimulate the neurological system. Furthermore, individual perception of what is possible and important, together with the preparation of actions through perception, are fundamental elements. An analysis of the concepts based on coding [52] and the results of the work also allowed us to understand that there is a direct relationship between the use of affordances and the completion of objective tasks in immersive environments.

Other works that address affordance in XR were also found. Stefan [46] built on Gibson's seminal work [17] to create a generalized affordances framework for immersive activities in relation to the affordances of physical reality. To do this, they compared the use of affordances in the real world with AR and VR. The framework facilitates the comparison of virtualized activities with non-virtualized activities and the comparison of similar activities between VR and AR. The work also highlights that designers can find success by focusing development on the specific affordances that VR or AR are best equipped to realize. However, the concept of affordance is
not explicitly described in the article and can be understood as any resource contained in the system, capable of replicating of a task as it occurs in the real world. A limitation of the article in relation to the study of affordances is the fact that no aspects related to UX were evaluated, other than those related to the replication of real-world affordances. It is understood that the possibilities in immersive environments go beyond those found in the physical environment, and that this enhances the use of affordances in XR.

Gu et al. [19] analyzed the implementation of XR technologies for autistic individuals based on technological affordances, highlighting the lack of studies on linguistic autonomy and participants' comfort. The work also does not present an explicit definition of the term affordance, but relates it to the resources that the system offers to users to generate autonomy (the ability to perform tasks alone).

Shin [43] wrote about the theory of affordance concretization, describing that an affordance does not produce results without concretization, which refers to the actions performed by users as they take advantage of one or more affordances through their interaction with technology to achieve a direct concrete result in support of their motivations. In this sense, the ability of users to perceive the possibilities of the environment is a fundamental aspect.

Although with a different focus than the present study on affordances in XR environments, Assaf et al. [3] examines the complexities of collaboration in XR and explores innovative visual cues that have recently been employed in research to enhance user understanding. Although it does not directly focus on affordances such as technological resources, the article offers valuable insights into how visual elements can facilitate effective collaboration in XR environments. The findings presented reinforce the importance of understanding how users perceive and interact with the collaborative environment, thus contributing to the field of study of affordances in immersive realities, albeit from a different perspective.

The works cited reinforce the fact that there is no well-established consensus on the definition of affordance in XR and present insights into: a) affordance as a technological resource of the system; b) affordance needing to be perceived, understood; c) affordance being useful for completing tasks. We suggest this concept below.

## 2.4 Discussion of Definition of Affordance in XR

We propose to have a discuss about the definition of affordance in XR through the origins of the term, contributions from authors in the field of HCI, literature analysis and insights from explicit concepts.

In fact, some of the affordances as defined by Gibson and Norman work very well in immersive systems, but there are special cases. Because of these special cases we suggest understanding affordance in XR as a **technological resource perceived by one of the human senses that explicitly aims to favor the completion of a task in the immersive environment**. This definition highlights the interaction between the user and the XR environment, where affordances are elements that, when perceived by the human senses, guide and facilitate the execution of tasks, thus contributing to the improvement of the user experience and the learning process.

What we understand from Gibson's definition [17] is that the object's design can include characteristics that allow us to understand how it works. This works well when we design physical objects or for objects that we can directly manipulate. However, when we work with XR, we are often including elements as plugins to existing objects to create "shapes", like a notation. Often, this affordance is a special type that allows you to create a connection between the virtual world and the real world.

Gibson's definition of affordance is related to the possibilities of action and perceived results. In the context of XR, there is a dissociation between the perception of immersive reality (an artificial image that does not configure something real) and affordances based on objects in the real world. In the case of AR, for example, affordances are placed on the object to explain what it can do. In other words, the design and shape of the object in the real world are its affordance. When we add something so that the person understands, we are not transforming the object. This concept is different and we found evidence that it is a topic that deserves to be discussed with the community. In this aspect, affordances come as something additional that can be placed in the environment or on an object, for example.

It should also be noted that this is not the same as what happens with 2D interfaces, where the effects activated by interaction with objects, such as hover for buttons activated by the mouse cursor, are part of the object's properties. Gibson [17] and Norman [33] [34] talks about design and what can be perceived from the design of the object. What we are suggesting as an affordance in XR is something superimposed on the original design of the object in the virtual world, different from a hover on a button in a 2D interface.

An example of affordance in XR that favors the understanding of the proposed concept are the limits in the virtual environment generated by the impossibility of moving beyond them in the real world. As there is no way to modify the real world and expand these limits, an object like a wall is superimposed when we get close. These affordances are necessary because they create a connection with the real world, which, in this case, cannot be modified. In this sense, we can consider that an association occurs from the dissociation created by the immersive reality.

While the concept of communicability from Semiotic Engineering provides valuable insights into the interaction between users and technological resources and also address the concept of affordance [31], the notion of affordance presented still offers a unique perspective on how users perceive and interact with immersive environments.

We seek to embrace the complexity of affordances in immersive environments and establish a solid conceptual foundation for exploring their role in promoting learning and interaction effectiveness in XR environments. It recognizes the importance of perceptual cues provided by human senses and their impact on the interaction between the user and the immersive environment. In the next sections, we explore how this definition relates to the data collected and how affordances can be applied effectively in different XR contexts.
# 3 SYSTEMATIC LITERATURE REVIEW OF AFFORDANCES IN XR

To understand the current state of research related to affordances as a strategy for discovery in XR, we conducted a SLR guided by Keele [23]. Our analysis was inspired by the grounded theory approach [37, 47, 52]. For planning and conducting the review, we use Parsifal<fnref n="1" />. For analysis, we used Google Sheet<fnref n="2" /> to tabulate data and create graphs and MGExplorer [29] to create a NodeEdge diagram of the relationship between the works and the affordance modality included in them.

## 3.1 Planning

Initially, we were interested in understanding the concepts of affordances in XR and how affordances were being used. To achieve this, we defined the following research questions (RQ):

* RQ01 What are the concepts of affordances in XR?

* RQ02 What kinds of affordances are being used in XR?

* RQ03 What purposes are affordances being used in XR?

We decided to collect papers from digital libraries, including the IEEE Xplore, the ACM Digital Library and the ScienceDirect database, because they have conferences and articles relevant to the research context. We inspected an initial set of articles, including titles, abstracts and main texts if necessary, to evaluate whether the criteria allowed for a well-balanced search or whether we needed to revise the initial search query. We realize multiple iterations to reach the final collection. Our search used boolean search queries with multiple search terms within the categories of XR, affordance and discovery. We used the following query for searching ScienceDirect and similar variants for IEEE and ACM.

TITLE=ABS-KEY(\"virtual reality\" OR "VR" OR "augmented reality" OR "AR" OR "mixed reality" OR "MR" OR "extended reality" OR "XR" OR "immersi* environment*" OR "3D virtual environment" OR "3D environment" OR "metaverse") AND ("affordance") AND ("discover*" OR "learn*" OR "user experience"))

One of the main motivations for such a configuration in the search string was to observe how people understood and used the word affordance within the context of immersive environments. We chose to use the term affordance to narrow the search and check how people are using it in XR.

Regarding the inclusion criteria, we included works that address experiences in immersive environments using HMDs and provide empirical or theoretical contributions to understanding affordances in XR. Exclusion criteria were:

* EC01 Affordance approach beyond the concept established by Gibson [17] and Norman [33]

* EC02 XR approach unrelated to immersive HMD experiences

* EC03 It does not present experiment related to the use of XR

* EC04 Short papers, technical reports, course conclusion works, dissertations, theses, posters, workshop proposals, tutorials and books

* EC05 It is not freely available, even after contacting the authors

* EC06 Language other than English

We use a question scheme with answers "yes" (1 point), "partially" (0.5 point) or "no" (0 points) to assess the quality of the articles. The maximum possible score was 7 points. This was useful for weighing the importance of individual studies when results were synthesized and served as a more detailed exclusion criteria. Articles with a score below 5 were excluded. The quality assessment (QA) was adapted from [23, 24] and involved the following questions:

* QA01 Title and abstract relevant to a research about affordances in XR?

* QA02 Clear objectives?

* QA03 Research Methods Described?

* QA04 Documented data collection and analysis?

* QA05 Clear and objective discussion of results?

* QA06 Research limitations described?

* QA07 Affordances clearly described?

The planning stage ended with the definition of the data extraction form from the articles. Four researchers went through an iterative coding process to derive orthogonal design dimensions of affordances in XR. The researchers inductively derived a stable consensus set of codes by inspecting a few random sampled papers through open, axial and selective coding [47]. Then, they looked for taxonomies that could cover the established dimensions. The dimensions and taxonomies that are part of the extraction form are described in detail in Subsection 3.3 and Section 4. The fields "affordance concept obtained in the article" and "classification of this concept (explicit or not)" completed the extraction form.

## 3.2 Conducting the review of the literature

The search was conducted in May 2023. We search titles, abstracts and keywords of scientific articles published at any time. We did not temporally limit our analysis to discover how long researchers have been exploring affordances as a tool to influence discovery in XR. Removing duplicate, we obtained an initial set of 216 articles.

We guided our selection of the papers from the initial set by applying inclusion and exclusion criteria. In the selection phase, 77 articles were included based on analysis of the title and abstract. After applying the exclusion criteria during the extraction phase, including the application of the quality questionnaire, 30 articles remained, which are studied in this work. Even though the number of articles can be considered small, we highlight our interest in carrying out an in-depth analysis of it. Two researchers evaluated each article in both phases.

## 3.3 Data analysis

The affordance concepts obtained in the articles were analyzed to answer RQ01. Explicit concepts were found in 7 (23%) works. They served to derive the concept that we established in Section 2.

During planning, eight dimensions related to affordances were considered important for the classification of works seeking to answer research questions RQ02 and RQ03: kind of environment,

<footnote n="1">¹Parsifal is available in https://parsif.al/ and is an online tool where geographically distributed researchers can work together within a shared workspace, designing the protocol and conducting the research.</footnote>

<footnote n="2">²Google Sheet is available in https://www.google.com/sheets and is a free, full-featured spreadsheet program maintained by Google.</footnote>
modality of interaction with the affordance, location of the affordance, temporal aspect, learning aspects, category of tasks, domain tasks and type of affordance. They were obtained by identifying emerging themes and codes as researchers analyzed an initial set of articles, as described previously, in Section 3.1. Based on the dimensions, the researchers looked for taxonomies that could support the classifications. Table 1 presents a relationship between the dimensions and taxonomies used.

## 3.4 Quantitative analysis

Regarding the classification of accepted works, all present primary research. This is due to the fact that we did not find secondary research describing experiences in XR environments with sufficient information to extract affordances (EC03 and QA07).

Among the accepted works, the oldest is from 2017. We highlight the growth of interest in research involving affordances in XR. The most recurrent sources were IEEE Conference Virtual Reality and 3D User Interfaces (3), Computers & Education (3) IEEE International Symposium on Mixed and Augmented Reality (2), IEEE Transactions on Visualization and Computer Graphics (2), Transactions on Learning Technologies (2) and Advanced Engineering Informatics (2).

Other data are related to the categories of each dimension presented in Table 1, which constitutes our design space. We built this design space as a way to explain what we found in the literature. The Figure 1 presents a radar chart with the percentage of category within each dimension. We discussed the percentages that appear on this radar chart in Section 4.

# 4 XR AFFORDANCES DESIGN SPACE

The dimensions of design broadly fall into three main categories: how, why and what. How is the affordance comprised: kind of environment, modality of interaction, location of the affordance and temporal aspect. Why is the affordance comprised: learning aspects, category of tasks and domain tasks. What is the affordance described what is the type of affordance. Figure 1 presents a radar chart with the configuration of the articles within the dimensions found. Following we describe the definition of each dimension and the current state of using affordances in XR.

## 4.1 Kind of Environment

The type of environment was classified based on the nature of the immersive interaction and the device used. Figure 2 shows the difference between the environments. Considering the XR context, we classify articles as VR, AR or MR [50]. We obtained 60% (18/30) of the works using VR environments. The use of AR environments was not far behind, appearing in 46.6% (14/30) of works. However, the numbers highlight greater interest from the community in research involving affordances in VR compared to research with AR. MR environments were explored in 10% (3/30) of the works.

Four studies (4/30) compared the use of VR and AR, and one study compared the use of the three types of environments. He highlights that the user tends to complete the task more easily depending on how stimulating and immersive the environment is [21].

## 4.2 Modality

We classify the modality of interaction with the affordance based on an adaptation of the taxonomy in Hertel et al. [22], establishing the modalities: visual - vision, gaze; sound - hearing; and haptics - touch. Visual appeared in 100% (30/30) of the works. Although it is not a universal rule, this is motivated by the fact that all environments have graphical interfaces, which tend to be the system's main form of communication with the user. Sound appeared in 50% (15/30) of the works, as well as haptic. Figure 3 presents examples of the three modalities used to design interactive affordances.

The use of haptic affordances has generally been associated with devices other than the HMD, serving to control the system or receive haptic feedback [26]. The fact that they are modalities associated with sensors that are not the most used in 3D interaction and appear in most works raises the possibilities due to the use of the other two human senses, which are not currently explored due to technological limitations: smell and taste. In an ideal technological scenario for a representation closer to reality, visual, sound and haptic modalities are used [46]. The next step involves developing technology to support other human senses.

The combination of the three modalities was present in 13.3% (4/30) of the works. It allowed us to favor discovery [9, 28] and UX [49], proving one of our research hypotheses.

When studying affordances in XR environments, it is crucial to recognize that modality is a property of the affordance itself, not the user's interaction with it. This understanding implies that the modality is determined by the design of the affordance, shaping how it is perceived and how it can be used by users. On the other side, the interaction modality, which refers to the way users interact with the system (for example, by gestures, voice or traditional control), can significantly influence the design of these affordances. XR designers must therefore carefully consider how different interaction modalities impact the perception and effectiveness of affordances, ensuring that they are intuitive and promote a good user experience. This involves a detailed analysis of the context of use and user capabilities, as well as a skillful integration of the technologies involved to create an interaction.

## 4.3 Location

The location of the affordance, that is, where it is placed within the XR environment, is a crucial aspect in determining how users interact with it. We classify the location based on Belani et al. [6], which can be: in the environment, in an object, on a dashboard, or on a physical device.

The most common approach, in 66.6% (20/30) of the works, was the placement of affordances in virtual objects. This means that interaction elements were associated with specific objects, such as tools or interactive elements. This approach can be useful when you want to highlight specific objects as interaction targets [39, 44].

In 56.6% (17/30), the affordances were placed directly in the 3D virtual environment. This means that the interaction elements were incorporated into the virtual scenario itself, providing users with perceptual cues on how to interact with the environment. This approach can create a sense of immersion and naturalness in the interaction, as affordances are integrated into the context [6].
<table><caption>Table 1: Relationship between dimensions and taxonomies used for classification of works.</caption><thead><tr><td>Dimension</td><td>Description</td></tr></thead><tbody><tr><td>Kind of environment</td><td>Kind of environment was categorized based on the nature of the immersive interaction and the devices used, and could be classified as AR, VR or MR [50].</td></tr><tr><td>Modality</td><td>The modality of interaction with the affordance was categorized based on an adaptation of the taxonomy in Hertel et al. [22]. This taxonomy classifies different modalities of interaction based on human senses that can be used in immersive environments (visual, hearing and touch), deriving the categories visual, sound and haptic.</td></tr><tr><td>Location</td><td>The location of the affordance was categorized based on Belani et al. [6], where the authors tested different locations for anchoring content in XR: in the environment, on an object, on a physical device or on a dashboard.</td></tr><tr><td>Temporal aspect</td><td>The temporal aspect of affordances was categorized into two main classes, according to Cecil et al. [10]: static or dynamic. The static aspect refers to affordances that remain constant over time, while the dynamic aspect refers to affordances that can change over time.</td></tr><tr><td>Learning aspects</td><td>Learning aspects are related to the learning objective in using the affordance, that is, what the user should be able to do or achieve after completing the interaction with the affordance. The taxonomy in Anderson & Krathwohl [2] was adapted from Bloom's Taxonomy [7] and used for this classification. It presents a matrix of two high-level dimensions: knowledge and cognitive process. However, we perform the classification at a low-level. Section 4 presents the classification in detail.</td></tr><tr><td>Category of tasks</td><td>The task category was classified based on Laviola et al. [25]. The taxonomy classifies interaction techniques for tasks into three main categories: selection and manipulation, navigation, and system control. These are common, low-level user tasks in 3D interfaces. For each of these tasks, there are a large number of possible interaction techniques (combinations of input device and user interface).</td></tr><tr><td>Domain tasks</td><td>For this dimension, the classification involved a thematic analysis based on key words in articles that explicitly described affordance concepts [11, 37]. The categories generated were: learning, 3D exploration, mounting and communication.</td></tr><tr><td>Kind of affordance</td><td>The kind of affordance was classified as perceptible or hidden, based on Gaver [16]. Perceptible affordances are those that are easily identified and understood by users in an environment, generally highlighted or evident. On the other hand, hidden affordances are less evident and may require exploration or discovery by the user to be recognized. This is related to how it is activated. The term hidden should be understood as inactive.</td></tr></tbody></table>

In a significantly number of articles, equivalent to 76.6% (23/30), the affordances were incorporated into a dashboard that was available to users within the XR environment. These dashboards often contained controls, buttons and indicators that facilitated interaction and provided additional information to users. This approach can be effective for organizing and centralizing interaction elements [53].

Less common, but still present in 33.6% (11/30) of articles, the physical device was the location of the affordances, described in most cases as controls. This approach is mainly related to the holistic modality and can be valuable when seeking closer integration between the real and virtual world, as well as increasing the feeling of immersion [6, 46].

The majority of articles included in the SLR (28/30) adopted a diversified approach, placing affordances in two or more defined locations. This multi-location approach suggests a strategic adaptation of affordances based on the specific needs of each XR context.

## 4.4 Temporal Aspect

The temporal aspect dimension refers to the nature of the affordance behavior over time. We use Cecil et al.'s [10] work to classify works as containing static, dynamic affordances or both.

In 40% (12/30) of the articles analyzed, affordances were classified as static. This means that these interaction elements maintained a

fixed and unchanging nature during the user's interaction with the XR environment. However, it is important to highlight that all cases in which affordances were classified as static, dynamic affordances were also found. We deepened the research and realized that this type of affordance appeared when the representation of an object from the real world was taken into the environment to influence the understanding of the objective of the interaction, but the affordance remained unchanged over time, for example, as happened with a book represented in the environment at [4].

All articles (30/30) included in the SLR described the use of dynamic affordances. This means that these interacting elements showed some degree of mutability or adaptability over time. Dynamic affordances can adjust in response to user actions, changes in the virtual environment, or other contextual factors [10]. This flexibility allows for a richer interaction adapted to the user's needs.

Dynamic affordances have the potential to make interactions more engaging, adaptable and effective, contributing to UX and facilitating discovery in virtual three-dimensional environments [44].

## 4.5 Learning Aspects

In the dimension related to the learning aspects, we classify the works according to the learning objectives, using Bloom's Taxonomy [7], adapted by Anderson & Krathwohl [2]. This allowed us
<figure><img src="image_2.png" /><figcaption>Figure 1: Radar graph with the percents of papers in each category. "Others" in the "Learning aspects" dimension corresponds to the works classified as "fewer papers" in Figure 4.</figcaption></figure>

another way of seeing the results, considering the completeness of the taxonomy used. Figure 4 shows the distribution of items over the classification and the dimensions of knowledge and cognitive process. Each work was classified into only one of the low-level objectives, considering the need for an analysis of the work objective associated with the affordances used. In other words, the objective in interacting with the environment as a whole was considered, and not each affordance individually.

The results show that the most common learning objectives were use (23%), followed by recognize (17%) and identify (14%), respectively. It can be seen that two of the three objectives are classified as metacognitive in the knowledge dimension of the taxonomy.

In fact, what we mean with learning is that one of the main purpose of affordances is to facilitate how to use the objects or

environments. Because of that, we look into Bloom's Taxonomy to try to understand where affordances are situated in the framework.

## 4.6 Category of Tasks

The category dimension of 3D interaction tasks focuses on specific activities that users perform in XR environments. We use Laviola *et al.*'s [25] classification of 3D interaction tasks: selection and manipulation, navigation, and system control.

In 66.6% (20/30) of the articles analyzed, affordances were associated with selection and manipulation tasks. This indicates that affordances played a fundamental role in the selection of objects or elements within the XR environment and the manipulation of these objects. This category of tasks involves direct interactions with
<figure><img src="image_3.png" /><figcaption>Figure 2: Demonstration of affordances in XR environments. In (a) [32], in the VR environment, affordances are visualized as graphic elements in the virtual space, such as a circle marking the point of interaction and a directional arrow that guides the user to the next step. In (b) [30], in the AR environment, affordances are incorporated as interactive frames (dashboards) that float in the field of vision, providing contextual information necessary to perform tasks. Finally, in (c) [35], in the MR environment, affordances are presented as indications about physical objects that must be manipulated to progress in the task, through virtual elements pointing to the object.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 3: Different modalities of interaction with affordances [14]. In (a), the affordance is represented by a highlight on the screen on the elements selected by the visual interaction, guiding the user as to which elements are interactive. In (b), the affordance is displayed on the HMD anchored in the environment, showing a graphic element that indicates the gesture necessary for the interaction to occur. In (c), sound affordances are activated by voice commands, with the system responding with audio that guides the interaction, exemplifying a hidden affordance [22] that reveals features not immediately visible. In this figure, (a) and (b) are immersive and (c) are not.</figcaption></figure>

virtual objects, such as picking up, moving, resizing, or modifying elements in 3D space.

Navigation was present in 46.6% (14/30) of articles. This implies that the affordances were designed to facilitate exploration and movement within the virtual environment, in these cases. Navigation tasks may include moving around virtual space, changing visual perspective, interacting with maps, or exploring 3D environments.

Some affordances were directed towards system control tasks (12/30). This means that affordances were used to manage configurations, adjust parameters or control specific functionalities of the XR environment. These tasks may be related to personalizing the experience or configuring devices.

Furthermore, it is important to note that several works were classified into more than one task category, reflecting the multifaceted nature of interactions in XR environments [6, 28].

## 4.7 Domain Tasks

The domain tasks dimension focused on the specific activities that users performed in XR environments. A thematic analysis helped identify four categories: learning, 3D exploration, mounting, and communication.

Learning (13/30) refers to tasks related to acquiring knowledge through interaction in XR environments. The affordances associated with this category are designed to help users understand concepts, acquire new skills or develop specific skills. This may include using affordances for learning educational content [36], professional training [9] or developing practical skills [28].

Mounting (4/30) involves tasks related to building, assembling or configuring objects within the XR environment and was the least listed category. In all cases it was related to AR environments and involved instructions inserted into dashboards or in the environment itself.

3D exploration (21/30) covers tasks related to exploring environments. The affordances associated with this category are designed
<figure><img src="image_5.png" /><figcaption>Figure 4: Percentage of classification of papers in Bloom's Taxonomy of Learning Objectives, adapted from [2].</figcaption></figure>

to assist users in navigating and interacting with 3D objects and scenarios. This may include using affordances to explore educational environments [53], virtual worlds [32] or entertainment scenarios [12].

Communication (6/30) involves tasks related to interacting with other users or entities within XR environments. The affordances found in this category are designed especially to assist users in collaborative tasks [18, 36, 44] and in interacting with immersive environments for feedback from the user [12, 32, 48].

The distribution of domain tasks highlights the versatility of affordances in XR environments, where they can play fundamental roles in a wide range of activities, from formal learning to 3D exploration in entertainment environments. The prevalence of the "3D exploration" category reflects the immersive nature of these environments and the importance of affordances that support spatial exploration and interaction with three-dimensional objects. This

understanding of domain tasks helps to contextualize the use of affordances in XR and how they can contribute to user experience and discovery in different scenarios.

## 4.8 Kind of Affordance

In kind of affordance, we focus on the nature of affordances in XR environments, based on Gaver [16], and classify whether they are explicitly perceived by the user or kept hidden.

The perceptible category (30/30) involves affordances that are easily identified and recognized by users. These affordances are designed to be visible, intuitive and accessible, providing clear and explicit clues on how to interact with the XR environment [16]. Users can perceive these affordances through visual, auditory or haptic feedback. The presence of perceptible affordances in all works reflects the importance of providing clear cues for users, making interaction more intuitive [33].
On the other hand, the hidden category (10/30) includes affordances that are deliberately hidden or not easily noticeable at first glance. These affordances may require users to discover or explore the XR environment to identify them [16]. Although less obvious initially, hidden affordances can add a layer of depth and discovery to UX, creating immersive experiences that encourage exploration.

It is important to note that all works classified as containing hidden affordances also have perceptible affordances. This suggests that combining perceptible and hidden affordances can be an effective strategy for creating XR experiences that balance accessibility with elements of discovery and challenge [36].

# 5 DISCUSSION

Our investigation revealed a series of significant observations, among which we highlight the positive impact of affordances on XR discovery [9, 28, 36], as well as the diversity of approaches identified [5, 9, 15, 19, 21, 38, 40, 43, 46, 49]. Furthermore, we also address the implications of exploring affordances in XR, the connections in design space and the strategies for using affordances found.

## 5.1 Exploration of Affordances in XR

The 30 articles included in this SLR after refining the search string highlights that the discussion about the use of affordances explicitly in XR is still little explored. We believe this is related to two main factors.

First, the inherent complexity of XR environments, which often involve a variety of sensors, input devices, and multimodal interactions, may have discouraged researchers from explicitly addressing affordances in their investigations. Integrating affordances into XR requires a deep understanding of users cognitive and perceptual nuances, as well as the technical capabilities of XR systems.

The second factor to consider is the lack of a common language and consolidated guidelines for the conceptualization and implementation of affordances in XR. As a result, researchers may have approached the topic in a heterogeneous way, making it difficult to identify and systematically analyze relevant work.

These considerations highlight the continued need for research and development around XR affordances as the technology evolves and new challenges and opportunities emerge. This need can be reinforced by the fact that the articles included in this research are dated from 2017.

## 5.2 Connections in Design Space

The connections in design space presents relevant insights, among which the combination of affordances in different modalities with their positioning in different locations stands out. This combination resulted in: a) increased interactivity [32, 49]; b) facilitating discovery / improving understanding of the environment [9, 12, 14, 28]; c) efficiency in completing tasks [10, 13, 21, 26]. These gains were also present in the description of the results of the works that presented more categories [12, 32, 49].

We chose to explore data visualization in the modality dimension through a NodeEdge Diagram, which can be seen in Figure 5. In it, we also insert the relationship of the articles with another concept related to what we study: spatial affordance. We found relevant works that explicitly discuss this category of affordance and we

understand that it is an emerging term in the area. Spatial affordance is defined as the functionality that a place or space offers to user activity [44]. Studies have indicated that the spatial configuration of the immersive environment is one of the factors that can favor learning to use and completing tasks [6, 28, 39, 44].

The diagram provided a clear visualization of the trends and patterns found in relation to the interaction modalities in the included articles. It was evident that the visual modality is widely predominant, being used in all articles analyzed. On the other hand, sound and haptic modalities, although less common, still play a significant role in various XR contexts. Analyzing the visual distribution in the diagram, it is clear that few works are positioned at the edges of the graph, highlighting the relevance of the set of works in this review for exploring modalities of interaction with affordances in XR.

Only one work addresses the three modalities of interaction and spatial affordance [28]. Among the relevant results in it, the authors highlight that capturing interactions is a relevant factor for discovery as it allows the diagnosis of risky behaviors or knowledge deficiencies that can impact the interaction. They also present the possibility of multisensory interaction in MR environments, using the three interaction modalities.

Regarding the learning aspects, based on Bloom's learning objectives [2] (Figure 4), the fact that the majority of work was classified under the "metacognitive" knowledge dimension suggests that researchers are emphasizing the importance of self-reflection, critical thinking, and understanding cognitive strategies in learning experiences in XR.

The emphasis on the metacognitive knowledge dimension suggests that researchers are interested not only in providing direct information or instructions to users, but also in enabling them to develop skills that may be relevant in XR environments, where interaction and active exploration play a key role in discovery.

Furthermore, the fact that the second scored category in the knowledge dimension is conceptual indicates that researchers also value the construction of knowledge and the understanding of fundamental concepts. This combination of emphasis on metacognitive and conceptual dimensions suggests a holistic approach to learning in XR environments, which not only conveys information but also promotes the development of advanced cognitive skills and a deep understanding of the underlying concepts.

The fact that most work has been classified under the cognitive process dimension "remember" within Bloom's taxonomy of learning objectives suggests that many of the learning approaches in XR environments focus primarily on information retention.

In Bloom's taxonomy, the remember dimension represents the most basic level of cognitive processing, where learners are able to retrieve previously learned information without necessarily understanding or applying that knowledge in a meaningful way [7]. This may involve memorizing facts, definitions, or procedures without a deep understanding or connection to other concepts.

The fact that the second highest scoring category was "apply" suggests that there is a recognition of the importance of applying knowledge in practical contexts, but this category is still below the "remember" level. This may indicate that, although there are efforts to apply knowledge, the main focus is still on information retention.
<figure><img src="image_6.png" /><figcaption>Figure 5: NodeEdge Diagram of the relationship of articles with the modality of interaction and spatial affordance. Papers are colored red. Modalities and spatial affordance appear in yellow.</figcaption></figure>

This distribution of categories may suggest that XR researchers are initially concerned with ensuring that users have a solid foundation of knowledge before moving on to higher levels of cognitive processing such as analysis, evaluation, and creation.

We noticed that the combination of visual and haptic modalities with affordances located on dashboards and physical devices (controls) associated with selection and manipulation tasks was what predominated in articles classified as "metacognitive" in the knowledge dimension of Bloom's taxonomy. This points to the possibility of experiments that explore these configurations of affordances in immersive environments, associated with learning objectives at higher levels of the cognitive process dimension, such as deconstructing, reflecting and creating.

## 5.3 Affordances Strategies

According to the data categorized in Figure 1, it was possible to identify five distinct strategies that reflect the diversity of approaches adopted in the integration of affordances into XR environments. These strategies emerged from the analysis of the classification of works in design space, seeking to find patterns in the use of affordances. They offer insights into designing interactions in XR environments to enhance user experience and task effectiveness. Next we present each strategy.

### 5.3.1 Multimodal tendency for system control.
This strategy provides users with greater flexibility by allowing them to employ a combination of gesture input, voice commands, touch controls, and receive feedback through audio, video, and haptic devices. This multimodal interaction enhances the user experience by catering to different preferences and needs, improving accessibility, and enabling more natural and intuitive interactions. For instance, in a virtual training environment for surgeons, professionals can manipulate virtual instruments using gestures or controls, access critical information via voice commands, and experience haptic feedback that simulates the sensation of touching virtual organs. Furthermore, they can receive auditory feedback on their progress during simulated procedures, which aids in real-time learning and situational awareness.

The integration of these various input and feedback methods can significantly enhance the realism and immersion of the training environment, making the learning process more effective and engaging. However, this approach also poses several challenges. Ensuring

seamless and synchronized interaction between different modalities requires sophisticated software and hardware integration. Additionally, designing intuitive interfaces that do not overwhelm the user with excessive information or feedback is crucial to maintain usability and prevent cognitive overload.

Moreover, the effectiveness of these technologies in improving training outcomes depends on their ability to accurately replicate real-world conditions and provide meaningful feedback. This necessitates continuous refinement and validation of the virtual environments and haptic devices. As these technologies evolve, ongoing research and development will be essential to address these challenges and fully realize the potential of multimodal interactions in enhancing training and other applications in immersive environments.

### 5.3.2 Touch and vision in navigation.
Using touch and vision as guides when navigating XR environments can significantly improve user interaction, providing multimodal feedback that facilitates complex tasks and reduces errors. For example, in a virtual tourism application, users may receive haptic vibrations through their controllers or wearable devices as they approach points of interest. This haptic feedback serves as an intuitive guide, allowing users to navigate the virtual space more naturally. Simultaneously, visual information such as pop-up descriptions or augmented reality overlays can provide contextual details about locations and landmarks. This dual-sensory approach tends to enrich the exploration experience.

Research has shown that multimodal feedback can lead to improved task performance and user satisfaction [46] [44]. They highlight the importance of perceptual cues to guide users' actions and increase the sense of presence in virtual environments. Thus, incorporating haptic and visual feedback mechanisms into XR applications aligns with these findings and can contribute to improving UX.

### 5.3.3 Sound and vision as guides for mounting.
This approach emphasizes the synergy between auditory guidance and visual perception to optimize efficiency in assembly tasks within XR environments. For example, in a training application for assembling electronic products, visual instructions are supplemented by sound guidance indicating the correct component locations and steps to follow. This multimodal guidance can help reduce errors and improve the assembly experience by catering to different learning
styles and ensuring users receive immediate and clear feedback on their actions.

5.3.4 *Relevance of touch and vision in the usage learning process.*

Touch and vision play vital roles in the learning process within XR systems. Haptic feedback enhances users' understanding of interactions, accelerating adaptation and reducing the learning curve. In a virtual reality 3D design application, haptic feedback indicates virtual resistance when modeling objects, while visual elements highlight critical areas or parts requiring attention, facilitating a more comprehensive understanding of actions.

5.3.5 *Prioritization of hearing and vision in general learning.* Prioritizing hearing and vision in learning enables users to focus on essential content, enhancing the learning experience, particularly in virtual educational environments. For instance, in a virtual reality app for foreign language learning, prioritizing hearing and vision allows users to concentrate on correct pronunciation and reading in specific contexts, thereby improving language assimilation.

# 6 OPPORTUNITIES AND CHALLENGES

We identify five fundamental insights that emerged from this SLR about affordances in XR environments. They offer important insights into how to advance XR research and development to maximize the potential of affordances in creating more effective discovery experiences.

The first insight leads us to consider the possibility of reclassifying articles on a more detailed scale across multiple dimensions, such as interaction modality. By doing so, we could identify specific nuances in the implementation of affordances in XR. This would allow for a more accurate understanding of how different approaches impact discovery and user experience, providing valuable insights for future research and development.

A second insight highlights the importance of the relationship between users' cognitive capabilities and the properties of XR objects. We recognize that the more aligned these elements are, the greater the potential for effective use of affordances in XR environments. This emphasizes the need to adopt a user-centered design approach, where the characteristics of XR environments are carefully tuned to meet the needs and capabilities of users, influencing the UX.

As a third insight, this work can be seen as the first step to creating a taxonomy of affordances in XR. The analysis of the articles revealed several facets of affordances in immersive environments, which can be grouped and organized into a taxonomic structure. This taxonomy could serve as a valuable guide for researchers, designers, and developers seeking to understand, create, and evaluate affordances in XR. Furthermore, the taxonomy can contribute to the consolidation of a common vocabulary and set of guidelines in the field, promoting the development of XR environments where tasks can be completed more easily.

Our SLR revealed strategic approaches for testing and implementing affordances in XR environments. The fourth insight involves the fact that these strategies present unique opportunities for refining XR experiences, encouraging researchers and developers to explore diverse ways in which users can interact with immersive environments. Leveraging multiple senses and modalities for enhanced system control, navigation, and learning processes, these strategies

offer a promising avenue for experimentation in the design and evaluation of XR applications to optimize user engagement and task performance.

Finally, the prospect of developing a framework to create affordances in XR emerges. This framework could serve as a practical guide for designers, providing a structured approach to implementing and evaluating affordances in immersive environments. By synthesizing the identified strategies and incorporating them into a unified framework, researchers and developers can streamline the design process and enhance the effectiveness of XR applications. Such a framework would contribute to the maturation of XR technology by establishing best practices, fostering innovation, and facilitating the creation of more intuitive and user-friendly experiences. This step forward holds the potential to provide a solid foundation for the continued evolution of affordances in immersive environments.

# 7 CONCLUSION

The objective of this research was to investigate, through a SLR, how researchers are defining and using affordances in immersive environments. We verified that there is no established consensus in the area regarding the concept of affordance. Through a thematic analysis of the concepts found, we suggest understanding affordance in XR as a technological resource perceived by one of the human senses that explicitly aims to favor the completion of a task in the immersive environment.

Next, we present a classification of SLR works into eight dimensions that favor the understanding of how affordances are being used in XR: type of environment, modality of interaction, location of the affordance, temporal aspect, learning aspects, task categories, domain tasks and types of affordances. This classification helped to understand strategies that can be useful to make environments easier to use, improve UX and increase interactivity.

We present five strategies that can be used to enhance UX in XR environments. These strategies address crucial aspects such as system control, efficient navigation, mounting guidance, efficient system discovery, and effective learning. The configuration of each strategy stands as a comprehensive guide for the design and implementation of affordances that can make XR environments more engaging and intuitive.

As a limitation, some dimensions were configured for high-level classification of articles. They could be configured in another way, covering low-level categories, for example, to find more accurate results about the best settings for a given learning objective or task category.

Furthermore, we consider usability and communicability, for example, as part of discovery, learning and UX in the search string. However, we recognize that this is a limitation of this work.

This research opens the door to the development of a taxonomy of affordances in XR, which can help consolidate the concept in the area and eliminate ambiguities, in addition to establishing new dimensions for classifying affordances based on more in-depth studies.
# 8 AI TOOLS

The final version of the paper benefited from the use of Chat-GPT 3.5 for the improvement of spelling, grammar, vocabulary and style. AI suggestions were carefully examined and often corrected by us, where we take full responsibility for the form and content of the paper.

# REFERENCES

[1] [n. d.]. *Afford. Merriam-Webster*. Accessed October, 2023. https://www.merriam-webster.com/dictionary/afford

[2] Lorin W Anderson and David R Krathwohl. 2001. *A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives: complete edition*. Addison Wesley Longman, Inc.

[3] Rodrigo Assaf, Daniel Mendes, and Rui Rodrigues. 2024. Cues to fast-forward collaboration: A Survey of Workspace Awareness and Visual Cues in XR Collaborative Systems. In *Computer Graphics Forum*. Wiley Online Library, e15060.

[4] Sarune Baceviciute, Thomas Terkildsen, and Guido Makransky. 2021. Remediating learning from non-immersive to immersive media: Using EEG to investigate the effects of environmental embeddedness on reading in Virtual Reality. *Computers & Education* 164 (2021), 104122. https://doi.org/10.1016/j.compedu.2020.104122

[5] George P Banky and Aaron S Blicblau. 2017. Doing it with a glass onion: Investigating affordances provided by a head-mounted augmented reality immersive device for the real-time online supervision of experimental learning. In *2017 IEEE 3rd International Conference on Engineering Technologies and Social Sciences (ICETSS)*. IEEE, 1–6. https://doi.org/10.1109/ICETSS.2017.8324168

[6] Manshul Belani, Harsh Vardhan Singh, Aman Parnami, and Pushpendra Singh. 2023. Investigating Spatial Representation of Learning Content in Virtual Reality Learning Environments. In *2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR)*. IEEE, 33–43. https://doi.org/10.1109/VR55154.2023.00019

[7] Benjamin S Bloom, Max D Engelhart, Edward J Furst, Walker H Hill, and David R Krathwohl. 1956. *Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain*. McKay New York.

[8] Vanessa Camilleri, Matthew Montebello, Alexiei Dingli, and Vince Briffa. 2017. Walking in small shoes: Investigating the power of VR on empathising with children's difficulties. In *2017 23rd International Conference on Virtual System & Multimedia (VSMM)*. IEEE, 1–6. https://doi.org/10.1109/VSMM.2017.8346253

[9] Joe Cecil, Sam Kauffman, Avinash Gupta, Vern McKinney, and MD Miguel Pirela-Cruz. 2021. Design of a human centered computing (HCC) based virtual reality simulator to train first responders involved in the Covid-19 pandemic. In *2021 IEEE International Systems Conference (SysCon)*. IEEE, 1–7. https://doi.org/10.1109/SysCon48628.2021.9447090

[10] Joe Cecil, Mary Sweet-Darter, Aaron Cecil-Xavier, and Avinash Gupta. 2021. Role of Affordance, Visual Density and Other HCC Criteria in Designing Virtual Learning Environments to Support STEM Learning for Autistic Students. In *2021 IEEE Frontiers in Education Conference (FIE)*. IEEE, 1–7. https://doi.org/10.1109/FIE49875.2021.9637130

[11] Michelene TH Chi. 1997. Quantifying qualitative analyses of verbal data: A practical guide. *The journal of the learning sciences* 6, 3 (1997), 271–315. https://doi.org/10.1207/s15327809jls0603_1

[12] Andreas Dengel. 2020. How important is immersion for learning in computer science replugged games?. In *Proceedings of the 51st ACM Technical Symposium on Computer Science Education* 1165–1171. https://doi.org/10.1145/3328778.3366837

[13] Paloma Diaz, Telmo Zarraonandia, Mónica Sánchez-Francisco, Ignacio Aedo, and Teresa Onorati. 2019. Do low cost virtual reality devices support learning acquisition? a comparative study of two different vr devices. In *Proceedings of the xx international conference on human computer interaction*. 1–8. https://doi.org/10.1145/3335595.3335629

[14] Connor Daniel Flick, Courtney J Harris, Nikolas T Yonkers, Nahal Norouzi, Austin Erickson, Zubin Choudhary, Matt Gottsacker, Gerd Bruder, and Greg Welch. 2021. Trade-offs in augmented reality user interfaces for controlling a smart environment. In *Proceedings of the 2021 ACM Symposium on Spatial User Interaction*. 1–11. https://doi.org/10.1145/3485279.3485288

[15] Francois R Foerster and Jeremy Goslin. 2021. Tool use and function knowledge shape visual object processing. *Biological Psychology* 164 (2021), 108143. https://doi.org/10.1016/j.biopsycho.2021.108143

[16] William W Gaver. 1991. Technology affordances. In *Proceedings of the SIGCHI conference on Human factors in computing systems*. 79–84.

[17] James Jerome Gibson. 1979. The Ecological Approach to Visual Perception. https://api semanticscholar org/ CorpusID:33656271

[18] Katelyn M Grasse, Marjorie Ann M Cuervo, and Edward F Melcer. 2021. Mad Mixologist: exploring how object placement in tangible play spaces affects collaborative interaction strategies. In *2021 IEEE Conference on Games (CoG)*. IEEE, 1–8. https://doi.org/10.1109/CoG52621.2021.9619086

[19] Peidi Gu, Xinhao Xu, Xueqin Qian, and Tsung-Han Weng. 2022. Leveraging Extended Reality for Autistic Individuals: A Scoping Review of Technical Features and Technology Affordances. *IEEE Transactions on Learning Technologies* (2022). https://doi.org/10.1109/TLT.2022.3197430

[20] Jie Guan, Jay Irizawa, and Alexis Morris. 2022. Extended reality and internet of things for hyper-connected metaverse environments. In *2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)*. IEEE, 163–168. https://doi.org/10.1109/VRW55335.2022.00043

[21] Negin Hamzeheinejad, Daniel Roth, Samantha Monty, Julian Breuer, Anuschka Rodenberg, and Marc Erich Latoschik. 2021. The impact of implicit and explicit feedback on performance and experience during VR-supported motor rehabilitation. In *2021 IEEE Virtual Reality and 3D User Interfaces (VR)*. IEEE, 382–391. https://doi.org/10.1109/VR50410.2021.00061

[22] Julia Hertel, Sukran Karaosmanoglu, Susanne Schmidt, Julia Bräker, Martin Semmann, and Frank Steinicke. 2021. A taxonomy of interaction techniques for immersive augmented reality based on an iterative literature review. In *2021 IEEE international symposium on mixed and augmented reality (ISMAR)*. IEEE, 431–440. https://doi.org/10.1109/ISMAR52148.2021.00060

[23] Staffs Keele et al. 2007. Guidelines for performing systematic literature reviews in software engineering.

[24] Barbara Kitchenham. 2004. Procedures for performing systematic reviews. *Keele, UK, Keele University* 33, 2004 (2004), 1–26.

[25] Joseph J LaViola Jr, Ernst Kruijff, Ryan P McMahan, Doug Bowman, and Ivan P Poupyrev. 2017. 3D user interfaces: theory and practice. Addison-Wesley Professional.

[26] Tsung-Chi Lin, Achyuthan Unni Krishnan, and Zhi Li. 2022. Comparison of haptic and augmented reality visual cues for assisting tele-manipulation. In *2022 International Conference on Robotics and Automation (ICRA)*. IEEE, 9309–9316. https://doi.org/10.1109/ICRA46639.2022.9811669

[27] Vitor Neves Lopes, Isabel Dantas, José Pedro Barbosa, and José Barbosa. 2022. Telestration in the teaching of basic surgical skills: a randomized trial. *Journal of Surgical Education* 79, 4 (2022), 1031–1042. https://doi.org/10.1016/j.jsgs.2022.02.013

[28] Heng Luo, Xu Han, Qinna Feng, and Gege Li. 2022. Integrating Inquiry-Based Pedagogy with Mixed Reality: Theories and Practices. In *2022 4th International Conference on Computer Science and Technologies in Education (CSTE)*. IEEE, 82–86. https://doi.org/10.1109/CSTE55932.2022.00021

[29] Aline Menin, Ricardo Cava, Carla Maria Dal Sasso Freitas, Olivier Corby, and Marco Winckler. 2021. Towards a visual approach for representing analytical provenance in exploration processes. In *2021 25th International Conference Information Visualisation (IV)*. IEEE, 21–28.

[30] Mohsen Moghaddam, Nicholas C Wilson, Alicia Sasser Modestino, Kemi Jona, and Stacy C Marsella. 2021. Exploring augmented reality for worker assistance versus training. *Advanced Engineering Informatics* 50 (2021), 101410. https://doi.org/10.1016/j.aei.2021.101410

[31] Simone Morgagni. 2012. Affordances as possible actions: elements for a semiotic approach. (2012).

[32] Peyman Najafi, Masi Mohammadi, Pieter van Wesmael, and Pascale M Le Blanc. 2023. A user-centred virtual city information model for inclusive community design: State-of-art. *Cities* 134 (2023), 104203. https://doi.org/10.1016/j.cities.2023.104203

[33] Don Norman. 2013. *The design of everyday things: Revised and expanded edition*. Basic books.

[34] Donald A Norman. 1999. Affordance, conventions, and design. *interactions* 6, 3 (1999), 38–43.

[35] Omobolanle R Ogunseiju, Nihar Gonsalves, Abiola A Akanmu, Diana Bairaktarova, Doug A Bowman, and Farrokh Jazizadeh. 2022. Mixed reality environment for learning sensing technology applications in Construction: A usability study. *Advanced Engineering Informatics* 53 (2022), 101637. https://doi.org/10.1016/j.aei.2022.101637

[36] Seungjae Oh, Hyo-Jeong So, and Matthew Gaydos. 2018. Hybrid augmented reality for participatory learning: The hidden efficacy of multi-user game-based simulation. *IEEE Transactions on Learning Technologies* 11, 1 (2018), 115–127. https://doi.org/10.1109/TLT.2017.2750673

[37] Anthony J Onwuegbuzie, Nancy L Leech, and Kathleen MT Collins. 2012. Qualitative analysis techniques for the review of the literature. *Qualitative Report* 17 (2012), 56.

[38] Amos Sunday Oyelere, Friday Joseph Agbo, and Solomon Sunday Oyelere. 2023. Formative evaluation of immersive virtual reality expedition mini-games to facilitate computational thinking. *Computers & Education: X Reality* 2 (2023), 100016. https://doi.org/10.1016/j.cexr.2023.100016

[39] Jocelyn Parong, Kimberly A Pollard, Benjamin T Files, Ashley H Oiknine, Anne M Sinai, Jason D Moss, Antony Passaro, and Peter Khooshabeh. 2020. The mediating role of presence differs across types of spatial learning in immersive technologies. *Computers in human behavior* 107 (2020), 106290. https://doi.org/10.1016/j.chb.2020.106290
[40] Shalaunda M Reeves, Kent J Crippen, and Erica D McCray. 2021. The varied experience of undergraduate students learning chemistry in virtual reality laboratories. *Computers & Education* 175 (2021), 104320. https://doi.org/10.1016/j.compedu.2021.104320

[41] Yvonne Rogers. 2022. *HCI theory: classical, modern, and contemporary*. Springer Nature.

[42] Helen Sharp, Jenny Preece, and Yvonne Rogers. 2019. Interaction design: beyond human-computer interaction.

[43] Donghee Shin. 2022. The actualization of meta affordances: Conceptualizing affordance actualization in the metaverse games. *Computers in human behavior* 133 (2022), 107292. https://doi.org/10.1016/j.chb.2022.107292

[44] Jae-Eun Shin, Boram Yoon, Dooyoung Kim, Hyung-Il Kim, and Woontack Woo. 2022. The Effects of Device and Spatial Layout on Social Presence During a Dynamic Remote Collaboration Task in Mixed Reality. In *2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)*. IEEE, 394–403. https://doi.org/10.1109/ISMAR55827.2022.00055

[45] Kay M Stanney, Hannah Nye, Sam Haddad, Kelly S Hale, Christina K Padron, and Joseph V Cohn. 2021. Extended reality (XR) environments. *Handbook of human factors and ergonomics* (2021), 782–815. https://doi.org/10.1002/9781119636113.ch30

[46] Jacob H Steffen, James E Gaskin, Thomas O Meservy, Jeffrey L Jenkins, and Iopa Wolman. 2019. Framework of affordances for virtual reality and augmented reality. *Journal of Management Information Systems* 36, 3 (2019), 683–729.

[47] Anselm Strauss and Juliet M Corbin. 1997. *Grounded theory in practice*. Sage.

[48] Anchen Sun, Yudong Tao, Mei-Ling Shyu, Shu-Ching Chen, Angela Blizzard, William Andrew Rothenberg, Dainelys Garcia, and Jason F Jent. 2021. Multimodal data integration for interactive and realistic avatar simulation in augmented reality. In *2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI)*. IEEE, 362–369. https://doi.org/10.1109/IRI51335.2021.00056

[49] Yang Tian, Chi-Wing Fu, Shengdong Zhao, Ruihui Li, Xiao Tang, Xiaowei Hu, and Pheng-Ann Heng. 2019. Enhancing augmented vr interaction via egocentric scene analysis. *Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies* 3, 3 (2019), 1–24. https://doi.org/10.1145/3351263

[50] Mythreye Venkatesan, Harini Mohan, Justin Ryan, Christian Schürch, Garry P Nolan, David H Frakes, and Ahmet F Coskun. 2021. Virtual and augmented reality for biomedical applications. *Cell reports medicine* 2, 7 (2021). https://doi.org/10.1016/j.ccrm.2021.100348

[51] William H Warren. 1984. Perceiving affordances: visual guidance of stair climbing. *Journal of experimental psychology: Human perception and performance* 10, 5 (1984), 683.

[52] Michael Williams and Tami Moser. 2019. The art of coding and thematic exploration in qualitative research. *International Management Review* 15, 1 (2019), 45–55.

[53] Ezequiel Roberto Zorzal, José Miguel Campos Gomes, Maurício Sousa, Pedro Belchior, Pedro Garcia da Silva, Nuno Figueiredo, Daniel Simões Lopes, and Joaquim Jorge. 2020. Laparoscopy with augmented reality adaptations. *Journal of biomedical informatics* 107 (2020), 103463. https://doi.org/10.1016/j.jbi.2020.103463