<figure><img src="image_1.png" /></figure>

HAL
open science

# Design and Evaluation of an Augmented Reality App for Learning Geometric Shapes in 3D

Pornpon Thamrongrat, Effie Lai-Chong Law

► To cite this version:

Pornpon Thamrongrat, Effie Lai-Chong Law. Design and Evaluation of an Augmented Reality App for Learning Geometric Shapes in 3D. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.364-385, 10.1007/978-3-030-29390-1_20. hal-02877679

HAL Id: hal-02877679

https://inria.hal.science/hal-02877679

Submitted on 22 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Design and Evaluation of an Augmented Reality App for Learning Geometric Shapes in 3D

Pornpon Thamrongrat, Effie Lai-Chong Law

Department of Informatics, University of Leicester, LE1 7RH Leicester, U.K.
{pt171|lcl9}@leicester.ac.uk

**Abstract.** Augmented Reality (AR) is increasingly used as an educational tool in a range of domains with the goal of enhancing students' performance as well as their learning experience, thanks to the interactivity and visual appeal of AR objects. While some attempts, albeit limited, have been undertaken to prove these beneficial effects in learning 3D geometry, the results remain inconclusive and there are some methodological issues such as under-evaluation of user experience. With the aim to enrich the applied body of knowledge on this specific topic, we developed an AR application that allows its users to learn about cross-sectional shapes and variables in 3D geometry. We compared the AR-based approach with the traditional chalk-and-board approach by involving sixty 12-16 year-olds from two schools. The AR class showed a significantly stronger learning effect than the traditional class, especially for the more complex geometric concepts. The AR class found the application engaging, regardless of their level of knowledge gain, which bore no significant relation with their intention to use it. The methodological challenge for implementing control groups and the practical challenge for affordable emerging educational tools should be tackled in future research.

**Keywords:** Augmented Reality, Geometry, Education, UX, User Engagement

## 1 Introduction

The rapid growth of computing technology enhances the use of software to support human activities in various fields such as education, health and entertainment. The rise of multimodal and multisensory interactions enabled by mobile devices, for instance accelerometer-based [1], camera-based [2], location-based [3], voice-based [4] and touch-based [5], leads to the development of a variety of specific applications.

In particular, Augmented Reality (AR) has attracted more and more research attention in different areas. AR is a technology used to create an enhanced real-time interactive experience by combining computer-generated 3D and real-world contents. Apart from visual images, some AR systems provide enhanced audio or haptic experiences [6]. Such enhanced experiences render AR a powerful tool in engaging customers in business [7], visitors in cultural heritage [8], players in games [9], and learners in education [10, 11], to name just a few examples of AR-enhanced human
activities. The research study presented in this paper focuses on the application of AR in education.

In the domain of computer science, Yilmaz [12] studied teachers' and children's opinions on the educational effect of AR-based magic toys and found that the 3D virtual objects in the magic toys could make children more interested in learning. Furthermore, Fotouhi-Ghazvini and colleagues [13] proposed a mobile AR for adding a real sense of learning to a game for students, which could enhance their learning motivation. According to the study of Kysela and Štorková [14] on tourism education, using AR to provide tourists with experiences at cultural heritage sites by overlaying 3D virtual objects onto the real scene could serve as an attractive environment for studying history. These examples illustrate that the attractiveness and interactivity of AR can be used to support people to learn with enjoyment and effectiveness.

Geometry is an area of mathematics that involves studying a broad range of shapes such as pyramid, rectangular, cuboid and sphere. They are difficult to learn in 2D format, especially when they can be cut at any angle to produce various cross-sectional shapes. Indeed, the study of Adolphus [15] showed the difficulty in teaching and learning geometry, as the conventional instructional environment is not conducive to learning geometric concepts in 3D. Geometry is also relevant to learning chemistry with abstract concepts as well as concrete examples such as molecular structures, which are composed of three-dimensional spaces of atoms that are not easily understood in a classroom or a laboratory. To explain such shapes, people use general presentation software (e.g., PowerPoint) where, however, learners can only view passively the learning objects presented because they cannot interact with them [16]. In contrast, AR allows dynamic and interactive manipulations of 3D images [17] and thus can be useful for teaching geometry. Students can move, rotate, reflect, or stretch the images. Nonetheless, only a small number of research studies (e.g. [18, 19, 20]) have been conducted to investigate the effectiveness of learning geometric 3D shapes with AR, and they reached inconsistent conclusions. Furthermore, some methodological limitations were identified in the existing work on this specific topic. First, the lack of control group to assess the effect of learning geometric shapes with the conventional teaching approach as compared with an AR-based approach. Second, the evaluation of the AR application focused on the learning effect, but overlooked the user experience aspect.

Consequently, we have been motivated to collect more empirical data to address the main research question: *To what extent can AR-enabled 3D virtual objects make learning more effective and more engaging for students as compared with the traditional 2D approach?* The answer to this question is important for informing the design of this new learning technology, which should be usable, useful and desirable. Accordingly, we developed a dedicated AR-based application that can be beneficial to children of 12-16 years old to learn geometric shapes in 3D and assess its learning and experiential effects with this target group. Overall, the main contribution of this work is to enrich the relatively small applied body of knowledge on this specific topic from the methodological and practical aspects.

The rest of the paper is structured as follows: We first present the literature review in Section 2, followed by the outline of our methodological framework in Section 3.
Then we report our empirical study and results in Section 4 and 5, respectively. In Section 6, we discuss three issues that we deliberated when planning and implementing our study. In Section 7, we conclude and draw implications for future research.

# 2 Literature Review

In this section we first provide an overview of AR in education, followed by a review on the existing technological approaches to learning geometric concepts. Then we outline the related work on user experience (UX) and user engagement (UE), especially the UE scale that we adopted for our empirical studies (Section 3).

## 2.1 AR in Education

One of the most essential functions of AR is the dual-presence of information where two information displays are overlaid on each other [21]. Wu and colleagues [22] reported that one of the common AR applications is education. Mayer [23] argued, based on the cognitive theory of multimedia learning, that AR comprising texts, 2D or 3D images could be used to improve learning because people tend to learn better from words and pictures combined together than from words alone. Sommerauer and Müller [24] examined the use of AR for education through conducting an experiment at a mathematics exhibition. Their experimental study was similar to that of Morrison and colleagues [21] in that both studies made a comparison between the use of AR and non-AR. While the results of Morrison and colleagues [21] were not conclusive, the results of Sommerauer and Müller [24] showed that museum visitors learned significantly better from the AR exhibition than from the non-AR one.

AR has also been applied to education in various domains such as cultural heritage [25] for which, however, no formal evaluation of the learning or experiential effect was performed. The study of Ibáñez and colleagues [26] showed that using AR was more effective in promoting students' knowledge of electromagnetic phenomena with a higher level of enjoyment as compared with using web-based applications. For learning astronomical phenomena, using AR with three markers - sun, earth and moon - could enhance the learners' perceptions of mobilization [27]. By using a new interaction method with a pico-projector as an AR system to enhance the digital display on static paper, teachers, students, and experts commented that AR was useful for learning [28]. Yang, Chu, and Yang [29] demonstrated the use of an inquiry-based AR learning system to learn about the properties of gas in a real-world context with 70 students of eleventh grade. It helped students to improve their learning performance on gas properties in terms of adjusting the parameters of pressure and temperature. In another research study [30], twenty computer science students took part in an experiment aimed to compare two educational approaches: an AR-based application and a non-AR approach to programming debugging. The results found that the AR application was more effective in increasing the students' computational thinking and learning outcomes when compared with the non-AR approach.

Overall, as indicated in the above literature review, in the last decade the use of AR for educational purposes has been expanded to a large range of domains, especially
STEM. However, the research work is scattered over the broad range and the number of empirical studies per domain, say mathematics, is small. This makes it difficult to draw any conclusive claims on the learning effect of AR. Hence, it is necessary to conduct more systematic research to substantiate the applied body of knowledge on applying AR as an educational tool in general and as an enabler to understand geometric concepts in particular.

## 2.2 Learning Geometric Concepts

Study of geometry as a branch of mathematics has a long tradition and is highly relevant to other science subjects as well as to art. Traditionally, basic geometric shapes are drawn with pencil, paper, ruler and compass. In recent decades, computing technologies have played an ever-increasingly important role in education in general and in study of geometry in particular. Using 3D computer-aided design (CAD), such as AutoCAD™, Maya™, 3DStudio Max™ and others, has become commonplace in teaching and learning geometric concepts in 3D. While such software applications can help create beautiful 3D shapes, they should also support the process of learning geometry (e.g., 3D shapes and geometric variables). Hence, their user interfaces should be easy to understand.

With computers having the built-in ability to record user activities, the related software was able to track how users created geometric shapes, to reproduce the process of construction with changes in parameters (e.g. size, coordinate, color) and to present the resulting shape instantly. Today, there are different proprietary software applications for dynamic two-dimensional geometry, and the most popular ones are Cinderella [31] and Euklid [32].

AR is the current technological advancement with strong potential for enhancing education in a broad range of domains [22], including geometry. In fact, AR is more than the combination of hardware, software and contextually relevant information [33]. Utilizing the power of the AR technology, especially its interactivity and visual appeal, can support people to learn 3D geometry with high effectiveness and efficiency, and positive learning experience.

## 2.3 User Experience and User Engagement

User experience (UX) and user engagement (UE) are very closely related concepts for designing interactive systems. UX puts emphasis on the experiential aspect of human-computer interaction, encompassing a broad range of emotional responses with engagement being one of the key qualities.

Don Norman [34] was one of the authors to use the word "User Experience" to describe a person's total experience with every aspect of a system. He argued that 'usability' was too narrow to show a holistic vision of human-computer interaction. UX as a study explores factors influencing the subjective experience humans have when interacting with technology. Moreover, Gegner and colleges [35] proposed that UX is a 'counter-movement to usability thinking' that focuses on pragmatic quality (cf. hedonic quality). While different definitions of UX exist (e.g. [36, 37]), researchers
tended to agree that UX involves users' emotional, cognitive and behavioral responses before, during and after interacting with a system, product or service [38].

Like UX, the notion of user engagement (UE) has different definitions. Grounded in relevant theoretical frameworks, O'Brien [39] defined user engagement as "a quality of UX characterized by the depth of an actor's cognitive, temporal, affective and behavioural investment when interacting with a digital system". Apart from the definitional challenge, O'Brien and colleagues [39] tackled the issue of operationalising UE by developing a standardized UE scale based on their systematic literature review and on their own empirical work in a range of domains, including e-learning, e-commerce and gaming. Specifically, they adopted the attribute-based approach [39]. Deriving from Jacques's UE work on educational multimedia [40] and Webster and Ho's on presentaion software [41], O'Brien and colleagues [39] identified a set of user-system attributes (e.g., attention, motivation, perception of control, curiosity, intrinsic interest) that constitute an engaging experience. The original UES consists of 31 items for six dimensions of engagement, and has been used in a variety of domains such as education, online news, and social network systems. Subsequently, O'Brien and colleges [38] validated a short form (UES-SF) where the number of items was reduced to 12 representing four dimensions of engagement: *focused attention (FA)*, *perceived usability (PU)*, *aesthetic appeal (AE)*, and *reward (RW)*, which are defined as follows:

* *FA: Focused attention*, feeling absorbed in the interaction and losing track of time.

* *PU: Perceived usability*, negative affect experienced as a result of the interaction and effort expended.

* *AE: Aesthetic appeal*, the attractiveness and visual appeal of the interface.

* *RW: Rewards*, the worthwhile and feeling interested in the experience.

The UES-SF was proved to have strong psychometric properties [39]; this characteristic together with its relevance to our research topic can well justify our choice of this instrument for our empirical study.

# 3 Methodology

In this section, we first present the methodological framework for the empirical studies of which details are reported in Section 4. Next, we describe the development of the AR application as the intervention for enhancing children's learning of 3D geometry.

## 3.1 Methodological Framework

As described in Introduction, this research aims to analyse the effectiveness of an AR application for supporting high school students to learn geometric shapes in 3D. The methodological framework is illustrated in Fig. 1. Accordingly, the empirical studies were conducted with students of 12-16 years old. The intervention was developed with the User-centered Design (UCD) approach [42], involving students and teachers, who were key stakeholders of the intervention, throughout the development process in order to understand their needs and preferences.

The phases of the UCD approach were implemented as follows (cf. Section 3.2):
1) *Expert interview:* We conducted interviews with teachers to get input how they typically teach 3D geometry and what the learning objectives and outcomes are.

2) *Specify the context of use:* To understand the context where the AR application would be used, we carried out in situ observations in classrooms to understand how children typically learn 3D geometry and also conducted interviews with some children to identify their preferences with regard to the alternative approach to learning geometry.

3) *Specify requirements:* Based on (1) and (2), a set of requirements was derived to inform the development of the AR application.

4) *Produce design solutions and evaluate them:* Early prototypes were designed and evaluated with the teachers and children. Feedback was obtained to improve the design.

For the experimental design, we employed the between-subject method. The two key independent variables (IVs) were *learning outcome* and *user experience*, which were evaluated with a domain-specific knowledge test (i.e. the pre-/post-test) on 3D geometry and the standardized UES-SF (Section 2.3), respectively. School children were recruited through their teachers whom the first author acquainted through his network. They were randomly allocated to two groups under different conditions. The impact of the learning approach typically used in classroom (Normal Class) was compared with that of the AR-based approach (AR Class). Post-intervention interviews with children were also conducted. This mixed-method approach is recommendable for triangulating the empirical data collected to draw valid conclusions [43].

## 3.2 Augmented Reality Application Design

With the user requirements gathered with the UCD approach (Section 3.1) and the results of the analysis of the learning content on 3D geometry, the AR application was developed. The tool Unity 3D [44] based on C# programming was used. 3D models are built by using Autodesk Maya and they can be displayed on a camera by using the Vuforia AR plug-in. In general, users interact with physical markers by putting them in front of a camera. The images are captured and processed by the AR system. After that, virtual 3D objects are overlaid onto the corresponding markers to be displayed on a computer screen. Users can view the resulting 3D objects and respond to feedback. In accord with the UCD approach, a prototype design needs input from relevant stakeholders. Three teachers in secondary schools who have been teaching mathematics for about 5 years were interviewed on how what they typically teach 3D geometry in classroom. In addition, in situ observations of classroom teaching were made. It was found that the teachers rely on the use of chalk and board to draw 3D shapes when explaining the concepts to their students. Furthermore, ten students aged 12-16 years old were interviewed to share their opinions about learning 3D geometry. They commented that 3D shapes, particularly cross-sectional shapes, were difficult to understand in 2D form such as hand-drawn geometric shapes on cardboard. They were enthusiastic about an interactive application that can show them the shapes clearly. This student feedback resonated with what was shared by the teachers interviewed, who stated that the seven important geometric shapes - cone, cube, cuboid, cylinder, prism, pyramid and sphere - and their cross-sectional shapes were difficult for students to
<figure><img src="image_3.png" /><figcaption>Fig. 1. The research methodology framework.</figcaption></figure>

understand when they were taught with cardboard or paper. The teachers proposed the need of a tool that can display 3D models clearly and allow such 3D models to be zoomed, scrolled, scaled, moved or rotated, thereby enabling students to view the models from different perspectives. The AR application prototype was designed accordingly and was later checked with the teachers to invite further feedback. The AR application prototype was developed with a set of AR markers consisting of seven shapes with five angles for learning cross-sectional shapes and geometry variables (Fig. 2 and Fig. 3).

<table><caption>Fig. 2. AR Marker of seven geometric shapes.</caption><thead><tr><th></th><th>Cone</th><th>Cube</th><th>Cuboid</th><th>Cylinder</th><th>Prism</th><th>Pyramid</th><th>Sphere</th></tr></thead><tbody><tr><th>AR Markers for cross-sectional shapes learning</th><td>45°</td><td>45°</td><td>45°</td><td>45°</td><td>45°</td><td>45°</td><td>45°</td></tr><tr><th>AR Markers for geometry variables learning</th><td>45°</td><td>45°</td><td>75°</td><td>45°</td><td>45°</td><td>45°</td><td>45°</td></tr></tbody></table>
<table><caption>Fig. 3. AR Markers of Cone shape in five different angles.</caption><thead><tr><th>45°</th><th>75°</th><th>90°</th><th>135°</th><th>180°</th></tr></thead><tbody><tr><td>▲<br>45°</td><td>▲<br>75°</td><td>▲<br>90°</td><td>▲<br>135°</td><td>▲<br>180°</td></tr></tbody></table>

Children can move and push the markers in front of the camera of a tablet, then the corresponding 3D geometric shapes are presented to enhance the learning process by enabling children to interact with the 3D digital objects displayed. Later on, children were given quizzes to assess their knowledge gain.

# 4 Empirical Study

## 4.1 Instruments

**AR application and Tablet.** The AR application (Section 3.2) is installed in a tablet with the following technical specifications: Screen size: 8 inches; Operating system: Android 7.X Nougat; Processor: Octa Core; Resolution: 1920 x 1200; Memory: 128 GB. This is a quality device with a relatively high price (£250/$330). To ensure that the learning process and user experience would not be compromised by a less powerful tablet, we opted for this choice. However, the drawback was the small number of tablets we could afford to purchase with a limited budget for the study. As the AR Class would be taken on an individual basis, we estimated that the impact of having a small group of five instead of thirty children working on their own tablet at the same time could be marginal, especially if the AR application was meant to engage their attention in the tasks at hand. Nonetheless, we acknowledge that this may not be an ideal arrangement.

**Pretest, Posttest and Quizzes.** The domain-specific knowledge pretest/posttest and quizzes were constructed by the first author in collaboration with three math teachers, who were also interviewed for gathering requirements (Section 3.2). The pretest and posttest were actually the same test administered *before* and *after* the intervention, and the differences between the two test scores were used as a measure of the *learning effect*. The quizzes, which were given *during* the intervention, were used to measure the immediate learning gain and to sustain participants' attention and interest in the learning activities. The answer format is multiple-choice, and the test consists of four parts. The maximum scores for the whole pre-/post-test and quizzes were both 40.

<table><caption>**Table 1.** The structure of pre-/post-test and quizzes; Part A to D are of increasing complexity.</caption><thead><tr><th>Parts</th><th>Number of questions</th><th>Description: The questions are about ...</th></tr></thead><tbody><tr><td>Part A</td><td>13</td><td>What the cross-sectional shapes are after the cutting.</td></tr><tr><td>Part B</td><td>13</td><td>At which angle the shapes are cut.</td></tr><tr><td>Part C</td><td>7</td><td>Geometric variables.</td></tr><tr><td>Part D</td><td>7</td><td>Formulae with geometric variables.</td></tr></tbody></table>
Pre-/post-test and quizzes had the same number of questions and structure but the questions were different, though they were all related to the topics covered in the Normal/AR class (Table 1). There was no time limit to answer them. Paper was used for the pre-/post-test in both the Normal class and AR class. For quizzes, paper was used in the Normal class while a tablet was used in the AR Class.

**Questionnaires and Interviews.** To evaluate the user experience of the AR application (Section 2.3), two research methods - questionnaire and interview – were employed. After interacting with the AR application, participants were asked to complete the questionnaire comprising the 12-item UES-SF [39] and 2 items on ‘Intention to Use’ [45] derived from the Technology Acceptance Model (TAM). Accordingly, an individual's behavioral intention to use a system is largely determined by the perceived usefulness and perceived ease of use [45]. After that, semi-structured interviews (group discussion) were conducted to collect further information about user experience. It took on average about ten minutes to complete both the questionnaire and interview.

## 4.2 Procedure

**AR Class.** It took place in a regular classroom where a group of children followed the given workflow (Table 2). The Class was taken on an individual basis when each participant interacted with the AR application on the tablet with a set of AR markers on his/her own. Participants were instructed not to discuss the tasks with their peers during or after the AR Class.

<table><caption>**Table 2.** The AR Class workflow</caption><thead><tr><th></th><th>Tasks</th><th>Display screens</th><th>Descriptions</th></tr></thead><tbody><tr><td>1</td><td>Pretest</td><td>Paper-based</td><td>It has four parts of questions: Part A, Part B, Part C and Part D</td></tr><tr><td>2</td><td>Introduction</td><td>Cross section shape learning (1/3)<br> little 13...<br>my brain is more active with little 3<br>the brain is active because I learn more when I play with little 3<br>learn more about learning<br> shapes</td><td>The introduction is about how to use the AR application for learning cross-sectional shapes.</td></tr><tr><td>3</td><td>Cross-sectional shapes learning</td><td>Cross section shape learning (2/3)<br> little 13...<br>my brain is more active with little 3<br>the brain is active because I learn more when I play with little 3<br>learn more about learning<br> shapes</td><td>This section is for children to learn cross-sectional shapes. Children need to put a marker in front of the tablet, which displays the 3D shape with the name of its cross-sectional shape. Children can press the cut button to view the cut.</td></tr></tbody></table>
<table><thead><tr><th>Tasks</th><th>Display screens</th><th>Descriptions</th></tr></thead><tbody><tr><td>4 Quiz on Cross-sectional shapes</td><td></td><td>Quiz of Part A: The question is about what is the cross-sectional shape after the cutting? When a child clicks one of answer buttons, the application will show the correct one.<br> Example: The question about a rectangular shape and the answer is Square.</td></tr><tr><td></td><td></td><td>Quiz of Part B: The question is about at which angle is the shape cut? When a child clicks one of answer buttons, the application will show the correct one.<br> Example: The question is about a cross sectional shape of a rectangular shape and the answer is 45.</td></tr><tr><td>5 Introduction</td><td></td><td>The introduction is about how to use the AR application for learning geometry variables.</td></tr><tr><td>6 Geometry variables learning</td><td></td><td>This section is for children to learn cross sectional shapes. Children need to put the marker in front of the tablet, which then displays the 3D shape with its variables such as length, width and the name of its cross-sectional shape.</td></tr><tr><td>7 Quiz of Geometry variables</td><td></td><td>Quiz of Part C: The question is about geometry variables.<br> Example: Which letters represent Radius? When a child clicks one of answer buttons, the application will show the correct one: the 2D plane and its variables.</td></tr></tbody></table>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>
<table><thead><tr><th>Tasks</th><th>Display screens</th><th>Descriptions</th></tr></thead><tbody><tr><td></td><td></td><td>Quiz of Part D: The question is about calculation with the geometry variables.</td></tr><tr><td></td><td></td><td>Example: Find the area of a circle, where area is Pi * Radius * Radius? When a child clicks one of answer buttons, the application will show the correct one.</td></tr><tr><td>8 Posttest</td><td>Paper-based</td><td>The children will take the posttest which has the same questions as the pretest.</td></tr></tbody></table>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

**Normal class.** It was delivered by a math teacher who was asked to follow the steps corresponding to those of the AR application (cf. Table 2). The teacher used chalk and board to draw the shapes, cross-sectional shapes, and variables of 3D geometry (Step 2, 3 & 5 in Table 3) and administered the quizzes (Step 5 & 7 in Table 3) in the way similar to the corresponding steps in Table 2. Participants were instructed not to discuss the tasks with their peers during the class.

<table><caption>**Table 3.** The Normal class workflow</caption><thead><tr><th>Tasks</th><th>Descriptions</th></tr></thead><tbody><tr><td>1 Pretest</td><td>It has four parts on a piece of paper: Part A, Part B, Part C and Part D.</td></tr><tr><td>2 Introduction</td><td>The teacher introduces the topic of cross-sectional shapes in 3D geometry.</td></tr><tr><td>3 Cross-sectional shape learning</td><td>The teacher describes the seven shapes of 3D geometry and shows the 3D geometry cross-sectional shapes (45°, 75°, 90°, 135°, 180°) in 2D representations.</td></tr><tr><td>4 Quiz on Cross-sectional shapes</td><td>The children take a quiz on cross-sectional shapes.</td></tr><tr><td>5 Introduction</td><td>The teacher introduces the topic of 3D geometry variables.</td></tr><tr><td>6 Geometry variables learning</td><td>The teacher shows the 3D geometry variables such as width, length and diameter in 2D representations.</td></tr><tr><td>7 Quiz on geometry variables</td><td>The children take a quiz on 3D geometry variables.</td></tr><tr><td>8 Posttest</td><td>The children take the posttest.</td></tr></tbody></table>

## 4.3 Participants

The empirical study was approved by the Ethics Review Committee of the University of Leicester. Altogether 60 children aged 12 to 16 years were involved in the study, which took place in two secondary schools. All participations were voluntary and consented by the children and their parents/guardians and school authorities. The participants were divided into two groups: the Normal Class for the first school and the AR Class for the second school with 30 each. For the AR class, all participants (18
male, 12 female; average age: 14.07) were asked to carry out a set of tasks on an individual basis in a session that lasted about 30 minutes (Table 4). None of them had any AR experience before taking part in the study. For the Normal class (11 male, 19 female; average age: 14 years old), the teacher used the conventional approach (Section 4.2) and the class lasted about 45 minutes. The longer duration was attributed to the fact that the teacher needed time to draw the shapes by hand whereas the AR application retrieved and displayed the shapes instantly.

<table><caption>**Table 4.** The descriptive statistics the AR class.</caption><thead><tr><th rowspan="2">Gender</th><th colspan="3">AR class</th></tr><tr><th>Mean (Minutes)</th><th>SD</th><th>N</th></tr></thead><tbody><tr><td>Female</td><td>30.56</td><td>11.85</td><td>12</td></tr><tr><td>Male</td><td>29.72</td><td>7.74</td><td>18</td></tr></tbody></table>

## 4.4 Research Model and Hypotheses

The data collected in this study were to evaluate our research model (Fig. 4) and verify the six associated hypotheses (H). The key assumption underpinning the hypotheses is that the AR application can support children to learn 3D geometry more effectively and in a more engaging manner than the traditional chalk-and-board approach.

<figure><img src="image_14.png" /><figcaption>**Fig. 4.** The diagram of the research model with the hypotheses</figcaption></figure>

In addition, we hypothesize that the learner attributes, including *gender*, *prior knowledge*, and *age*, may influence the relations between the IV (i.e. learning approach) and two DVs (i.e. learning effect, user experience). Note that the participants' prior knowledge was assessed only by the pretest. However, as the learning effect was operationalized in terms of the differences between the posttest and pretest scores, we cannot evaluate the role of prior knowledge as a covariate. It could have been measured, for instance, by the participants' academic performance at school (e.g. math exam grades), but such data could not be made available to us. Furthermore, as the participating children were from the same year group, there was basically no difference
in age, which cannot be verified as a co-variate in this context. Another covariate could be 'previous experience in computing technology'. Apart from asking the participants of the AR class if any of them had used AR (none of them put up their hand), we did not collect any other data in this regard. Hence, we cannot verify this co-variate.

* **H1:** Participants in the Normal Class have the same 3D geometry background knowledge level as those in the AR Class as measured by the pretest scores.

* **H2:** Participants in the Normal Class have the same 3D geometry knowledge level as those in the AR Class during the learning process as measured by the quiz scores.

* **H3:** Participants in the AR Class gain a significantly higher level of 3D geometry knowledge than those in the Normal Class, as measured by the *difference scores* (i.e. the discrepancy between the posttest and pretest scores).

* **H4a:** There is a significant correlation between the total quiz scores and posttest scores of the Normal Class.

* **H4b:** There is a significant correlation between the total quiz scores and posttest scores of the AR Class.

* **H5:** There is a significant correlation between the learning effect as measured by the difference scores and (a) user engagement as measured by the self-reported questionnaire; and (b) intention to use of the AR Class.

* **H6a:** There is a significant difference in the learning effect between two genders in the AR Class.

* **H6b:** There is a significant difference in the UES scores between two genders in the AR Class.

# 5 Results

## 5.1 Learning Effect

The Shapiro-Wilk test was applied to individual datasets to assess if they are normally distributed, and the result thereof informed the selection of appropriate parametric or non-parametric statistical methods for verifying each of the hypotheses. The statistical results from Pretest, Posttest and Quizzes are shown in Table 5.

<table><caption>**Table 5.** Results of Normal classroom and AR classroom.</caption><thead><tr><th>Knowledge Test</th><th>Class</th><th>Medium</th><th>Parts</th><th>Mean</th><th>SD.</th></tr></thead><tbody><tr><td rowspan="5">Pretest</td><td rowspan="4">Normal class</td><td rowspan="4">Paper</td><td>A</td><td>8.37</td><td>1.56</td></tr><tr><td>B</td><td>7.63</td><td>2.19</td></tr><tr><td>C</td><td>4.67</td><td>1.12</td></tr><tr><td>D</td><td>4.57</td><td>1.43</td></tr><tr><td></td><td></td><td><strong>Total</strong></td><td><strong>25.23</strong></td><td><strong>2.73</strong></td></tr><tr><td rowspan="5">AR class</td><td rowspan="5">Paper</td><td></td><td>A</td><td>8.60</td><td>1.25</td></tr><tr><td></td><td>B</td><td>7.23</td><td>2.60</td></tr><tr><td></td><td>C</td><td>5.13</td><td>1.04</td></tr><tr><td></td><td>D</td><td>4.73</td><td>1.34</td></tr><tr><td></td><td></td><td><strong>Total</strong></td><td><strong>25.70</strong></td><td><strong>3.79</strong></td></tr></tbody></table>
<table><thead><tr><th>Knowledge Test</th><th>Class</th><th>Medium</th><th>Parts</th><th>Mean</th><th>SD.</th></tr></thead><tbody><tr><td rowspan="12">Quizzes</td><td rowspan="4">Normal class</td><td rowspan="4">Paper</td><td>A</td><td>10.60</td><td>1.40</td></tr><tr><td>B</td><td>8.63</td><td>3.05</td></tr><tr><td>C</td><td>5.23</td><td>1.19</td></tr><tr><td>D</td><td>4.53</td><td>1.48</td></tr><tr><td rowspan="4">AR class</td><td rowspan="4">Tablet</td><td><strong>Total</strong></td><td><strong>29.00</strong></td><td><strong>5.17</strong></td></tr><tr><td>A</td><td>9.83</td><td>1.12</td></tr><tr><td>B</td><td>9.87</td><td>1.76</td></tr><tr><td>C</td><td>6.20</td><td>0.71</td></tr><tr><td rowspan="4">Posttest</td><td rowspan="4">Normal class</td><td rowspan="4">Paper</td><td>D</td><td>6.07</td><td>0.69</td></tr><tr><td><strong>Total</strong></td><td><strong>31.97</strong></td><td><strong>2.91</strong></td></tr><tr><td>A</td><td>9.67</td><td>1.58</td></tr><tr><td>B</td><td>8.77</td><td>3.01</td></tr><tr><td rowspan="4">AR class</td><td rowspan="4">Paper</td><td>C</td><td>5.47</td><td>1.33</td></tr><tr><td>D</td><td>4.90</td><td>2.06</td></tr><tr><td><strong>Total</strong></td><td><strong>28.80</strong></td><td><strong>5.72</strong></td></tr><tr><td>A</td><td>8.97</td><td>1.25</td></tr><tr><td rowspan="4"></td><td rowspan="4"></td><td>B</td><td>9.57</td><td>1.89</td></tr><tr><td>C</td><td>6.07</td><td>0.91</td></tr><tr><td>D</td><td>5.87</td><td>0.73</td></tr><tr><td><strong>Total</strong></td><td><strong>30.47</strong></td><td><strong>3.07</strong></td></tr></tbody></table>

**H1:** To ensure that the children in the two experimental groups started with the same level of 3D geometry knowledge, a statistical analysis of their total pretest scores was carried out. The independent samples t-test was used to verify the significance of the difference between the two groups. The results showed that participants in the Normal Class had the same 3D geometry background knowledge level as those in AR Class as measured by the pretest scores (t(58) = .55, p = 0.586).

**H2:** The independent samples t-test was used to verify the significance of the difference between the two groups. The results (Table 6) showed that participants in the Normal Class did *not* have the same 3D geometry knowledge level *during* the learning process as those in the AR Class as measured by the *total* quiz scores (t(58) = 2.74, p = 0.008). However, when breaking it down into four parts, the two groups had the same 3D geometry knowledge level as measured by the quiz scores of Part B (t(58) = 1.92, p = 0.06). Interestingly, for Part A with the least difficult questions, the Normal Class performed significantly better than the AR Class whereas for Part C and Part D, the AR Class performed significantly better. The results suggest that the interactivity and visual-spatial representation enabled by AR can help children learn more complex geometric concepts.

<table><caption>**Table 6.** Independent sample test of the quiz scores of the Normal Class and AR class.</caption><thead><tr><th rowspan="2"></th><th colspan="3">t-test</th></tr><tr><th>t</th><th>df</th><th>p</th></tr></thead><tbody><tr><td>Part A</td><td>-2.34*</td><td>58</td><td>0.023</td></tr><tr><td>Part B</td><td>1.92</td><td>58</td><td>0.060</td></tr><tr><td>Part C</td><td>3.80**</td><td>58</td><td>0.000</td></tr><tr><td>Part D</td><td>5.14**</td><td>58</td><td>0.000</td></tr><tr><td>Total</td><td>2.74**</td><td>58</td><td>0.008</td></tr></tbody></table>
**H3:** The independent samples t-test was used to verify the significance of the difference between the AR and Normal Class in terms of the difference scores (Posttest – Pretest). The results (Table 7) showed that there was no significant difference in the learning effect between the two groups ($t(58) = 1.04, p = 0.303$), although the AR Class had higher scores in Part B, C, and D.

These results were inconsistent with those of the quiz scores (H2), which showed that the AR Class could learn better. The contrast might be attributed to the fact that for the quizzes the AR Class used the same digital presentation medium whereas for the posttest the AR Class needed to shift from the digital tablet to paper. This might have impact on the children's perceptual and cognitive load. Indeed, some attempts have been undertaken to investigate the relation between the assessment medium and performance [46, 47, 48], though the findings are inconclusive.

<table><caption>**Table 7.** Independent sample test of the difference scores of the Normal Class and AR class.</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="3">t-test</th></tr><tr><th>t</th><th>df</th><th>p</th></tr></thead><tbody><tr><td>Part A</td><td>2.25</td><td>58</td><td>0.028</td></tr><tr><td>Part B</td><td>1.60</td><td>58</td><td>0.117</td></tr><tr><td>Part C</td><td>0.36</td><td>58</td><td>0.719</td></tr><tr><td>Part D</td><td>1.88</td><td>58</td><td>0.066</td></tr><tr><td>Total</td><td>1.04</td><td>58</td><td>0.303</td></tr></tbody></table>

**H4a:** The result showed that there was a significant correlation between the total quiz and posttest scores (N = 30, $r_s = .468, p < .01$). It implies that the higher the quiz scores, the higher the posttest scores the children in the Normal class would gain.

**H4b:** The result showed that there was a significant correlation between the total quiz and posttest scores (N = 30, $r_s = .431, p < .05$). It implies that the higher the quiz scores, the higher the posttest scores the children in the AR class would gain.

## 5.2 User Experience

As shown in Fig. 1, only the participants in the AR Class completed the questionnaires on the user experience of the AR application. Results are shown in Table 8.

<table><caption>**Table 8.** Results of the UES-SF and Intention to Use</caption><thead><tr><th></th><th>#</th><th>Statements</th><th>Mean (N= 30)</th></tr></thead><tbody><tr><td rowspan="2">UES</td><td>FA-1</td><td>I lost myself in this experience.</td><td>3.70</td></tr><tr><td>FA-2</td><td>The time I spent using this AR just slipped away.</td><td>4.10</td></tr><tr><td rowspan="5">short form</td><td>FA-3</td><td>I was absorbed in this experience.</td><td>4.37</td></tr><tr><td>PU-1</td><td>I felt frustrated while using this AR.</td><td>2.03</td></tr><tr><td>PU-2</td><td>I found this AR confusing to use.</td><td>1.83</td></tr><tr><td>PU-3</td><td>Using this AR was taxing.</td><td>1.97</td></tr><tr><td>AE-1</td><td>This AR was attractive.</td><td>4.10</td></tr><tr><td></td><td>AE-2</td><td>This AR was aesthetically appealing.</td><td>4.07</td></tr><tr><td></td><td>AE-3</td><td>This AR appealed to my senses.</td><td>4.23</td></tr></tbody></table>
<table><thead><tr><th>#</th><th>Statements</th><th>Mean (N= 30)</th></tr></thead><tbody><tr><td>RW-1</td><td>Using AR was worthwhile.</td><td>4.60</td></tr><tr><td>RW-2</td><td>My experience was rewarding.</td><td>4.43</td></tr><tr><td>RW-3</td><td>I felt interested in this experience.</td><td>4.73</td></tr><tr><td>Intention to use</td><td>IU-1 Assuming I have access to this AR, I intend to use it.</td><td>4.60</td></tr><tr><td></td><td>IU-2 Given that I have access to this AR, I predict that I would use it.</td><td>4.33</td></tr></tbody></table>

**H5a:** The results of Spearman's rank test showed no significant correlations between the difference scores and the average scores of Focused Attention, Perceived Usability, Aesthetic Appeal and Reward Factors (N = 30, r<sub>s</sub> = -.246, -.005, -.019, -.114, respectively, *p* > .05).

**H5b:** The result of Spearman's rank test showed no significant correlation between the difference scores and Intention to Use (N = 30, r<sub>s</sub> = 0.075, *p* > .05).

Overall, the above findings showed that the learning effect and user experience had no significant correlation. Nonetheless, the ratings of the four dimensions of UES-SF indicated that the AR application could engage its users in the learning activities (focused attention [FA] = 4.1) with good usability (perceived usability [PU] = 4.1, reversed), high attractiveness (aesthetic appeal [AE] = 4.1) and very high level of worthiness and interest (reward [RW] =4.6). However, the level of engagement in interacting with the AR application bore no relation with the level of knowledge gain. Interestingly, the user experience also bore no relation with the intention to use, although it was rated very high with the mean of 4.5.

**H6a:** The independent samples t-test was used to verify the significance of the difference in the learning effect as measured by the difference scores between two genders in the AR Class (M<sub>male</sub> = 4.61, SD<sub>male</sub> = 3.10; M<sub>female</sub> = 5.00, SD<sub>female</sub>=3.88). The results showed no significant difference between the male and female participants (t(28) = .304, *p* = 0.764).

**H6b:** The Mann-Whitney test was used to verify the significance of the differences in the UES-SF four dimensions between two genders. The results (Table 9) showed no significant difference in the perceived user engagement quality of the AR application between two genders.

<table><caption>**Table 9.** Mann-Whitney test of the UES-SF scores between male and female participants</caption><thead><tr><th rowspan="2">UES-SF</th><th colspan="2">Gender (Mean)</th><th colspan="3">Mann-Whitney test results</th></tr><tr><th>Male</th><th>Female</th><th>U</th><th>Z</th><th>P</th></tr></thead><tbody><tr><td>Focused Attention</td><td>16.17</td><td>14.50</td><td>77.50</td><td>-1.45</td><td>.628</td></tr><tr><td>Perceived Usability</td><td>15.92</td><td>14.88</td><td>98.50</td><td>-.57</td><td>.759</td></tr><tr><td>Aesthetic Appeal</td><td>15.08</td><td>16.13</td><td>96.50</td><td>-.65</td><td>.759</td></tr><tr><td>Reward Factors</td><td>14.19</td><td>17.46</td><td>89.00</td><td>-.99</td><td>.304</td></tr></tbody></table>
**Interview/group discussion.** The original plan was to conduct individual semi-structured interviews, but it was found practically challenging to do so, given the time constraint. Instead, a group discussion with the whole class was conducted with the first author being the moderator. We summarize some key comments shared.

First, participants stated that they could learn 3D geometry with the AR app in a way better than with the traditional approach using chalk and board and paper. It was because the geometric shapes were displayed in 3D, which could be seen much more clearly than they were in 2D.

Second, the visualization effect enabled by the AR app helped children better understand the geometric shapes, because they could be zoomed, scrolled, scaled and easy to view from different angles. The most interesting part was the use of the camera to display the images and 3D shapes with quizzes.

Third, all users intended to recommend the AR app to others. It was fun to use with playful interactions and, unlike the classroom teaching, cross-sectional shapes resulted from cutting could be observed instantly with precise measures, although some participants suggested that the app support a wider range of cutting angles.

# 6 Discussion

In this section we discuss three practical issues on which we have reflected when planning and implementing our empirical work. The issues – high costs of emerging technologies and organizational constraints for running field studies - are relevant for other researchers pursuing along this line of enquiry to consider.

## 6.1 Affordability of Digital Devices

Mobile touch devices like tablets can be used in education in many ways as shown by our AR app. Learners can benefit from their compactness, portability, interactivity and personalisation. Nonetheless, our study was constrained by the costs of such devices; the issue of affordability should be addressed, considering the risk of digital divide. While we cannot estimate the future costs for immersive technology and tablets, we anticipate that they will drop in about five years, given the trend of increasing usage<fnref n="1" />. The extent of cost reduction much depends on the pace of the growing market. Above all, educationalists need to take the equality into account when employing new technology to enhance teaching and learning.

## 6.2 Augmented Reality (AR) vs. Virtual Reality (VR)

In VR environments users can manipulate objects by moving, rotating and resizing them [49]. Virtual reality provides users with immersive experience as if they were in the physical world being mimicked, thanks to sensors (e.g. accelerometer) and

<footnote n="1">¹https://www.statista.com/topics/3286/augmented-reality-ar;/2532/virtual-reality-vr/; 841/tablets/</footnote>
instruments (e.g. camera) that support multimodal and multisensory interactions [50]. Although VR can be used as an integrated educational tool to make children more engaged, it demands extra infrastructure and equipment such as head-mounted display headsets and hand-held controllers, which are expensive. The use of VR may make the issue of affordability (Section 6.1) more acute, because schools normally do not have sufficient resources to buy extra equipment. Consequently, we pursue a more promising approach to learning with AR.

## 6.3 Longitudinal study

The learning effect of an educational intervention should be long-lasting, and such effect can be evaluated with retention tests in the context of a longitudinal study [51]. Unfortunately, we were not able to implement it because we had only been given a short window of time to carry out our empirical work at the schools. In fact, getting cooperation from schools to support research study is challenging. The schools where we conducted our study could not accommodate our request to track the children's related progress in a longer-term. Outreach activities to promote benefits of getting involved in educational research may facilitate the realization of empirical work.

# 7 Conclusion

The results of this study showed that the AR Class performed better than the Normal class when the children were learning complex 3D geometry. Furthermore, the children had a high level of user engagement as indicated by the ratings of the four dimensions of the UES-SF, irrespective of the strength of the learning effect demonstrated. One encouraging finding was that the AR application could engage the children in the learning activities. However, there was no significant relation between the level of engagement in interacting with the AR application and the level of knowledge gain, which bore no relation with the intention to use the AR application in future either.

There are some limitations in our study. First, the sample size was relatively small because the challenge of recruiting schools to participate. Second, the number of tablets constrained the number of children who could take the AR Class at the same time. While this constraint should not affect the children's performance, as it was individual-based learning, it made the study much more time-consuming. A practical implication of this issue is how to contain the price of emerging technologies to make them affordable for their wide adoption as educational technology that can benefit a large number of children. Third, the target group of our study was secondary school children. It would be interesting to examine whether the results would be similar for children of different ages and with varied levels of prior experience in technology.

Overall, our study has demonstrated the potential of AR technology as an effective tool for learning complex geometric concepts. Based on our ongoing review of the burgeoning area of digital game-based learning [52], we are motivated to explore how the promise of AR for technology-enhanced learning tool will further be strengthened with the motivational power of gamification [53].
References

1. Browne, K., & Anand, C.: An empirical evaluation of user interfaces for a mobile video game. Entertainment Computing, **3**(1), 1-10 (2012)

2. Bucolo, S., Billinghurst, M., & Sickinger, D.: Mobile maze: a comparison of camera based mobile game human interfaces. In: Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, pp. 329-330). ACM, (2005)

3. Puja, J. C., & Parsons, D.: A location-based mobile game for business education. In: 11th IEEE International Conference on Advanced Learning Technologies (ICALT), pp. 42-44. IEEE, (2011)

4. Zyda, M. J., Thukral, D., Ferrans, J. C., Engelsma, J., & Hans, M.: Enabling a voice modality in mobile games through voicexml. In: Proceedings of the 2008 ACM SIGGRAPH symposium on Video games, pp. 14147. ACM, (2008)

5. Hung, C. Y., Chang, T. W., Yu, P. T., & Cheng, P. J.: The problem-solving skills and learning performance in learning multi-touch interactive jigsaw game using digital scaffolds. In: 2012 IEEE Fourth International Conference on Digital Game and Intelligent Toy Enhanced Learning (DIGITEL), pp. 33-38. IEEE, (2012)

6. Wei, S., Ren, G., & O'Neill, E.: Haptic and audio displays for augmented reality tourism applications. In: 2014 IEEE on Haptics Symposium (HAPTICS), pp. 485-488. IEEE, (2014)

7. Scholz, J., & Smith, A. N.: Augmented reality: Designing immersive experiences that maximize consumer engagement. Business Horizons, **59**(2), 149-161 (2016)

8. Pedersen, I., Gale, N., Mirza-Babaei, P., & Reid, S.: More than meets the eye: the benefits of augmented reality and holographic displays for digital cultural heritage. Journal on Computing and Cultural Heritage (JOCCH), **10**(2), 11 (2017)

9. Zsila, Á., Orosz, G., Bőthe, B., Tóth-Király, I., Király, O., Griffiths, M., & Demetrovics, Z.: An empirical study on the motivations underlying augmented reality games: The case of Pokémon Go during and after Pokémon fever. Personality and individual differences, **133**, 56-66 (2018)

10. Bower, M., Howe, C., McCredie, N., Robinson, A., & Grover, D.: Augmented Reality in education-cases, places and potentials. Educational Media International, **51**(1), 1-15 (2014)

11. Chow, E. H., Thadani, D. R., Wong, E. Y., & Pegrum, M.: Mobile technologies and augmented reality: early experiences in helping students learn about academic integrity and ethics. International Journal of Humanities, Social Sciences and Education, **2**(7), 112-120 (2015)

12. Yilmaz, R. M.: Educational magic toys developed with augmented reality technology for early childhood education. Computers in Human Behavior, **54**, 240-248 (2016)

13. Fotouhi-Ghazvini, F., Earnshaw, R. A., Robison, D., & Excell, P. S.: Designing augmented reality games for mobile learning using an instructional-motivational paradigm. In: International Conference on CyberWorlds, 2009. CW'09, pp. 312-319. IEEE, (2009)

14. Kysela, J., & Štorková, P.: Using augmented reality as a médium for teaching history and tourism. Procedia-Social and Behavioral Sciences, **174**, 926-931 (2015)

15. Adolphus, T.: Problems of teaching and learning of geometry in secondary schools in Rivers State, Nigeria. International Journal of Emerging Sciences, **1**(2), 143-152 (2011)

16. Irwansyah, F. S., Yusuf, Y. M., Farida, I., & Ramdhani, M. A.: Augmented Reality (AR) Technology on The Android Operating System in Chemistry Learning. In IOP Conference Series: Materials Science and Engineering (Vol. 288, No. 1, p. 012068). IOP Publishing.

17. Krichenbauer, M., Yamamoto, G., Taketomi, T., Sandor, C., & Kato, H.: Augmented Reality vs Virtual Reality for 3D Object Manipulation. IEEE Transactions on Visualization and Computer Graphics, (2018)
18. Lin, H. C. K., Chen, M. C., & Chang, C. K.: Assessing the effectiveness of learning solid geometry by using an augmented reality-assisted learning system. Interactive Learning Environments, **23**(6), 799-810 (2015)

19. Young, J. C., & Santoso, H. B.: Preliminary Study of JunoBlock: Marker-Based Augmented Reality for Geometry Educational Tool. In: International Conference on User Science and Engineering, pp. 219-230. Springer, Singapore (2018)

20. Liu, E., Li, Y., Cai, S., & Li, X.: The Effect of Augmented Reality in Solid Geometry Class on Students' Learning Performance and Attitudes. In: International Conference on Remote Engineering and Virtual Instrumentation, pp. 549-558. Springer, Cham (2018)

21. Morrison, A., Mulloni, A., Lemmelä, S., Oulasvirta, A., Jacucci, G., Peltonen, P., ... & Regenbrecht, H.: Collaborative use of mobile augmented reality with paper maps. Computers & Graphics, **35**(4), 789-799 (2011)

22. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C.: Current status, opportunities and challenges of augmented reality in education. Computers & Education, **62**, 41-49 (2013)

23. Mayer, R. E.: Multimedia learning (2nd ed.). Cambridge: Cambridge University Press, (2009)

24. Sommerauer, P., & Müller, O.: Augmented reality in informal learning environments: A field experiment in a mathematics exhibition. Computers & Education, **79**, 59-68 (2014)

25. Ridel, B., Reuter, P., Laviole, J., Mellado, N., Couture, N., & Granier, X.: The revealing flashlight: Interactive spatial augmented reality for detail exploration of cultural heritage artifacts. Journal on Computing and Cultural Heritage (JOCCH), **7**(2), 6 (2014)

26. Ibáñez, M. B., Di Serio, Á., Villarán, D., & Kloos, C. D.: Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness. Computers & Education, **71**, 1-13 (2014)

27. Fleck, S., Simon, G., & Bastien, J. C.: [Poster] AIBLE: An inquiry-based augmented reality environment for teaching astronomical phenomena. In: 2014 IEEE International Symposium on Mixed and Augmented Reality-Media, Art, Social Science, Humanities and Design (ISMAR-MASH'D), pp. 65-66. IEEE, (2014)

28. Kim, J., Seo, J., & Han, T. D.: AR Lamp: interactions on projection-based augmented reality for interactive learning. In: Proceedings of the 19th international conference on Intelligent User Interfaces, pp. 353-358. ACM, (2014)

29. Yang, K. J., Chu, H. C., & Yang, K. H.: Using the augmented reality technique to develop visualization Mindtools for chemical inquiry-based activities. In: 2015 IIAI 4th International Congress on Advanced Applied Informatics (IIAI-AAI), pp. 354-357. IEEE, (2015)

30. Alrashidi, M., Gardner, M., & Callaghan, V.: Evaluating the use of pedagogical virtual machine with augmented reality to support learning embedded computing activity. In: Proceedings of the 9th International Conference on Computer and Automation Engineering, pp. 44-50. ACM, (2017)

31. Kortenkamp, U., & RICHTER-GEBERT, J. U. R. G. E. N.: The interactive geometry software Cinderella. In Mathematical Software: Proceedings of the First International Congress of Mathematical Software: Beijing, China, 17-19 August 2002 (Vol. 1, p. 208). World Scientific. (2002)

32. Mechling, R.: EUKLID DynaGeo von innen-ein Blick hinter die Kulissen. (1999)

33. Holden, C.: The local games lab ABQ: Homegrown augmented reality. TechTrends, **58**(1), 42-48 (2014)

34. Norman, D., Miller, J., & Henderson, A.: What you see, some of what's in the future, and how we go about doing it: HI at Apple Computer. In: Conference companion on Human factors in computing systems, p. 155. ACM, (1995)
35. Gegner, L., Runonen, M., & Keinonen, T.: Oscillating between extremes: A framework for mapping differing views on user experience. In: Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces, p. 57. ACM, (2011)

36. Desmet, P. M. A., & Hekkert, P.: Framework of product experience. International Journal of Design, 1, 57–66 (2007)

37. Law, E., Roto, V., Hassenzahl, M., Vermeeren, A. & Kort, J.: Understanding, scoping and defining UX: a survey approach. In: Proceedings of the ACM conference on human factors in computing systems (CHI 2009), ACM, Boston, USA (2009) doi: http://dx.doi.org/10.1145/1518701.1518813.

38. Roto, V., Law, E., Vermeeren, A. P. O. S., & Hoonhout, J.: User Experience White Paper–Bringing Clarity to the Concept of User Experience (2011). (2011)

39. O'Brien, H. L., Cairns, P., & Hall, M.: A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies, 112, 28-39 (2018)

40. Jacques, R. D.: The nature of engagement and its role in hypermedia evaluation and design (Doctoral dissertation, South Bank University). (1996)

41. Webster, J., & Ho, H.: Audience engagement in multimedia presentations. ACM SIGMIS Database: the DATABASE for Advances in Information Systems, 28(2), 63-77 (1997)

42. Jokela, T., Iivari, N., Matero, J., & Karukka, M.: The standard of user-centered design and the standard definition of usability: analyzing ISO 13407 against ISO 9241-11. In: Proceedings of the Latin American conference on Human-computer interaction, pp. 53-60. ACM, (2003)

43. Cohen, L., Manion, L., & Morrison, K.: Research methods in education. (2007)

44. Unity 3D website. https://unity3d.com/unity/

45. Venkatesh, V., & Davis, F. D.: A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management science, 46(2), 186-204 (2000)

46. Noyes, J. M., & Garland, K. J.: Computer-vs. paper-based tasks: Are they equivalent?. Ergonomics, 51(9), 1352-1375 (2008)

47. Bayazit, A., & Aşkar, P.: Performance and duration differences between online and paper–pencil tests. Asia Pacific Education Review, 13(2), 219-226 (2012)

48. Cheesman, M. J., Chunduri, P., Manchadi, M. L., Colthorpe, K., & Matthews, B.: Student Interaction with a Computer Tablet Exam Application Replicating the Traditional Paper Exam. Mobile Computing, 4, 10-21 (2015)

49. Mendes, D., Caputo, F. M., Giachetti, A., Ferreira, A., & Jorge, J.: A survey on 3D virtual object manipulation: From the desktop to immersive virtual environments. In Computer Graphics Forum, 38(1), 21-45 (2019)

50. Alshaer, A., Regenbrecht, H., & O'Hare, D.: Immersion factors affecting perception and behaviour in a virtual reality power wheelchair simulator. Applied Ergonomics, 58, 1-12 (2017)

51. Ruiz-Ariza, A., Casuso, R. A., Suarez-Manzano, S., & Martínez-López, E. J.: Effect of augmented reality game Pokémon GO on cognitive performance and emotional intelligence in adolescent young. Computers & Education, 116, 49-63 (2018)

52. All, A., Castellar, E. P. N., & Van Looy, J.: Assessing the effectiveness of digital game-based learning: Best practices. Computers & Education, 92, 90-103 (2016)

53. Kapp, K. M.: The gamification of learning and instruction. San Francisco: Wiley, (2012)