IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

1

Augmented Reality Learning Experiences:
Survey of Prototype Design and Evaluation
Marc Ericson C. Santos, Angie Chen, Takafumi Taketomi, Member, IEEE, Goshiro
Yamamoto, Member, IEEE, Jun Miyazaki, Member, IEEE, Hirokazu Kato, Member, IEEE
Abstract—Augmented reality (AR) technology is mature for creating learning experiences for K-12 (pre-school, grade school, andhigh school) educational settings. We reviewed the applications intended to complement traditional curriculum materials for K-12. We
found 87 research articles on Augmented Reality Learning Experiences (ARLEs) in the IEEE Xplore Digital Library and other learningtechnology publications. Forty-three of these articles conducted user studies, and seven allowed the computation of an effect size tothe performance of students in a test. In our meta-analysis, research show that ARLEs achieved a widely variable effect on student
performance from a small negative effect to a large effect, with a mean effect size of 0.56 or moderate effect. To complement thisfinding, we performed a qualitative analysis on the design aspects for ARLEs: display hardware, software libraries, content authoring
solutions and evaluation techniques. We explain that AR incur three inherent advantages: real world annotation, contextual visualization,and vision-haptic visualization. We illustrate these advantages through the exemplifying prototypes, and ground these advantages tomultimedia learning theory, experiential learning theory and animate vision theory. Insights from this review are aimed to inform thedesign of future ARLEs.
Index Terms—augmented reality learning experience, evaluation method, learning theory, meta-analysis, prototyping✦

1

I NTRODUCTIONT

ECHNOLOGY affordances affect instructional design
and the manner of teaching. Aside from the content,
Dede [30] argues that the technological media (such as
computers) have affordances which change the learning
experience. Thus, it is important to study the effects
of integrating technology in educational settings, andhow such technologies can be maximized to improvelearning. In an attempt to show whether or not people learn better with technology, Tamim et al. [103]conducted a second-order meta-analysis of technological approaches (using computers for word processing, computer-assisted instruction, distance education,
simulation and hypermedia) against computer-free approaches to learning. Based on 25 meta-analyses representing 1055 primary studies for the past 40 years,
Tamim et al. have shown that technology slightly tomoderately improve student performance (effect size =0.35).

1.1 Objectives
The development of Augmented Reality (AR) and related technologies enabled the creation of AR educational content. Progress in hardware computing power,• M. E. C. Santos, A. Chen, T. Taketomi, G. Yamamoto, H. Kato are with
the Graduate School of Information Science of Nara Institute of Scienceand Technology, 8916-5 Takayama, Ikoma, Nara, 630-0192, Japan.
E-mail: {chavez-s, chen-a, goshiro, takafumi-t, kato}@is.naist.jp• J. Miyazaki is with the Graduate School of Information Science andEngineering of Tokyo Institute of Technology, 2-12-1 Ookayama, Meguroku, Tokyo, 152-8552, Japan.
E-mail: miyazaki@cs.titech.ac.jp

real-time tracking, graphics rendering, and AR authoring
tools contributed to applications that are already usable
in educational settings. As a first goal, this paper aims
to measure the effect of AR educational content to showwhether or not it is useful.
Although there are many educational AR prototypes
in the current literature, only a few are developed byinterdisciplinary groups and base their work on learning theory. Even if the current state-of-the-art executionof AR educational content is effective, it can only be
replicated to other contexts if a guideline exists for
applying AR for education. As a second goal, we provide
a guideline for effective AR content by first summarizing
the state-of-the-art implementation and evaluation of AR
prototypes. Then, we enumerate the affordances of AR
for learning and discuss learning theories relevant tofuture AR educational content.
1.2 Organization
The rest of the paper is organized as follows: Section
2 proposes our definition of Augmented Reality andAugmented Reality Learning Experiences (ARLEs). Section 3 describes our methods for meta-analysis andqualitative analysis. Section 4 discusses the result ofour meta-analysis of the effect of AR applications in
educational settings. We attribute these effects to the useof natural affordances of AR technology and the useof design strategies. Section 5 discusses the results of
our qualitative analysis of ARLEs covering the state-ofthe-art implementation (display and content) techniquesand evaluation techniques. Section 6 proposes a theoretical basis for the unique affordances of AR interaction,

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

starting from theories of human cognition to theories oflearning. Lastly, Section 7 concludes the paper with ourrecommendation for future educational AR prototypes.

2

BACKGROUND

2.1 Augmented Reality
Augmented reality (AR) offers a different set of affordances from the various technological interventions.
Thus, AR will be used differently from other technologies when it is applied to learning experiences. AR canbe maximized in terms of improving learning experiences by leveraging on its natural capabilities.
Before reviewing applications of AR, we need to define
AR. Azuma [9] defines AR to be when ”3-D virtual
objects are integrated into a 3-D real environment inreal time.” First, it requires the combination of virtual
elements and real environment. It is helpful to think of
AR as part of a virtuality continuum conceptualized byMilgram and Kishino [68]. On one side of the virtuality
continuum is the purely real environment, and on theother side is the purely virtual environment. AR sits
between these two extremes. The second AR requirement
is three-dimensional registration such that the virtual
elements are aligned to the real environment. The third
AR requirement is real-time interactivity with the virtual
elements. Thus, the virtual elements must behave like areal element in the real environment. This may mean, but
is not limited to, the AR system responding to changes
in the perspective of the user, changes in lighting conditions, occlusion and other physical laws.
When applied strictly to the current literature, whether
or not an application adheres to the requirements is
debatable. Many applications do not use 3D virtual
objects. Instead, they add 2D images on flat surfaces liketable-tops and books. Furthermore, many applications
do not have perfect integration or 3D registration. The
quality of implementation of integration in the current
literature varies from imitating the effect of AR [96] to afull integration in an outdoor environment [104]. In theformer, the effect of AR is simulated only by flashingrelevant information on a screen. It does not employ any
kind of tracking. Still, the author uses AR as a keyword.
On the other hand, [104] is able to integrate a virtual
butterfly on a real leaf. This is very difficult due to the
dynamics of outdoor lighting.

2

Lee [55], Billinghurst and Duenser [12] discuss some of
the notable ARLEs for pre-school, grade school and highschool education.
Chang et al. enumerates the many applicable contents
that ARLE may facilitate. In his paper, he talks about
examples of ARLEs for various subjects like physics,
chemistry, geography and mathematics, as well as, educational games for primary education. Aside from thesecontents, Lee notes the use of ARLEs for astronomy,
biology, geometry and cultural heritage. Billinghurst andDuenser explain that these kinds of content depend on
the abilities of AR to:
1. illustrate spatial and temporal concepts,
2. emphasize contextual relationships between realand virtual objects,
3. provide intuitive interaction,
4. visualize and interact in 3D,
5. facilitate collaboration.
For example, the work of Matsutomo et al. [66] createdan ARLE that demonstrates the five abilities of AR
mentioned by Billinghurst and Duenser. In their ARLE,
virtual magnetic fields were integrated with painted
blocks acting as magnets (Figure 1). Students can movethe magnets around to see how various positions would
affect the shape of the magnetic fields. In this example,
the space covered by a magnetic field and its variationin time is illustrated. The magnetic field moves with its
magnet and changes its shape as it nears another magnet.
The block magnets themselves can be moved by handproviding tangible interaction which is very natural forstudents. Lastly, this kind of system allows face-to-facecollaboration wherein students can discuss the learningmaterial in front of them.

2.2 Augmented Reality Learning Experiences
Augmented Reality, as a next-generation interface, affords a different way of interaction with information.
This interaction can be used to design better learning
experiences. We define the term Augmented Reality
Learning Experiences (ARLEs) to refer to learning experiences facilitated by AR technology. For this review, thedefinition of AR by Azuma is relaxed to accommodate
more prototypes that could help us understand how AR
can be used for education. The works of Chang et al. [18],Fig. 1. ARLE by Matsumoto et al. [66] demonstrating thefive abilities of AR that designers can leverage on. Virtuallines representing magnetic field lines are augmented
onto blocks.

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

2.3 Design Factors
There are three factors affecting ARLE design. The factor
may be hardware-related, software-related, or contentrelated. All these are inter-related with each other fromthe point of view of an ARLE application.
2.3.1 Hardware
The hardware dictates the computing power, and thephysical interface, both the kind of display and the kindof input that can be accommodated. ARLEs mainly use
desktop computers or smartphones as an AR device. Researchers using desktop computers as the platform havethree options for the display namely a computer monitor,
an overhead projector, or a head-mounted display. The
choice of device alone affects which software and content
would be appropriate. On one hand, desktop systems
have bigger screens and higher computing power. On
the other hand, a smartphone is more personal andmore mobile. We discuss some examples of content andappropriate target displays in Section 5.1.
2.3.2 Software
The software design is about maximizing the computing
power of the hardware, as well as managing content
display and handling user inputs. The unique aspects ofreal-time tracking and 3D rendering are mostly achieved
using open-source or commercial AR libraries. This aspect of AR is mature enough to enable ARLEs. Theirfurther development in the future will directly benefit
the creation of ARLEs. Currently, there are many opensource and commercial AR development kits suitable formany types of platforms. Among those mentioned in
ARLEs are: ARToolkit (FLARToolkit, NyARToolkit), Eyesweb, HUMANAR, Junaio, Opira Registration Library,
Popcode, Wikitude, and Zooburst.
2.3.3 Content
Content-related issues would be instructional design,authoring tools, and content management tools. Thissurvey focuses on exploring learning theories as basis for
effective learning experiences through AR. It discusses
the design practices in the current literature to identify
what has worked for other researchers. Instructional design is largely affected by the authoring tools available.
Authoring tools are interfaces that allow a teacher tocreate a learning experience. In cases wherein the teacher
is not familiar with programming (the more commoncase), simple authoring tools are necessary which would
allow a teacher to select and load virtual information
on a real environment. Design practices, related learningtheories and authoring tools are discussed in Sections 4,
6 and 5.2, respectively.
Content management tools are tools that handle thecontent from storage to delivery to the device. ARLE
content can be stored in the desktop PC itself. In cases
wherein the desktop PC is in a network, some prototypes
have used a server internal to the school. In the case3of some commercial AR development kits, a service for
hosting the virtual data is made available. Delivering
location-aware content to handheld AR is a big technical challenge. However, there are existing solutions
that are already explored under the fields of mobile
learning (m-learning) and adaptive hypermedia. Existing
technologies such as the Hewlett-Packard MediascapeToolkit (mscape) can be used to design systems thatdeliver location-aware content [100]. Li et al. implemented a visual interactive mobile learning system thatcan fetch multimedia content by sending an image ofa real object or by entering a predefined geographicalarea [57]. Moreover, Chang et al. have explored on
an architecture for fetching relevant data to a target
learning object in its natural environment. Their goal isto provide location-adaptive mobile ARLE [19]. In the
remote education context, [16] discusses one-to-many
remote video learning infrastructure wherein not only
do the students receive lectures from the teacher, they
can also receive AR content that they can view at homeusing a simple set up involving a handheld device andprinted markers.

3

M ETHODS

3.1 Meta-analysis
We conducted a systematic literature review based on
the work of Schmid et al. [91]. Their meta-analysis
aimed to measure the impact of technology integration
to outcomes in higher education. Their analysis of 315
effect sizes (from 231 primary studies) show that theeffects associated with technology have not changed
substantially over the years. The mean effect size oftechnology applied to higher education remains to havea wide variability around 0.28 or low to moderate effect.
The methodology for the systematic literature review is
as follows:
3.1.1 Search for Prototypes
A literature search was conducted in May 30, 2012, in
the IEEE Xplore Digital Library. The search string used
was: (”augmented reality”) AND (educat* OR instruct*
OR learn* OR teach* OR train*). To complement the candidate articles found in the IEEE Xplore Digital Library,
the search string ”augmented reality” was used to search
the publications listed by the Centre of Learning Sciences
and Technologies in [1]. Most of the 74 journal titles weresearchable through the following online bibliographic
databases:
1. EdITLib Digital Library,
2. IATED Digital Library,
3. Inderscience,
4. Sage Journals,
5. ScienceDirect,
6. Springer,
7. Taylor & Francis Online,
8. Wiley Online Library.

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

The search is limited to journal articles and conference
proceedings that are written in English, and are accessible before June 2012. A total of 503 articles (458 conference proceedings, 42 journal and magazines, 3 early
access articles) resulted from this initial search in theIEEE Xplore Digital Library. Another 150 articles wereretrieved from other online bibliographic databases.
3.1.2

Inclusion Criteria

The focus of this survey is ARLEs for pre-school, grade
school and high school education. Thus, for the research
paper to be included, the following criteria must be met:
1. The research paper must have at least a preliminary
working ARLE prototype.
2. The prototype should be applied to learning a newconcept or skill.
3. The content should be relevant to kindergarten,
primary and/or secondary education. (However,content need not be tested on these target students.)
4. The full research paper is publicly accessible.
5. The paper reports an effect size or provided ameans to calculate the effect size (reports bothmean and standard deviation).
Applying these criteria resulted in 7 articles.
3.1.3

Data Gathering

We computed an effect size using the formula:d=x̄e − x̄cs(1)where x̄e is the mean of the experimental treatment, x̄cis the mean of the control, and s is the pooled standard
deviation:s̄e + s̄c(2)s=2where s̄e is the standard deviation of the experimental
treatment, s̄c is the standard deviation of the control.
We interpret the calculated effect size based on Cohen’s recommendation, that is, an effect size of 0.8 orhigher is considered large, around 0.5 is considered
moderate, and around 0.2 is considered small.
3.2 Qualitative Analysis
3.2.1

Search for Prototypes and Inclusion Criteria

We conducted the same search as in section 3.1.1 andapplied the same inclusion criteria as in section 3.1.2
except the fifth criterion requiring an effect size. Relaxing
the inclusion criteria resulted in 87 articles, with 62
articles found in the IEEE library. Note that these 87
articles do not represent 87 unique prototypes because asmall fraction of these papers discusses advances in thedevelopment of the same prototype.
Moreover, not all these prototypes strictly adhere to
the definition of AR: integrating 3D virtual objects onto
a 3D real environment in real-time. For the purposes ofgathering experiences in implementing and evaluating

4

AR prototypes, we have decided to include the prototypes that make use of 2D images instead of 3D virtual
objects. We also included prototypes that simulate theeffect of AR but did not implement the tracking of thetarget object which is necessary to consider it as anintegration of real and virtual elements.
3.2.2 Data Gathering
A survey questionnaire was drafted to facilitate thegathering of data from the 87 included articles. The
questionnaire has four main parts namely:
1. publication details,
2. prototype description,
3. use of AR,
4. design and results of the user study.
The publication details refer to the title of paper, nameof authors, name of publications, etc. The prototype
description covers hardware, software, and content descriptions.
The use of AR refers to the possible functions oftechnology and the natural affordances of AR. Schmid
et al. [91] listed some of the primary functions of technology in the education setting. For example, technology
is commonly used to enrich and/or increase the efficiency of content presentation. The works of Brill andPark [15], and Blalock and Carringer [13] identify somenatural affordances of AR as exemplified in the previous
literature. For example, many AR applications use theannotation capability of AR to rapidly and accurately
identify objects in the real world.
The design and results of the user study refer tothe description of the control and experimental groups,
the phenomena being measured, the effect of AR onthat phenomena, etc. Aside from the performance ofstudents in pre-tests and post-tests, other aspects of thelearning experience such as motivation and satisfaction
are usually observed.
The survey was designed to easily recognize common
trends in the included papers, and to support further
data gathering about the trends that have emerged.
Whether or not an ARLE performs a particular function,
or takes advantage of a particular AR affordance isdebatable because of the diversity of ARLEs and the lackof details included in the research article. However, thegoal is not to correctly decide whether or not a particular ARLE has a particular function or uses a certain
affordance, but to gather enough examples of prototypes
that are helpful in illustrating the possible functions andaffordances of AR in the learning process. Further closer
analysis was conducted after these example prototypes
were identified.
The clarity of the survey was evaluated by having two
researchers use it separately on 20 papers out of the 87that pass the inclusion criteria. There were only minor
misunderstandings of the questionnaire and these wereclarified before proceeding to reading the remaining 67
papers. Each of the 67 papers was read only by one
researcher.

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

4

M ETA - ANALYSIS R ESULTS

Eleven articles evaluated their prototypes by conducting
experiments to compare the performance of students
who use their system versus a non-AR based approach.
Seven of these articles allow the computation of aneffect size. The seven AR applications for the educational setting, and their corresponding effect sizes are
summarized in Table 1. The four additional articles
that conducted other student performance evaluation arelisted in Table 2. ARLEs achieved a widely variable effecton student performance ranging from a small negative
effect to a large effect. The mean effect size is 0.56 which
is higher than the reported mean effect of technology (d= 0.35 or slight to moderate effect).
4.1 Affordances of Augmented Reality
The researchers designed their ARLE to take advantageof the affordances of AR technology. These affordances
are derived from the very nature of AR: the real-time
integration of virtual elements to a real environment. By
the definition, augmented reality affords:
1. Real world annotation - to display text and othersymbols on real world objects; E.g. [43] annotatesa real moving ball with values of velocity and thecorresponding graph.
2. Contextual visualization - to display virtual content
in a specific context; E.g. [22] uses AR to teachlibrary skills by adding virtual information to alibrary.
3. Vision-haptic visualization - to enabled embodied
interactions with virtual content. E.g. [63] allows theuser to view a 3D model on a marker which he canmanipulate using his hands.
4.2 Design Strategies
Aside from the natural affordances of AR, design strategies have been applied to the creation of more effective
ARLEs. In ARLEs, researchers have used the following
strategies:
1. Enable exploration - designing AR content thatis non-linear and encourages further study; E.g.
[56] allows students to try out different kinds ofscenario of collision of two balls and see if thecollision will happen in the way they hypothesize
it to be.
2. Promote collaboration - designing AR content that
requires students to exchange ideas; E.g. In [76],students were given different roles and tasked tonegotiate with each other to arrive at a solution.
3. Ensure immersion - designing AR content that allows students to concentrate more and be engaged
at a constant level. E.g. In [31], students were able
to concentrate more using AR as opposed to a
standard slide presentation.

5

4.3 Recommendation
The mean effect size of 0.56 of ARLEs to student performance should be taken critically. On one hand, it is agood snapshot of the effect of AR technology when usedin educational settings. However, we must not thinkof AR as homogeneous interventions in the learning
process given that it has a wide design space. The sevenarticles presented in Table 1 includes different display
devices, content, and experimental design. Moreover,reading these papers individually also reveals that factors such as instructional design may have played acrucial role in the success of the ARLE.
It can be argued that factors such as learning objectives, pedagogy, teaching expertise, subject matter,
grade level, consistency of use of technology, and factors
other than the augmented reality intervention may havegreater influence on the effect sizes [103]. However,as Dede [30] had argued, technology has affordancesthat affect how the content is designed. The changes in
instructional design may either be because of imperfectcontrol of the variables, or because of the technology
used. Thus findings should be interpreted carefully, andshould only be used as a guide given the variable effectsizes and the very small sample size of seven papers.
For future ARLEs, we recommend that researchers do
the following:
1. Measure learning that can be attributed to ARLEs.
There must be an experimental group who will
receive the technological intervention, and there
should be a proper control group. Factors such as
instructional design, pedagogical approaches andteaching practices should be carefully controlled
so that only the augmented reality intervention ismade variable. Imperfections in controlling these
aspects should be taken into account in the interpretation of the results. A heuristic for this is
to ensure that both the AR approach and the ARfree approach are both the best possible design for theparticular content.
2. Report both the mean and standard deviation of
the performance of students. From these values, aneffect size can be reported to measure the relative
effect of the augmented reality intervention.
3. Apply the affordances and advantages of AR as
needed by the educational setting. Augmented reality is multi-faceted and we have conducted aparallel qualitative study (Section 5) to complement
the findings of our meta-analysis. AR involves avariety of devices, interactions, and applications
that can be taken advantage of in creating learning
experiences.

5

Q UALITATIVE A NALYSIS R ESULTS

Eighty-seven papers were found in the current literaturewhen the inclusion criteria were applied to the initial
search result. The graph in Figure 2 shows the distribution of the publication year of these papers. StartingIEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

6

TABLE 1
Studies Evaluating Student Performance With Effect Sizes
Ref.

Year

Content

Participant(Sample)Control Group

Experimental Treatment

EffectSize(Average)

[58]

2009

EnglishGrade
School,Teachers (67)

Printed material and audio material

AR situated learning around thecampus

1.00

[43]

2010

Kinematics
graphsHigh School (80)didactic teaching

Physics props are annotatedwith measurements and graphs
using AR.

0.86

[63]

2010

Spatial ability

University students(49)

No spatial ability training using
AR

With spatial ability training using AR

0.70,(0.71)[31]

2012

Renaissanceart

High School (69)

Lecture with slides

AR annotated print out replicas
of art pieces

0.67

[56]

2011

Elastic collision

University students(36)

Non-AR instructional material

CollaborativeAR
whereinstudents
collision.

learning
simulate

0.33,(0.58)[41]

2011

EnglishGrade School (Sixclasses)

Lecture using audiovisual data

AR learning using magic book

0.37

[22]

2012

Library skills

Grade School (116)

Librarian teaches in the library

AR situated learning in the library

-0.28

0.72

0.83

TABLE 2Other Studies Evaluating Student Performance
Ref.

Year

Content

Participant(Sample)Control Group

Experimental Treatment

Result[98]

2010

Solar system

High School (40)

Reading the textbook and teaching aid

AR learning using magic book

Increase
29%

[76]

2009

Math game

Grade School (123)

Board game version of AR gameOn-location AR gameNo Significant
Difference

[69]

2012

Euleriangraphs

University students(20)

notebook; handouts; tablet pc;

Note-taking on top of projected
AR

No Difference

[32]

2006

Spatial ability

High School (215)

Geometry classes

CollaborativeARlearning
wherein students interact andexplore 3D shapes

No Significant
Difference

2007, there is an increasing number of papers discussing
ARLE prototypes. This review included papers published before June 2012, as such it does not include
papers from July to December, 2012. Of these 87, 72 areconference papers, whereas 15 are journal articles; 61 areindexed by IEEE Xplore, whereas the other 26 are foundin other digital libraries.

Fig. 2. ARLE publications per year until June 2012.

by

5.1 Display Metaphors
Choosing the appropriate display is an important design
decision. In the current literature, there are four typesof ARLE systems by the device used for display namely
computer monitor, handheld device (smartphone, tablet,etc.), overhead projector, and head-mounted display. Table 3 lists display devices with exemplifying ARLEs andtheir corresponding contents.
Researchers have also distinguished the displays as
either using a mirror metaphor or glasses metaphor. In[95] and [88], researchers have made this distinction
as perspectives. The glasses metaphor is a 1st person
perspective or AR that is based on what the user can see
in front of him. On the other hand, the mirror metaphor
is a 3rd person perspective wherein the user becomes
an observer of himself. We can say that it is a matter ofwhich real environment is being augmented: in front of
the user, or directed towards the user.
5.1.1 Mirror Metaphor
The mirror metaphor is when a screen appears to be areflection facing the user, except the user can also see thevirtual images integrated to his or her reflection. Figure
3 (left) shows an example of the mirror metaphor. We

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

TABLE 3Sample ARLES with Corresponding Display Devices
DisplayDesktop monitor

Handheld devices

Overhead Projector
Head-mounted display

Ref.

Content[31]

Visual art pieces[14]

Human anatomy[34]

Chemistry concepts[104]

Butterfly life cycle[36]

Electrical circuit[49]

Architectural history[37]

Physical education[87]

Spelling[108]

Playing the drums[82]

Chinese characters[98]

Solar system[47]

Endangered animals

see the reflection of the person as well as the virtual
information (vertebral column).

Fig. 3. Mirror metaphor (left) the where real environment
used is behind the user [14]; and Glasses Metaphor(right) where the real environment used is in front of theuser [104].
Desktop computers are used for the mirror metaphor
ARLEs. The mirror metaphor has been applied in ARLEs
to provide students with compelling learning experiences. For example, Blum et al. 2012 [14] used themirror metaphor in presenting an x-ray-like application
wherein the user is given an illusion of being able to see
inside his body through a computer screen. This kindof system would be useful for students studying humananatomy and sports science to help them connect theirunderstanding of human movements and muscles. In
this type of application, the mirror metaphor becomes
advantageous since the content is about studying thehuman body itself.
5.1.2

Glasses Metaphor

The glasses metaphor refers to displays wherein a user
appears to be looking into the world with a pair of
special glasses. In this case, virtual information is integrated to what the user sees in front of him. Figure 3(right) shows an example of the glasses metaphor. Three7

devices have been applied for ARLEs under the glasses
metaphor:
1. Head-mounted Display - In [98], Sin and Zaman
used the glasses metaphor to present virtual heavenly bodies on AR markers which they can manipulate in front of them. Students wore a headmounted display so that both of their hands would
be free to handle the markers containing the virtual solar system. A similar study by Shelton andHedgely [95] had argued that this visualizationis advantageous because students can more easily
understand concepts such as day and night when
they can test for themselves what happens when
one side of the earth is occluded.
2. Handheld - In the work of Tarng and Ou [104],
students can view virtual butterflies in a real school
garden to understand the butterfly life cycle. In
this case wherein students need to move around an
area, the use of handheld devices is advantageous
because all the processing, data connectivity anddisplay are found in one light-weight device. Some
researchers point out that nowadays, many people
own smartphones and tablets which are ideal forindoor and outdoor AR experiences. These handheld devices are equipped with fast processors,graphics hardware, large touchscreens, and various
sensors like camera, GPS, compass and accelerometer [12].
3. Projector - Projector-based AR affords more user
movement in a confined space, say a room, thanusing desktop systems with monitors. However, it
does not afford as much movement as in handheld
AR. The display of projector-based AR is bigger
than computer monitors and smartphone screens.
The projector-based system have been successfully
used to create a training system for playing a drumset [108], wherein, the drums are annotated withsignals for when to hit the drums. Researchers havepointed out that desktop computers and overhead
projectors are already available in most schools
making them economical for ARLEs.
5.2 Content Creation
The main concern in creating AR content for ARLEs areauthoring tools and instructional design. Developers of
ARLEs usually use AR libraries such as the ARToolkit tocreate the prototype. However, teachers need authoring
tools that would allow them to create content without
having to be proficient in programming.
According to Wang et al. [107], authoring tools for
non-programmers can be low-level or high-level. Lowlevel tools require some coding or scripting skills,
whereas, high-level tools use visual authoring techniques. Both types usually would make use of drag anddrop interfaces as well as menus. Currently, there areseveral authoring tools for any type of AR application
targeting non-programmers such as DART, ComposAR,

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

8

AMIRE and MARS which are discussed briefly in [107].
A basic authoring tool would be BuildAR [107] whichallows the teacher to scale, translate and position virtual
objects with respect to a fiducial marker. For example,
virtual art pieces can be displayed in a real room tocreate an AR museum experience [102].
5.2.1

Magic Book

In the current literature, some researchers have developed ways to author AR content for learning. Researchers are exploring mainly three kinds of educational
material namely, magic books, learning artefacts andlocation-based content. Researchers consider the book
metaphor as one of the main modes of how AR will
be used in education [12]. Using the book as the realelement, additional virtual content can be added onto it
using AR making it a magic book. Students are alreadyfamiliar with books which afford many natural ways ofinteraction like flipping pages and viewing the book at
different perspectives. For example, the work of Jee et al.
[41] and [42] talks about an authoring tool for markerlessbooks. The work of Vate-U-Lan [106] used a commercial
authoring tool, Zooburst [4], for making an ARLE basedon the magic book metaphor.
5.2.2

Learning Artefact

Another trend in the current literature is the use oflearning artefacts, wherein students can learn about aparticular object. Rahman et al. [84] developed an authoring tool that would allow a teacher to present asmall object on a camera, and annotate it with virtual
information using a live video visualization. Aside froma desktop PC, they used a depth camera to capture thecolor image and corresponding depth information. They
used the ”polygonal annotation scheme” wherein theteacher could draw a polygon on the screen where heor she could see the object he or she wants to annotatewith information.
In [97], Simeone and Iaconesi have pre-trained a system to recognize parts of a sculpture called the Minkisi.
Users can then annotate virtual information on the realparts of the Minkisi via a desktop system. They describe
their use case to involve multiple types of users such as
curators, teachers and students. Each of them can annotate information on specific parts of the artefact basedon various online sources. Users can then evaluate eachother’s information based on accuracy and usefulness.
Thus, content is developed through a process of communication among the different author-consumer withthe physical object as the central point of conversation.
Figure 4 shows the Minkisi and the two parts wherestudents can annotate information.
5.2.3

Location-based Content

Location-based game is an emerging learning technologythat takes advantage of providing relevant information
to a place such as buildings and artefacts. Researchers

Fig. 4. The Minkisi artifact and the parts of interest[97]. Systems can be trained to recognize parts of an
object, which later on can be augmented with information,
making the object the locus of discussion.
have shown that novel e-learning experiences can beprovided by delivering content relevant to a place (e.g.
history). Chang et al. [20] have demonstrated this using handheld devices equipped with RFID tag readersand GPS positioning to deliver information regarding
objects found in the real world. Although the game itself
represents most of the pedagogical design, AR is aneffective display for learners because it integrates thevirtual information directly onto the target real world
object. Moreover, AR has the advantage of using thecamera for detecting and tracking an object in the realworld, as opposed to using RFID readers which are
not readily available in handheld devices. Furthermore,
putting RFID tags on the real world may not be feasible
in cases wherein objects should not be touched such as
some historical artefacts.
ARLEs can offer location-based content usually used
for outdoor AR experiences with handheld devices.
Klopfer and Sheldon [52] have developed the AR GameBuilder that allows teachers to create educational AR
mobile games in a specific locality. Their system offers
predefined maps and GPS coordinates to place virtual
objects such as characters in a specific place in the world.
According to Klopfer and Sheldon, their platform also
encourages students to transfer an existing game, toanother place such as their own city; or even create theirown game.
5.2.4 Collaborative Content
AR authoring itself is seen as an educational experience. For example, Billinghurst and Duenser have used
BuildAR to let students create their own AR scene.
In this process, students develop skills such as mathabilities, drawing digital content, and telling stories. In
this use case, it is important that the authoring tool isIEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

easy enough even for grade school students to use. In
the case of the prototype of Simeone and Iaconesi, thecontent is developed in a conversational style wherein
the cultural artefact becomes the locus of the discussion.
Students could verify each other’s inputs by researching,
and teachers can be able to focus the discussion of theclass based on the learning objectives.
The prototype by Odeh et al. 2012 [73] allows students
to add virtual wires onto a real circuit and conduct experiments. Such systems make the student feel the realityof components and instruments used in an engineeringexperiment without the students physically accessing thelaboratory. Moreover, the remote laboratory is available
to them for much longer time than a regular laboratory.
Lastly, Klopfer and Sheldon have tested the AR GameBuilder and the students have successfully used it for
creating linear games which involve both technical skills
and creative writing. Table 4 summarizes some authoring tools that have been used for authoring ARLEs.
TABLE 4
Authoring Activities in ARLEs
Ref.

Authoring Activity[74]

Students can add virtual wires on a real circuit ina remote laboratory set up.

[41], [42]Teachers can author e-learning applications usinga markerless magic book metaphor.

[97]

Teacher, students and the curator can all access andadd information related to a cultural artefact.

[24]

Real props for physics classes such as balls, carts
and rods are augmented with virtual informationsuch as text and arrows.

[52]

Teachers and students can create and access educational games using predefined maps and GPS coordinates, virtual characters, and file management
support.

[102]

Teachers can make a web-based virtual museum
by selecting 3D models, associating them to markers, and arranging them in a room.

[84]

Allows a video annotation approach in order tocatalogue and add virtual information on physicallearning artefacts in a scene.

[99]

The CONNECT Visual Designer allows educators
to specify the interactions the learner can havewithin the AR environment by creating rule-based
scenarios.

5.2.5

Architecture

Learning objects - ”any entity, digital or non-digital, thatcan be used, reused, or referenced during technology
supported learning” [25] - can be used as the model
for packaging and distributing ARLE content. In [90],Santos et al. recommends thinking of ARLE content tohave three internal components that can be edited tomake it adaptable to multiple educational settings. The
first component is the context which refers to the target
object or environment. The second component is thedigital content which includes 3D graphics, annotative
words and symbols, and images. The last component

9is the instructional activity. For example, the work [52]allows teachers and students to author, and re-author alocation-based game. The context is the initial real placeof the game with landmarks in the real world. The digitalcontent are GPS coordinates, virtual characters, etc. The
instructional activity is the game itself. Such game wasmade adaptable into a different context or a different realplace in the world by allowing teachers and students
to have re-authoring capabilities. Teachers or students
themselves can make changes in the digital content suchas moving virtual characters. Therefore, one locationbased game designed for one city can be adapted into a
different city.
5.2.6 Instructional Design
Aside from authoring tools, instructional design is also
an important consideration in building ARLEs. To adapt
ARLEs in formal education, a special curriculum should
be designed to carefully integrate the use of AR and
its various accompanying devices into classroom use.
In [65], Mathison and Gabriel suggest three stages inintroducing ARLE to an existing curriculum. The assumption is that the students have never used special
devices and AR technology before to learn a concept.
The objective of the first stage is to teach the student
skills such as exploration, investigation and evaluation
using the chosen AR display such as smartphones anddesktop computers. The next stage is to introduce the capabilities of AR such as the allowed interactions. Lastly,
the students can now experience the ARLE, as well as,
build their own AR experience. Furthermore, Mathison
and Gabriel, recommends a carefully designed day-today curriculum that would state the learning objective,
AR time allocation, AR experience, and list of related
documents necessary for conducting the ARLE.
5.3 Evaluation Techniques
Of the 87, 43 papers have performed user studies on
the system regarding ease of use, satisfaction, immersion,
student motivation and performance, among others. The
number of students involved in the study varied from4 [23] up to 419 [38] with a median sample size of 36
students [56]. The proper choice of evaluation method
for an ARLE depends on the purpose of the evaluation.
In our review, we observed two primary purposes: toshow whether or not an ARLE is beneficial to learning,
and to measure user experience and discover possible
improvements.
5.3.1 Proving ARLE Benefits
Researchers need to prove the benefits of using their
ARLE. Thus, they compare either the performance or
the motivation of students when using an ARLE (theexperimental treatment) and when using a more traditional medium of instruction (the control). To measure
student performance, the students take a test to measure their mastery of the content. Scores of students

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

belonging to the experiment group and control groupare then compared to see any differences in learning.
Such comparison between ARLE users and non-users are
summarized in Table 1. The most critical considerationin this kind of evaluation is the execution of a control
group. As much as possible, the researcher must provide
the best control possible. For example, to evaluate alocation-based AR game, O’Shea et al. [76] designed aboard game version of the AR game for students to use
in a control scenario.
Aside from possibly improving student performance,
ARLEs can be used to increase the motivation of students in educational settings. Abstract constructs suchas motivation can be measured by expertly designed
questionnaires such as [50] and [2]. In [50], motivational
design process is modelled to have four steps. First is
to gain the attention of student. Then, guarantee that
content is relevant to the student. Next, learners should
build confidence by feeling in control and expecting tosucceed. Lastly, the reflection of the students on theirperformance will determine their degree of satisfaction.
Whereas [31] focused on designing motivational learning
experiences, [40] focused on motivation towards selflearning. In [40], the researchers used a part of the Intrinsic Motivation Inventory (IMI) to analyse four subscales:
enjoyment, competence, usefulness, and tension. The IMI
had been previously applied to successfully measuresubjective experiences related to a performed activity.
5.3.2 Discovering Usability Issues
Researchers also evaluate their ARLEs to measure some
aspect of user experience and discover possible improvements to the current prototype. In this evaluation,user study participants are observed while they use theARLE, and asked questions in the form of an interview
or questionnaires after using the ARLE. Survey questionnaires are the most commonly used evaluation toolin the current literature. Questionnaires are designed
to measure an abstract construct such as the user’s
feelings of satisfaction, enjoyment or immersion while
using the system. After researchers decide on a construct
to observe, they either use existing questionnaires, orcreate their own questionnaire.
Expertly designed questionnaires have been tested
for validity and reliability, therefore, they give a bettermeasure of the construct of interest. However, crafting
a questionnaire specific to a prototype is sufficient todiscover possible improvements in that prototype. Currently, there is a need for expertly designed questionnaires to accurately measure ARLE-relevant constructs(e.g. immersiveness). Moreover, an expertly designed usability questionnaire is needed to systematically improve
ARLEs under iterative prototyping. Table 5 shows the listof ARLEs that used questionnaires, their corresponding
metric or construct evaluated, and the tool that theyused.
Some researchers, [101] and [48], have adapted
ISONORM, a general software usability questionnaire.

10

Using this questionnaire, they were able to observe
aspects of interface design such as conformity withuser expectations, controllability, error tolerance, selfdescriptiveness, suitability for learning, and suitability
for the task.
Among the most observed construct are ease of use,
usefulness and intention to use. In the current literature,researchers usually ask directly if a system is easy touse, if the user thinks it is useful, and if they would
use the same system for other subject matter. Therefore,
most of the available literature measure perceived easeof use and perceived usefulness. However, it is possible
to measure ease of use such as counting errors whenusing the interface and time on a certain task.
5.3.3 Other Evaluation Methods
Other evaluation methods also have their own advantages depending on the context of evaluation.
1. Interviews are useful for learning about qualitative
data that cannot be captured by written responses
to questionnaires. For example, interviews were
useful in learning about technology acceptance[101], [28]; possible advantages of ARLE than thecurrent practice of a teacher [51], [26]; and learners’
opinion about technology including perceived easeof use, perceived usefulness, intention to use, etc.
There are also cases in evaluating ARLEs thatinterviews would be preferred compared to questionnaires. In cases wherein the respondents areyoung children or persons with disabilities [110],
it is better to conduct interviews in order to communicate effectively what is being asked.
2. Observing and coding overt behaviors have been
adopted by several papers to see how their target
user would interact with an ARLE prototype. Observation is done to reveal possible improvements
for better performance and satisfaction of the user.
Behaviors can be divided into two: verbal and nonverbal. Verbal behaviors can be specific keywords,
expressions, questions or statements a learner says
while using the ARLE. Aside from verbal behaviors, nonverbal behaviors can also be revealing of aparticipant’s experience of an ARLE. These include
facial expressions (frowning, smiling, surprise, etc.)
or body language (fidgeting, leaning close to ARLE,
scratching the head, etc.) [105].
3. Expert review was used by Margetis et al. [62] toevaluate touch-based interactions with ARLE basedon the book metaphor. They employed 4 usability
and interaction design experts to perform heuristic
evaluation with questionnaires based on the workof Nielsen and Mack [72]. The main goal of theexpert review is to identify potential usability problems, and check conformity against five dimensionsof usability: effective, efficient, engaging, error tolerant, and ease of learning [83].

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

11

TABLE 5
Summary of Preliminary Studies Using Survey Questionnaires
Ref.

Metrics or Constructs

Tools

[31]

attention, confidence, relevance, satisfaction

Instructional Materials Motivation Survey [50][39]

enjoyment, competence, usefulness, tension

Intrinsic Motivation Inventory [2][86]challenge, collaborativeness, competition, ease of use, movement, rewards, situated learning

Constructivist Multimedia Learning Environment Survey[61], Preferences for Internet Learning[56]

collaborativeness, interest, perceived skill development

Learning Effectiveness [5][80]

use of computer, use of video game, science process

Self-Efficacy in Technology and Science Short Form [6][21]attitude to e-learning, e-learning experience

Technology Acceptance Model [29][98]ease of use, effectiveness, learnability

Crafted their own questionnaire[63]

attractiveness, ease of use, usefulness

Crafted their own questionnaire[46]ease of use, enjoyment, perceived skill development

Crafted their own questionnaire[47]ease of use, enjoyment, usefulness

Crafted their own questionnaire[102]

enjoyment, perceived presence

Crafted their own questionnaire[101]

controllability, ease of use, learnability, self-decriptiveness

ISONORM Usability Questionnaire [81][7]

wearability

Crafted their own questionnaire[45]ease of use, engagement, perceived presence, usefulness

Crafted their own questionnaire[60]ease of use, intention to use, usefulness

Crafted their own questionnaire[59]ease of use, usefulness

Crafted their own questionnaire[99]

attitude, ease of use, interest

Crafted their own questionnaire[8]ease of use, intention to use, learnability, perceived correctness andresponsiveness of system

Crafted their own questionnaire[93]ease of use, intention to use, perceived correctness and responsivenessof system

Crafted their own questionnaire[54]ease of use

Crafted their own questionnaire[28]ease of use, expectations of AR, perceived affordances, usefulness

Crafted their own questionnaire[33]ease of use, perceived efficiency, usefulness, preferred subjects to use
ARLE

Crafted their own questionnaire[108]

comfort, enjoyment, intention to use, interest, perceived skill development, usefulness

Crafted their own questionnaire[48]

conformity with user expectations, controllability, error tolerance, selfdescriptiveness, suitability for learning, suitability for the task

ISONORM Usability Questionnaire [81]6

A PPLYING T HEORY TO ARLE

The design of ARLEs should take advantage of theaffordances of AR as enumerated in Section 4.1. Theseaffordances of ARLEs to learning are supported by theories of cognition and learning. These theories help explain how and why ARLEs can be beneficial to learning.
Furthermore, insights from these theory can be used to
improve the design of ARLEs.

activated so that a person can operate cognitive processes. These pieces of information are either perceived
from the environment or retrieved from the LTM [44].
The LTM contains the knowledge in inactive state. It
organizes information into structures that facilitate theretrieval of information. Figure 5 illustrates the flow of
information through the human memory system.

6.1 Hypotheses Based on Cognition
Researchers of human-computer interaction apply psychological theory and methods in understanding humanbehavior. Among the many contributions of the fieldof psychology, we have relied heavily on the current
understanding of the human memory in designing betterinterfaces. Based on theory, we hypothesize how ARLEs
can be beneficial to learning.
Cognition theories suggest that the human memory iscomprised of two related structures: short-term working
memory (STM) and long-term memory (LTM). The STM
stores information temporarily, whereas, the LTM maintains information more permanently. The STM acts as alimited working memory where information is temporarily

Fig. 5. Theory of memory as proposed by Atkinson and
Shiffrin emphasizing the interaction of sensory stores,
STM and LTM. Input from the environment is held fora few hundred milliseconds in the sensory register. The
STM acts as a working memory holding data from both
sensory registers and long-term memory, and performing
cognitive processes [85].
People store much more information than what ispresented to them. Aside from the concepts to be remembered, we also generate associations that help us makeIEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

12

sense of, and thus remember, information. Elaborationis the creation of such additional associations. There
are two kinds of elaboration: imprecise elaboration andprecise elaboration.
Imprecise elaboration is the creation of additional
associations unrelated to remembered information.
Whereas, precise elaboration is the creation of additional
material closely related to remembered material [85].
When learning, students sometimes take advantage ofelaborative rehearsal: a memory technique that involves
thinking about the meaning of the term to be remembered. Aside from this semantic processing, the use ofvisual imagery and haptic information can also be a
powerful form of elaboration of knowledge.
Based on this understanding of human memory, wehypothesize that there are three ways ARLEs help students learn better:
1. Real world annotation improves perception. It juxtaposes real objects, and virtual text and othersymbols. This reduces cognitive load in the limited working memory so that a bigger fractionof the STM can be used for operating cognitive
processes(e.g. storing in the LTM).
2. Contextual visualization improves elaboration. ARLEs provide more meaningful cues found in thereal environment that help a student construct a
more elaborate network of knowledge.
3. Vision-haptic visualization improves elaboration
based on embodied imaging. It presents visual
information in two modalities: sense of sight andsense of touch.
Real world annotation, contextual visualization, andvision-haptic visualization are discussed under Sections
6.2, 6.3 and 6.4, respectively.

of learning. The system of objects is augmented withtext information or other symbols with the purposeof providing a learning experience. By definition, thevirtual information is the text or symbol, and the realenvironment is the system of objects. To be consistent to
the definition of AR, AR annotation requires tracking thephysical objects such that the text information appears
as labels that follow the real object.

6.2 Real World Annotation
The most basic application of AR is the annotation ofreal world objects and environments. Merriam-Webster
defines annotation as ”a note added by way of commentor explanation.” The data being annotated would be text
meant to explain a concept. Many AR applications, including ARLEs, use text as the virtual information being
overlaid to the real environment. However, annotation
with AR is not limited to text. It could also involve othersymbols and icons. This includes, but is not limited to,arrows and basic shapes such as circles and lines used
to highlight or direct a user’s attention.
AR annotation is the juxtaposition of real world objects
or environments with virtual text or virtual symbols that
help explain a concept to a user. Some researchers also
use the term annotation to refer to information-tagging
such that a physical object becomes a link to accessimages, videos and webpages. In this paper, informationtagging is not considered as AR annotation.

From this theory, it can be argued that learning with
AR annotated objects is better than learning about thesame object with other reference material such as amanual or a separate online source. For example, inlearning how to play a guitar, it will be better to learn
about the finger positions highlighted onto the guitar,than referring to a sheet summarizing the finger positions for each chord. By the definition of AR annotation,
three empirically-proven principles namely Multimedia
Principle, Temporal Contiguity Principle, and Spatial
Contiguity Principle guarantee that learning with AR
annotated physical objects will lead to better learning
performance than a more traditional way of learning.
The extensions of these principles to AR annotation are
shown in the Table 6.

6.2.1 Real Object-centered
ARLEs that use AR annotation are a class of ARLEs
wherein a system of objects become the central point

6.2.2 Multimedia Learning Theory
The benefits of AR annotation can be directly explainedusing Multimedia Learning Theory [67]. In this theory,
multimedia refers to words (written or spoken) andpictures. Multimedia learning theory has three assumptions namely dual channels, limited capacity, and active
processing. The first assumption is that there are two
separate channels for visual information and auditory
information. The second assumption is that both thesechannels can only accommodate a limited amount of
information at a given time. Lastly, the third assumption
is that humans are active learners. Incoming informationfrom the channels are processed by organizing them into
coherent mental representations, and integrated to previously acquired knowledge. Using these three assumptions, Mayer has derived and empirically proven designprinciples in authoring multimedia learning materials.
Of these principles, the following are directly related to
AR annotation applications namely: Multimedia Principle,
Spatial Contiguity Principle, and Temporal Contiguity Principle.
Multimedia learning theory can be extended to AR
annotation by doing two substitutions:
1. The system of real objects replaces the picture.
2. The virtual texts and symbols replaces the words.

6.2.3 Memorization
The principles of multimedia learning theory were tested
both on printed materials and computer-assisted instructions. It has not yet been tested for AR annotation
applications for learning. However, Fujimoto, et al. [35]has demonstrated how the annotative abilities of AR canIEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

TABLE 6
Multimedia Learning Principles Supporting theEffectiveness of AR Annotated Objects [67]Principle

Extension to AR Annotation

Multimedia andTime Contiguity

People learn better from annotated virtual
words onto physical objects than from separate multimedia (e.g. illustrated manual) andphysical objects.

Spatial Contiguity

People learn better when corresponding virtual words and physical objects are presented
near rather than far from each other on thescreen.

ease memorization tasks. In their study, the memorization abilities of users were tested for when they memorized symbols by annotating information near the target
object (Display 1), against displaying the information ona random place while connected by a line (Display 2),
and on the same place, say at the top left of the display(Display 3) as shown in Figure 6.

Fig. 6. Three display methods. On the left is AR Annotation wherein a label is placed near the relevant part of themap. The middle and right are control scenarios wherein alabel is displayed randomly on the map with a connecting
line, and a label is displayed consistently at the top leftof the HMD screen without a connecting line, respectively[35].
Fujimoto, et al. conducted two types of memory tests:
identification and association. In these tests, each of theparticipants are shown 10 symbols one at a time. The
identification test asks the participants to identify the 10
symbols they just saw from 25 symbols. Whereas, theassociation test asks the participants to identify wherein the map they saw the image. In both tests, Fujimoto
et al. measured the accuracy of answers, as well as thetime it took for the participant to answer.
Results show that annotating the information on top ofan object (Display 1) allowed the users to memorize thelabels significantly better than when displaying the information on random places (Display 2), or on the sameplace on a display (Display 3). In the identification tests,
participants were able to able to achieve an accuracy of
99% with Display 1, 95% with Display 2 and 96% with
Display 3. The bigger difference is in the answer time
wherein users of Display 1 answered in a shorter timeof 45 seconds, compared to 53 seconds and 52 seconds
for Displays 2 and 3, respectively.
In the association tests, participants were 87% accurate

13

with Display 1. Whereas, they are only 70% and 75% accurate with Displays 2 and 3, respectively. Furthermore,participants who used Display 1 finished the test in 96
seconds, compared to 112 seconds and 99 seconds for
Displays 2 and 3, respectively.
All of these tests were proved to be statistically significant in the paper of Fujimoto et al. Annotating virtual
information near an object makes perception easier, andcan be use to better present information in educationalsettings.
6.2.4 Examples
One example of AR annotation is the work of Simeone
and Iaconesi [96]. In their work, they trained their system
to recognize 3D parts of a model airplane (Figure 7.a) sothat they can display relevant information for each 3D
part. Their system makes use of a personal computerequipped with a webcam. The virtual information canbe viewed on the computer monitor together with thereal environment including the airplane model and theuser.
The authors mentioned two use cases. First, instructions on how to assemble the several pieces into an
airplane can be annotated on to the 3D part. When a user
picks up a piece and puts it near the webcam, an instruction relevant to that part is displayed at the bottom of
the screen. Instead of the student going back and forthfrom a manual to the actual objects, the airplane model
pieces can be augmented with the manual instructions.
Second, the airplane model can be layered with several
kinds of information that the students can access byfocusing on specific parts of the plane. The information
was taken from various online sources. This prototype islimited in its annotating capabilities because the systemdoes not have a tracking feature. With a tracking feature,
the annotated information can be able to follow the real
object wherever it is on the screen. However, for the purposes of a prototype, this work is a good approximationof how AR toys can be used in the near future.
Instead of text information, other symbols and shapes
can be used to annotate objects. In physics education,
magnetic field lines (Figure 1) [66] and directions offorces acting on an object (Figure 6.c) [99] have beenannotated to real objects like magnets and carts, respectively. With this feature, students can visualize abstract
phenomena like magnetic field and force.
Another set of compelling examples can be found in
ARLEs with the goal of teaching how to play musicalinstruments. AR applications have been developed toteach people how to play the guitar [70], drums [108]and piano [39]. In [70], a desktop system was used torender a virtual hand on top of a real guitar. The student
can then position his hands correctly on a guitar. Insteadof the student translating a chord sheet indicating which
strings to press at which fret, this information is alreadyannotated on the guitar itself.
In [108], a projector-based AR annotation was used toindicate on a drumset which drums to hit by projectingIEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

14

TABLE 7
Examples of AR Annotation

Fig. 7. Some ARLEs demonstrating annotation. (a)shows the parts of the airplane that can be recognized
and annotated with words in the work of Simeone and
Iaconesi [96]. (b) shows the virtual hands and letter annotated on a real guitar [70]. (c) shows a cart augmented
with arrows representing forces acting on it [99].
circles on the top of the appropriate drum. Instead of ateacher demonstrating to the student and pointing whichdrums to hit, this information can be augmented directly
onto the drum set.
Lastly, in [39], a desktop AR annotation system is use
to demonstrate finger positions on a piano. Instead of a
piano teacher demonstrating the proper finger positions,this information can be augmented on a keyboard. Thesesystems do not intend to replace formal training withmusic teachers. However, these systems are very usefulfor self-studying outside of the formal class, and formusic teachers to create learning materials for theirstudents. The other examples of ARLEs using annotation
are listed in Table 7.
6.3 Contextual Visualization
The ARLE developers have designed ARLEs such thatit makes use of contexts that students can relate to. The
current literature on ARLEs suggests that AR inherently
offers a contextual visualization of virtual information.
Contextual visualization refers to the presentation ofvirtual information in the rich context of a real environment. The virtual information is always presented withina context, that is, a real environment. This context is filledwith cues that help students construct their knowledge[89].
6.3.1 Experiential Learning Theory
Multimedia learning theory provides a learning theoryof how real world annotation by AR can help students

Ref.

Real Objects

Virtual Annotations[31]

Printed replicas ofRenaissance art

Text details relevant to art[22]

Library

Verbal hints from a virtual character about the structure of thelibrary[66]

Magnets

Magnetic field lines[11]

University
campusSignages

[39]

Piano

Highlights of correct finger position of a musical note[40]

Go Boardgame

Boardgame pieces (stones) addedin strategic patterns[75]

Playing cards

Scoreboard and strategic use ofcards on hand

[97]

Cultural Artefact

Text details relevant to anthropology[108]

DrumsetHighlights of sequence for hitting
drums

[96]

Airplane model

Description and instruction relevant to the airplane part

[70]

GuitarHighlights the finger position of
chord

[99]

Cart and other objects

Forces acting on the object as it
moves

[24]

Ball

Arrows representing instantaneous velocity, acceleration, andcentripetal force

learn better based on human cognition and related processes in the brain. This theory is complemented byExperiential Learning Theory which views entire experiences as the source of learning. Experiential learningtheory is different from cognitive theories of learningthat give primary emphasis on acquisition, manipulation, and memorization of symbols.
Experiential learning theory explains that people learn
by creating meaning from their personal experiences.
It describes four stages in the learning cycle as shownin Figure 8. Learning starts with having a concrete
experience, which becomes the basis of observation andreflection. From our observations, we formulate theories,
which we test for implications in new situations. Resultsof this testing stage provide new concrete experiences[53].

Fig. 8. Lewinian Experiential Learning Model

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

6.3.2

TABLE 8
Examples of Contextual Visualization

Contextual Learning

Contextual learning is a curriculum design philosophy
that applies experiential learning theory. It recognizes
the importance of context in learning experiences, thatis, learning only occurs when students process new
information with their personal experiences [3]. In theclassroom, contextual learning is observed when using
five strategies [27]:1. Relating - to link a new concept to something
students are familiar with,
2. Experiencing - to let students explore, discover orinvent so that they can learn by doing,
3. Applying - to give students an opportunity to use
the concepts in realistic and relevant exercises,
4. Cooperating - to give the students a chance to share,
respond and communicate with other learners,
5. Transferring - to give the students the chance to use
their new knowledge in a new context or novel
situation.
Similarly, contextual visualization using ARLEs canbe used as a strategy to link virtual information to an
object or an environment that a student is familiar withto provide more effective learning experiences.
6.3.3

15

ExamplesIn [58], students learn the English language around a
campus using a handheld ARLE prototype. The lesson
included vocabulary and sentence patterns used in theclassroom, gallery, library, gym, laboratory, cafeteria, andhealth clinic. In this instructional design, students are
already familiar with the school campus and the language used in the different types of places on campus.
By presenting information in these real environments,
the virtual information is automatically related to what
the students are familiar with. This makes it easier forthem to relate it with their previous experiences.
Moreover, some of these ARLEs offer some form ofexperience that would otherwise be difficult to observein real life. An example would be [104] which allows thevisualization of virtual butterflies in the school garden.
In the normal setting, it is difficult to observe a butterfly
from egg to a full butterfly. In some cases when students
live in a very urbanized area, butterflies may not evenbe readily observable. Moreover, the method in [104] isa better visualization because the students are alreadyfamiliar with the school garden. It makes it easier forthem to learn that such garden ecosystems may include
butterflies that undergo metamorphosis.
The last example is [22] which aims to teach students
library skills inside a library setting. This gives thestudents the chance to try looking for books and finding
information for themselves. The library was layered withadditional information that both directs the students andscaffolds them into being able to find information theyneed. The other examples are summarized in Table 8.

Ref.

Content

Context[58]

English language

School campus[104]

Butterfly life cycle

School garden[22]

Library skills

Library[86]

Animal and plant life

Local park

[65]

AnimalsZoo[49]

Architectural history

Building[78]

Finding offices of institutions designed fordeaf students

Street in front of school[52]

Location-based
game

Local neighborhood

story

[77]

Asian art

[7]

Airflow,
Force

Museum[64]

Location-based
gamefor learning scientificlanguage

Local neighborhood[71]

Historical events

Original place where event
happened

Magnetism,

Science Centre

6.4 Vision-haptic Visualization
Recently, researchers have argued the benefits of tangible interfaces for learning based on the additional
physical actions they afford and the possible face-toface collaborative activities they support. Moreover, thecoupling of tangible user interfaces with AR enables aclose mapping between tangible input (manipulations ofa physical object) and digital output (on-screen display)[92].
Vision-haptic visualization is the integration of both thesense of sight and the sense of touch in perceiving virtual
information. This is mainly observed when changing the
viewpoint of a user such that in ARLEs: The user canpick up an object, and inspect the corresponding virtual
content from different angles by rotating the object, andmoving the object nearer or farther from them. Moreover,the users can move around an object to see the otherparts of the corresponding virtual object.
6.4.1 Animate Vision Theory
Shelton and Hedley [95] argue that embodied interactions are more natural visualizations of virtual information based on the Animate Vision Theory [10]. This theory links visual perception to acting and moving in thephysical world. Visualizations in learning experiences
should take advantage of visual stimuli and motor responses. Straight-forward examples are providing feedback using virtual reality (VR) for teaching a person how
to dance [109]. In a test involving 8 participants, Chan etal. have demonstrated a VR-based dance training systemwith a very large effect size of 1.66 [17].
Aside from VR, AR is applicable for this purpose
because it allows users to use their hands and entire bodies to change the perspective of visualization.

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

In an empirical experiment involving 100 participants,
AR users have been shown to accurately finish spatialproblem-solving tasks in 1.57 seconds (22%) faster thandesktop users during the first use. However, this margindecreased to 0.3 seconds (5%) after the desktop users
have sufficient practice using the desktop system [95].
The work of El Sayed et al. [33] provides a straightforward illustration of vision-haptic visualization. They
proposed a system wherein each student is given AR
marker cards. Depending on the current lesson, 3D
virtual models of related objects can be assigned to themarkers and then viewed by the students on top of thecards. Instead of illustrating the learning object as animage projected on a screen or printed on a book, the students are presented with a 3D virtual object. (They recommend using the mirror metaphor since many schools
are already equipped with desktop computers.) Basedon the work of Shelton and Hedley, this interaction with
virtual information is superior because of two reasons:
1. The interaction of students with the virtual object
is not mediated by a mouse or keyboard. They
can move around the virtual object or change its
orientation using their bare hands. This kind ofinteraction is closer to what we normally do when
we study an object. We pick it up, sometimes wetilt our head to see the sides, we hold it closer
to us to make it bigger, etc. In [32], the students
can move around a virtual geometric shape. The
user’s body becomes the way to interact withinformation against the more conventional mouseand keyboard.
2. This interaction is better for some applications
compared to virtual reality which presents the user
with both virtual environment and virtual objects.
In virtual reality, we try hard to provide the illusionof transporting the user to a different world. In
some cases, it is better to keep the users’ awarenessof themselves and their immediate surroundings.
When using AR, the students’ awareness of theirbodies and their real environment remains intact.
For example, the work of [14] allows the students tovisualize the muscles and organs inside their bodies. If they move their shoulder or arms, they can
see the body organs at different angles. This ARLE
is able to provide a more compelling experienceby contextualizing the virtual information to thelearner’s body.
6.4.2 Examples
Several ARLEs (Table 9) provide embodied interactions
when visualizing virtual data. For example, [63] displays a virtual object on a notebook. Then, students canrotate and tilt the notebook to see other parts of thevirtual object. Another example would be [66] wherein
students move around magnets (painted wooden blocks)
using their hands to see how the virtual magnetic fields
change. In [102], users can tour a room converted intoa virtual museum wherein the artefacts on display are

16

virtual. Similar to how users would behave in an actual
museum, they can inspect a virtual object by peeringover different angles with respect to a fixed virtual
object.
TABLE 9
Examples of Vision-haptic Visualization

7

Ref.

Content

Embodied Interaction

[14]

Human
anatomyUsers can move around to see hisinternal organs[63]

Spatial ability
training

Users can rotate a virtual model displayed on his notebook[66]

Magnetic
concepts

field

Users can move around magnets
and see how the magnetic fieldchanges.

[82]

Chinese characters

Users can hold 3D virtual objects
representing the characters[56]

Elastic collision

Users can view two colliding balls at
different perspectives.

[33]

3D objects related to the lesson

Users are presented with 3D models
on cards they can manipulate by
hand.

[106]

Children’s story

Users are presented with a 3D popup book.

[58]

Solar system

Users are presented with heavenly
bodies on a book.

[79]

Digestive andcirculatory
systems

Users are presented with 3D virtual
models of internal organs associatedwith an AR marker.

[102]

Art and culturalobjects

Users can tour an AR museum.

[23]

Amino acids

Users are presented with 3D virtual
models of amino acids.

[32]

Geometry andspatial ability

Users can move around a virtual
geometrical shape.

[34]

Electronegativity
anddipolemomentUsers are presented with 3D virtual
models of molecules.

[94]

Solar system

Users are allowed to change the position of the Earth with respect tothe sun and observe the effect on the
Earth.

C ONCLUSION

Augmented Reality (AR) has unique affordances that canaffect the learning experience. Developments in AR technology have enabled researchers to develop and to evaluate Augmented Reality Learning Experiences (ARLEs).
These developments encompass hardware, software andthe authoring of content. Currently, ARLEs have a meaneffect size of 0.56 to student performance with wide
variability due to the many possible ways to use AR,
as well as, differences in experimental design.
In the course of the development of ARLEs, researchers must test their prototypes for benefits in thelearning process, and for usability. Such tests must use
sensible control groups, report an effect size, and use
expertly designed tools such as questionnaires. ARLEs
have been evaluated through student performance tests,
survey questionnaires, interviews, observations of user

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

behavior, and expert reviews. Currently, there is a needfor valid and reliable questionnaires to measure constructs related to ARLEs to iteratively improve ARLE
design.
A review of existing ARLE prototypes led us to theconclusion that there are three inherent affordances of
AR to educational settings namely: real world annotation, contextual visualization, and vision-haptic visualization. Furthermore, researchers have used design
strategies such as enabling exploration, promoting collaboration, and ensuring immersion to create compelling
learning experiences. Depending on the learning objective, these affordances and strategies can be employed
to create successful ARLEs.
These three affordances are supported by existing
theories namely multimedia learning theory, experiential
learning theory and animate vision theory. Essentially,
AR affords interactions with information that may help
us perceive and remember information based on multimedia learning theory and animate vision theory. Realworld annotation may reduce cognitive load significantly, and vision-haptic visualization allows embodied
interactions enabling more natural ways of acquiring
information. These highly cognitive approaches to understanding AR technology in educational settings arecomplemented by experiential learning theory and contextual learning which treats the whole experience ofthe students as the source of learning. Adapting AR
technology changes the learning experience and may
afford new compelling experiences that lead to better
learning.
Researchers, teachers and students are already collaborating towards a more participatory design of ARLEs.
In the future, we believe ARLEs will be created by moreinterdisciplinary groups and will be grounded in theory.
Future work in the area of ARLE would be empiricallyproving (or disproving) the principles of multimedia
learning theory for AR annotation, ARLEs and AR ingeneral. Basic research should also be done in exploring
contextual visualization and vision-haptic visualization.
This includes, but is not limited to experiments measuring learning with varying degrees of contextual cues,
and when using varied, embodied interactions.

R EFERENCES
[1]
[2]
[3]
[4]
[5]
[6]

Advancedlearning
technologies
journallist.
http://celstec.org/content/advanced-learning-technologiesjournal-list. Accessed: August 2012.
Intrinsic
motivation
inventory(imi).
http://www.selfdeterminationtheory.org/questionnaires/10questionnaires/50. Accessed: August 2012.
What is contextual learning? http://www.cord.org/contextuallearning-definition/. Accessed: October 2012.
Zooburst: create your own interactive 3d pop-up books.
http://www.zooburst.com/. Accessed: October 2012.
Maryam Alavi. Computer-mediated collaborative learning: An
empirical evaluation. MIS quarterly, 18(2):159–174, 1994.
Leonard Annetta and Stephen Bronack. Serious Educational Game
Assessment: Practical Methods and Models for Educational Games,
Simulations and Virtual Worlds. Amsterdam, The Netherlands.
Sense Publishers, 2011.

[7]

[8]

[9]
[10]
[11]

[12]
[13]

[14]

[15]

[16]

[17]
[18]

[19]

[20]
[21]

[22]
[23]

[24]

[25]
[26]

17

Theodoros N Arvanitis, Argeroula Petrou, James F Knight,
Stavros Savas, Sofoklis Sotiriou, Michael Gargalakos, and Elpida
Gialouri. Human factors and qualitative pedagogical evaluationof a mobile augmented reality system for science education usedby learners with physical disabilities. Personal and ubiquitous
computing, 13(3):243–250, 2009.
K. Asai, H. Kobayashi, and T. Kondo. Augmented instructions- a fusion of augmented reality and printed learning materials.
In Advanced Learning Technologies, 2005. ICALT 2005. Fifth IEEE
International Conference on, pages 213 – 215, july 2005.
Ronald T. Azuma. A survey of augmented reality. PresenceTeleoperators and Virtual Environments, 6(4):355–385, 1997.
Dana H Ballard. Animate vision. Artificial intelligence, 48(1):57–
86, 1991.
Sylvie Barma and Sylvie Daniel. Mind your game, game yourmind! mobile gaming for co-constructing knowledge. In World
Conference on Educational Multimedia, Hypermedia and Telecommunications, volume 2011, pages 324–334, 2011.
M. Billinghurst and A. Duenser. Augmented reality in theclassroom. Computer, 45(7):56 –63, july 2012.
Joe Blalock and Jacob Carringer. Augmented reality applications
for environmental designers. In World Conference on Educational
Multimedia, Hypermedia and Telecommunications, volume 2006,
pages 2757–2762, 2006.
T. Blum, V. Kleeberger, C. Bichlmeier, and N. Navab. mirracle:
An augmented reality magic mirror system for anatomy education. In Virtual Reality Short Papers and Posters (VRW), 2012 IEEE,
pages 115 –116, march 2012.
Jennifer M Brill and Yeonjeong Park. Facilitating engaged
learning in the interaction age taking a pedagogically-disciplined
approach to innovation with emergent technologies. International
Journal of Teaching and Learning in Higher Education, 20(1):70–78,
2008.
Su Cai and Qian Song. Ar-based remote video learning system. In Wireless, Mobile and Ubiquitous Technology in Education(WMUTE), 2012 IEEE Seventh International Conference on, pages
322 –324, march 2012.
J.C.P. Chan, H. Leung, J.K.T. Tang, and T. Komura. A virtual
reality dance training system using motion capture technology.
Learning Technologies, IEEE Transactions on, 4(2):187–195, 2011.
George Chang, Patricia Morreale, and Padmavathi Medicherla.
Applications of augmented reality systems in education. In
Society for Information Technology & Teacher Education International
Conference, volume 2010, pages 1380–1385, 2010.
W. Chang, Qing Tan, and Fang Wei Tao. Multi-object oriented
augmented reality for location-based adaptive mobile learning.
In Advanced Learning Technologies (ICALT), 2010 IEEE 10th International Conference on, pages 450 –451, july 2010.
Wen-Chih Chang, Te-Hua Wang, F.H. Lin, and Hsuan-Che Yang.
Game-based learning with ubiquitous technologies. Internet
Computing, IEEE, 13(4):26–33, 2009.
Yuan-Jen Chang, Chin-Hsing Chen, Wen-Tzeng Huang, and WeiShiun Huang. Investigating students’ perceived satisfaction,
behavioral intention, and effectiveness of english learning using
augmented reality. In Multimedia and Expo (ICME), 2011 IEEE
International Conference on, pages 1 –6, july 2011.
Chih-Ming Chen and Yen-Nung Tsai. Interactive augmented
reality system for enhancing library instruction in elementary
schools. Computers & Education, 2012.
Yu-Chien Chen. A study of comparing the use of augmented
reality and physical models in chemistry education. In Proceedings of the 2006 ACM international conference on Virtual reality
continuum and its applications, VRCIA ’06, pages 369–372, New
York, NY, USA, 2006. ACM.
Christopher Coffin, Svetlin Bostandjiev, James Ford, and Tobias
Hollerer. Enhancing classroom and distance learning throughaugmented reality. In World Conference on Educational Multimedia,
Hypermedia and Telecommunications, volume 2010, pages 1140–
1147, 2010.
Learning Technology Standards Comittee. Draft standard forlearning object metadata. PDF, 2002.
A.G.D. Correa, I.K. Ficheman, M. do Nascimento, andR. de Deus Lopes. Computer assisted music therapy: A casestudy of an augmented reality musical system for children withcerebral palsy rehabilitation. In Advanced Learning Technologies,
2009. ICALT 2009. Ninth IEEE International Conference on, pages
218 –220, july 2009.

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

[27]
[28]

[29]

[30]
[31]
[32]

[33]
[34]

[35]

[36]
[37]
[38]

[39]

[40]

[41]
[42]

[43]

[44]
[45]

[46]

Michael L Crawford. Teaching contextually: Research, rationale,
and techniques for improving student motivation and achievement in mathematics and science. Texas: CORD, 2001.
M. Davidsson, D. Johansson, and K. Lindwall. Exploring the useof augmented reality to support science education in secondary
schools. In Wireless, Mobile and Ubiquitous Technology in Education(WMUTE), 2012 IEEE Seventh International Conference on, pages
218 –220, march 2012.
Fred D Davis and Viswanath Venkatesh. A critical assessment of potential measurement biases in the technology acceptance model: three experiments. International Journal of HumanComputer Studies, 45(1):19–45, 1996.
Chris Dede. The evolution of distance education: Emerging technologies and distributed learning. American Journal of Distance
Education, 10(2):4–36, 1996.
Ángela Di Serio, Marı́a Blanca Ibáñez, and Carlos Delgado Kloos.
Impact of an augmented reality system on students motivation
for a visual art course. Computers & Education, 2012.
Andreas Dünser, Karin Steinbügl, Hannes Kaufmann, and JudithGlück. Virtual and augmented reality as spatial ability training
tools. In Proceedings of the 7th ACM SIGCHI New Zealandchapter’s international conference on Computer-human interaction:
design centered HCI, CHINZ ’06, pages 125–132, New York, NY,
USA, 2006. ACM.
Neven AM El Sayed, Hala H Zayed, and Mohamed I Sharawy.
Arsc: Augmented reality student card. Computers & Education,
56(4):1045–1061, 2011.
M. Fjeld, D. Hobi, L. Winterthaler, B. Voegtli, and P. Juchli.
Teaching electronegativity and dipole moment in a tui. In Advanced Learning Technologies, 2004. Proceedings. IEEE International
Conference on, pages 792 – 794, aug.-1 sept. 2004.
Yuichiro Fujimoto, Goshiro Yamamoto, Hirokazu Kato, and Jun
Miyazaki. Relation between location of information displayedby augmented reality and user’s memorization. In Proceedings ofthe 3rd Augmented Human International Conference, AH ’12, pages
7:1–7:8, New York, NY, USA, 2012. ACM.
Jian Gu, Nai Li, and H.B.-L. Duh. A remote mobile collaborative
ar system for learning in physics. In Virtual Reality Conference(VR), 2011 IEEE, pages 257 –258, march 2011.
Kuei-Fang Hsiao. Using augmented reality for students healthcase of combining educational learning with standard fitness.
Multimedia Tools and Applications, pages 1–15, 2012.
Kuei-Fang Hsiao and Habib F Rashvand. Body language andaugmented reality learning environment. In Multimedia and Ubiquitous Engineering (MUE), 2011 5th FTRA International Conferenceon, pages 246–250. IEEE, 2011.
Feng Huang, Yu Zhou, Yao Yu, Ziqiang Wang, and Sidan Du.
Piano ar: A markerless augmented reality based piano teachingsystem. In Intelligent Human-Machine Systems and Cybernetics(IHMSC), 2011 International Conference on, volume 2, pages 47–52, aug. 2011.
T. Iwata, T. Yamabe, and T. Nakajima. Augmented reality go:
Extending traditional game play with interactive self-learning
support. In Embedded and Real-Time Computing Systems andApplications (RTCSA), 2011 IEEE 17th International Conference on,volume 1, pages 105 –114, aug. 2011.
Hyung-Keun Jee, Sukhyun Lim, Jinyoung Youn, and JunsukLee. An augmented reality-based authoring tool for e-learning
applications. Multimedia Tools and Applications, pages 1–11, 2011.
Hyung-Keun Jee, Sukhyun Lim, JinYoung Youn, and JunsukLee. An immersive authoring tool for augmented reality-basede-learning applications. In Information Science and Applications(ICISA), 2011 International Conference on, pages 1 –5, april 2011.
T.F.L. Jerry and C.C.E. Aaron. The impact of augmented reality
software with inquiry-based learning on students’ learning of
kinematics graph. In Education Technology and Computer (ICETC),
2010 2nd International Conference on, volume 2, pages V2–1 –V2–5,
june 2010.
Peter Johnson. Human computer interaction: psychology, task analysis, and software engineering. McGraw-Hill, 1992.
C. Juan, F. Beatrice, and J. Cano. An augmented reality system
for learning the interior of the human body. In Advanced Learning
Technologies, 2008. ICALT ’08. Eighth IEEE International Conferenceon, pages 186 –188, july 2008.
C.M. Juan, E. Llop, F. Abad, and J. Lluch. Learning words using
augmented reality. In Advanced Learning Technologies (ICALT),

[47]

[48]
[49]

[50]
[51]

[52]
[53]
[54]

[55]
[56]

[57]

[58]
[59]

[60]

[61]
[62]

[63]

[64]

[65]

[66]

18

2010 IEEE 10th International Conference on, pages 422 –426, july
2010.
C.M. Juan, G. Toffetti, F. Abad, and J. Cano. Tangible cubes
used as the user interface in an augmented reality game foredutainment. In Advanced Learning Technologies (ICALT), 2010
IEEE 10th International Conference on, pages 599 –603, july 2010.
H. Kaufmann and D. Schmalstieg. Designing immersive virtual
reality for geometry education. In Virtual Reality Conference, 2006,
pages 51 – 58, march 2006.
J. Keil, M. Zollner, M. Becker, F. Wientapper, T. Engelke, andH. Wuest. The house of olbrich - an augmented reality tour
through architectural history. In Mixed and Augmented Reality Arts, Media, and Humanities (ISMAR-AMH), 2011 IEEE International Symposium On, pages 15 –18, oct. 2011.
John M Keller. Development and use of the arcs model of
instructional design. Journal of instructional development, 10(3):2–
10, 1987.
Lucinda Kerawalla, Rosemary Luckin, Simon Seljeflot, andAdrian Woolard. making it real: exploring the potential ofaugmented reality for teaching primary school science. Virtual
Reality, 10(3):163–174, 2006.
Eric Klopfer and Josh Sheldon. Augmenting your own reality:
Student authoring of science-based augmented reality games.
New directions for youth development, 2010(128):85–94, 2010.
David A. Kolb. Experiential learning: experience as the source oflearning and development. Prentice Hall, 1984.
F. Kusunoki, M. Sugimoto, and H. Hashizume. Symphony-q: a
support system for learning music in collaborative learning. In
Systems, Man and Cybernetics, 2002 IEEE International Conferenceon, volume 4, page 6 pp. vol.4, oct. 2002.
Kangdon Lee. Augmented reality in education and training.
TechTrends, 56(2):13–21, 2012.
Nai Li, L. Chang, Yuan Xun Gu, and H.B.-L. Duh. Influencesof ar-supported simulation on learning effectiveness in face-toface collaborative learning for physics. In Advanced Learning
Technologies (ICALT), 2011 11th IEEE International Conference on,
pages 320 –322, july 2011.
Yiqun Li, Aiyuan Guo, J.A. Lee, Yan Gao, and Yii Leong Ling.
Visual interactive and location activated mobile learning. In
Wireless, Mobile and Ubiquitous Technology in Education (WMUTE),
2012 IEEE Seventh International Conference on, pages 235 –237,
march 2012.
T-Y Liu. A context-aware ubiquitous learning environment for
language listening and speaking. Journal of Computer Assisted
Learning, 25(6):515–527, 2009.
Tsung-Yu Liu, Tan-Hsu Tan, and Yu-Ling Chu. 2d barcodeand augmented reality supported english learning system. In
Computer and Information Science, 2007. ICIS 2007. 6th IEEE/ACISInternational Conference on, pages 5 –10, july 2007.
Wei Liu, Adrian David Cheok, Charissa Lim Mei-Ling, and YinLeng Theng. Mixed reality classroom: learning from entertainment. In Proceedings of the 2nd international conference on Digitalinteractive media in entertainment and arts, pages 65–72. ACM,
2007.
Dorit Maor. A teacher professional development program on using a constructivist multimedia learning environment. Learning
Environments Research, 2(3):307–330, 1999.
G. Margetis, P. Koutlemanis, X. Zabulis, M. Antona, andC. Stephanidis. A smart environment for augmented learning
through physical books. In Multimedia and Expo (ICME), 2011
IEEE International Conference on, pages 1 –6, july 2011.
Jorge Martı́n-Gutiérrez, José Luı́s Saorı́n, Manuel Contero, Mariano Alcañiz, David C Pérez-López, and Mario Ortega. Designand validation of an augmented book for spatial abilities development in engineering students. Computers & Graphics, 34(1):77–
91, 2010.
James Mathews. Sick at south shore beach: A place-based
augmented reality game as a framework for building academic
language in science. In World Conference on Educational Multimedia, Hypermedia and Telecommunications, volume 2008, pages
2992–2997, 2008.
Carla Mathison and Kitty Gabriel. Designing augmented reality
experiences in authentic learning environments. In Society for
Information Technology & Teacher Education International Conference,
volume 2012, pages 3470–3473, 2012.
S. Matsutomo, T. Miyauchi, S. Noguchi, and H. Yamashita. Realtime visualization system of magnetic field utilizing augmented

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

[67]
[68]
[69]

[70]

[71]

[72]
[73]
[74]
[75]
[76]

[77]

[78]

[79]

[80]

[81]

[82]

[83]
[84]

[85]
[86]

reality technology for education. Magnetics, IEEE Transactions on,
48(2):531 –534, feb. 2012.
Richard E. Mayer. Multimedia Learning. Cambridge University
Press, 2009.
Paul Milgram and Fumio Kishino. A taxonomy of mixed reality
visual displays. IEICE TRANSACTIONS on Information and
Systems, 77(12):1321–1329, 1994.
Hiroyuki Mitsuhara, Y Yano, and T Moriyama. Paper-top interface for supporting note-taking and its preliminary experiment.
In Systems Man and Cybernetics (SMC), 2010 IEEE International
Conference on, pages 3456–3462. IEEE, 2010.
Y. Motokawa and H. Saito. Support system for guitar playing
using augmented reality display. In Mixed and Augmented Reality,
2006. ISMAR 2006. IEEE/ACM International Symposium on, pages
243 –244, oct. 2006.
H. Nakasugi and Y. Yamauchi. Past viewer: Development ofwearable learning system for history education. In Computersin Education, 2002. Proceedings. International Conference on, pages
1311 – 1312 vol.2, dec. 2002.
] J. Nielsen and R. L. Mack. Usability Inspection Methods. Wiley& Sons, Inc. New York, 1994.
S. Odeh, S.A. Shanab, M. Anabtawi, and R. Hodrob. Remoteaugmented reality engineering labs. In Global Engineering Education Conference (EDUCON), 2012 IEEE, pages 1 –6, april 2012.
S. Odeh, S.A. Shanab, M. Anabtawi, and R. Hodrob. Remoteaugmented reality engineering labs. In Global Engineering Education Conference (EDUCON), 2012 IEEE, pages 1 –6, april 2012.
H. Okada and H. Arakawa. Application and evaluation ofaugmented reality user interface to a card game x201d;. In SICE
Annual Conference, 2008, pages 3237–3240, 2008.
Patrick O’Shea, Chris Dede, and Rebecca Mitchell. A study ofthe efficacy of an augmented reality curriculum. In Society for
Information Technology & Teacher Education International Conference,
volume 2009, pages 1510–1514, 2009.
Patrick O’Shea, Kitty Gabriel, and Vasundhara Prabhu. The
crane: Creating, implementing and evaluating an augmented
reality art curriculum. In Society for Information Technology &Teacher Education International Conference, volume 2010, pages
2013–2019, 2010.
F. Pais, S. Vasconcelos, S. Capitao, V. Carlos, and M. Almeida.
Mobile learning and augmented reality. In Information Systems
and Technologies (CISTI), 2011 6th Iberian Conference on, pages 1–4, june 2011.
D. Perez-Lopez, M. Contero, and M. Alcaiz. Collaborative development of an augmented reality application for digestive andcirculatory systems teaching. In Advanced Learning Technologies(ICALT), 2010 IEEE 10th International Conference on, pages 173–175, july 2010.
Erin Peters Burton, Wendy Frazier, Leonard Annetta, Richard
Lamb, Rebecca Cheng, and Margaret Chmiel. Modeling augmented reality games with preservice elementary and secondary
science teachers. Journal of Technology and Teacher Education,
19(3):303–329, 2011.
Jochen Prümper and Michael Anft. Die evaluation von software
auf grundlage des entwurfs zur internationalen ergonomie-norm
iso 9241 teil 10 als beitrag zur partizipativen systemgestaltungein fallbeispiel. In Software-Ergonomie, volume 93, pages 145–156,
1993.
Li Qi-ming, Chen Yi-min, Ma De-yi, Huang Chen, Xu Sheng,
Lu Ren-miao, Liu Yan, and Wang Xi-chen. Design and implementation of a chinese character teaching system based on augmented reality interaction technology. In Computer Science andAutomation Engineering (CSAE), 2011 IEEE International Conferenceon, volume 2, pages 322 –326, june 2011.
W. Queensberry.
Using the 5es to understand users.
http://wqusability.com/articles/getting-started.html. Accessed:
August 2012.
A.S.M.M. Rahman, Jongeun Cha, and A. El Saddik. Authoring edutainment content through video annotations and 3dmodel augmentation. In Virtual Environments, Human-Computer
Interfaces and Measurements Systems, 2009. VECIMS ’09. IEEE
International Conference on, pages 370 –374, may 2009.
Stephen K Reed. Cognition: Theory and applications. Wadsworth
Publishing Company, 2007.
Gerhard Reitmayr and Dieter Schmalstieg. Collaborative augmented reality for outdoor navigation and information brows-

19

ing. In Proc. Symposium Location Based Services and TeleCartography, pages 31–41, 2004.
[87] R.A. Roberto and V. Teichrieb. Arblocks: A projective augmented
reality platform for educational activities. In Virtual Reality Short
Papers and Posters (VRW), 2012 IEEE, pages 175 –176, march 2012.
[88] P. Salamin, T. Tadi, O. Blanke, F. Vexo, and D. Thalmann.
Quantifying effects of exposure to the third and first-person perspectives in virtual-reality-based training. Learning Technologies,
IEEE Transactions on, 3(3):272–276, 2010.
[89] M. E. C. Santos, G. Yamamoto, T. Taketomi, J. Miyazaki, and
H. Kato. Augmented reality x-ray interaction in k-12 education.
In Advanced Learning Technologies, 2013. ICALT 2013. 13th IEEE
International Conference on, pages 141 – 145, july 2013.
[90] M. E. C. Santos, G. Yamamoto, T. Taketomi, J. Miyazaki, and
H. Kato. Authoring augmented reality learning experiences aslearning objects. In Advanced Learning Technologies, 2013. ICALT2013. 13th IEEE International Conference on, pages 506 – 507, july
2013.
[91] Richard F Schmid, Robert M Bernard, Eugene Borokhovski, Rana
Tamim, Philip C Abrami, C Anne Wade, Michael A Surkes,
and Gretchen Lowerison. Technologys effect on achievement inhigher education: a stage i meta-analysis of classroom applications. Journal of computing in higher education, 21(2):95–109, 2009.
[92] B. Schneider, P. Jermann, G. Zufferey, and P. Dillenbourg. Benefits of a tangible interface for collaborative learning and interaction. Learning Technologies, IEEE Transactions on, 4(3):222–232,
2011.
[93] Gerhard Schwabe and Christoph Göth. Mobile learning with amobile game: design and motivational effects. Journal of Computer
Assisted Learning, 21(3):204–216, 2005.
[94] B.E. Shelton and N.R. Hedley. Using augmented reality forteaching earth-sun relationships to undergraduate geography
students. In Augmented Reality Toolkit, The First IEEE International
Workshop, page 8, 2002.
[95] Brett E Shelton and Nicholas R Hedley. Exploring a cognitive
basis for learning spatial relationships with augmented reality.
Technology, Instruction, Cognition and Learning, 1(4):323, 2004.
[96] L. Simeone and S. Iaconesi. Toys++ ar embodied agents as tools
to learn by building. In Advanced Learning Technologies (ICALT),2010 IEEE 10th International Conference on, pages 649 –650, july
2010.
[97] L. Simeone and S. Iaconesi. Anthropological conversations:
Augmented reality enhanced artifacts to foster education in
cultural anthropology. In Advanced Learning Technologies (ICALT),2011 11th IEEE International Conference on, pages 126 –128, july
2011.
[98] Aw Kien Sin and H.B. Zaman. Live solar system (lss): Evaluation of an augmented reality book-based educational tool. In
Information Technology (ITSim), 2010 International Symposium in,volume 1, pages 1 –6, june 2010.
[99] S. Sotiriou, S. Anastopoulou, S. Rosenfeld, O. Aharoni, A. Hofstein, F. Bogner, H. Sturm, and K. Hoeksema. Visualizing theinvisible: The connect approach for teaching science. In Advanced
Learning Technologies, 2006. Sixth International Conference on, pages
1084 –1086, july 2006.
[100] S.P. Stenton, R. Hull, P.M. Goddi, J.E. Reid, B.J. Clayton, T.J.
Melamed, and S. Wee. Mediascapes: Context-aware multimedia
experiences. MultiMedia, IEEE, 14(3):98 –105, july-sept. 2007.
[101] D.D. Sumadio and D.R.A. Rambli. Preliminary evaluation onuser acceptance of the augmented reality use for education.
In Computer Engineering and Applications (ICCEA), 2010 Second
International Conference on, volume 2, pages 461 –465, march 2010.
[102] Stella Sylaiou, Katerina Mania, Athanasis Karoulis, and Martin
White. Exploring the relationship between presence and enjoyment in a virtual museum. International journal of human-computerstudies, 68(5):243–253, 2010.
[103] Rana M Tamim, Robert M Bernard, Eugene Borokhovski,
Philip C Abrami, and Richard F Schmid. What forty yearsof research says about the impact of technology on learninga second-order meta-analysis and validation study. Review of
Educational Research, 81(1):4–28, 2011.
[104] Wernhuar Tarng and Kuo-Liang Ou. A study of campus butterfly
ecology learning system based on augmented reality and mobile
learning. In Wireless, Mobile and Ubiquitous Technology in Education(WMUTE), 2012 IEEE Seventh International Conference on, pages
62 –66, march 2012.

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 0, NO. 0, MONTH 2000

[105] Thomas Tullis and William Albert. Measuring the user experience:collecting, analyzing, and presenting usability metrics. Morgan
Kaufmann, 2008.
[106] P. Vate-U-Lan. Augmented reality 3d pop-up children book:
Instructional design for hybrid learning. In e-Learning in Industrial Electronics (ICELIE), 2011 5th IEEE International Conferenceon, pages 95 –100, nov. 2011.
[107] Yuan Wang, Tobias Langlotz, Mark Billinghurst, and Tim Bell.
An authoring tool for mobile phone ar environments. In Proceedings of New Zealand Computer Science Research Student Conference,
volume 9, pages 1–4. Citeseer, 2009.
[108] T. Yamabe, H. Asuma, S. Kiyono, and T. Nakajima. Feedback
design in augmented musical instruments: A case study with anar drum kit. In Embedded and Real-Time Computing Systems andApplications (RTCSA), 2011 IEEE 17th International Conference on,volume 2, pages 126 –129, aug. 2011.
[109] Yang Yang, H. Leung, LiHua Yue, and LiQun Deng. Automaticdance lesson generation. Learning Technologies, IEEE Transactions
on, 5(3):191–198, 2012.
[110] N.M.M. Zainuddin, H.B. Zaman, and A. Ahmad. A participatory
design in developing prototype an augmented reality book fordeaf students. In Computer Research and Development, 2010 Second
International Conference on, pages 400 –404, may 2010.

Marc Ericson Santos is a doctoral student at
the Nara Institute of Science and Technology(NAIST), Japan. He received his B.S. degree
in electronics and communications engineering, and M.S. degree in electronics engineeringfrom Ateneo de Manila University (Philippines) in2009 and 2011, respectively. Under the Monbukagakusho scholarship, he studies augmented
reality applications for educational settings. His
main research interests are in educational technology and usability engineering.

Angie Chen is a master’s student at the Nara
Institute of Science and Technology (NAIST),
Japan. She received her B.E. from the National
Taipei University, Taiwan in 2010. She joined
the Mixed Reality Laboratory in the Interactive &Digital Media Institute of the National Universityof Singapore from May to September 2010 as
as an intern, and continued her stay as research
assistant until January 2012.

Takafumi Taketomi received his B.E. degreefrom the National Institute for Academic Degrees
and University Evaluation in 2006. He received
his M.E. and Dr. Eng. degrees in information
science from Nara Institute of Science and Technology in 2008 and 2011, respectively. He has
been an assistant professor at Nara Institute of
Science and Technology since 2011.

20

Goshiro Yamamoto is an assistant professor
at the Nara Institute of Science and Technology(NAIST), Japan. He received his B.E., M.E., andDr. Eng. degrees from Osaka University in 2004,
2006, and 2009, respectively. He was a JSPS
Research Fellow from 2008 to 2009. In 2009, hejoined the Graduate School of Natural Scienceand Technology at the Okayama University. He
was a visiting research professor in the Department of Information Processing Science at the
University of Oulu, Finland in 2012. His major interests are in human-computer interaction, projection-based augmented
reality, and wearable computing systems.

Jun Miyazaki joined the Graduate School ofInformation Science and Engineering, Tokyo Institute of Technology, Japan, in 2013, where heis currently a professor. He received his B.E. degree from Tokyo Institute of Technology in 1992,
and both M.S. degree and Ph.D. from Japan
Advanced Institute of Science and Technology(JAIST) in 1994, 1997, respectively. From 2000
to 2001, he was a visiting scholar at Departmentof Computer Science and Engineering, University of Texas at Arlington. He worked as an associate professor of the Graduate School of Information Science, Nara
Institute of Science and Technology (NAIST) from 2003 to 2013. Since
2003, he was also a researcher of the PRESTO program of Japan Science and Technology Agency until 2008. His research interests include
database systems, information retrieval, information recommendation,and user-computer interface. He is a member of IEEE-CS, ACM, IEICE,IPSJ, DBSJ, and HIS.

Hirokazu Kato is a professor in the Nara Institute of Science and Technology (NAIST), Japan.
He received his B.E., M.E. and Dr. Eng. degrees
from Osaka University, Japan in 1986, 1988
and 1996, respectively. He joined the Faculty ofEngineering Science at Osaka University, from1989 to 1999. In 1999, he joined the Facultyof Information Sciences at Hiroshima City University, Japan. In 2003, he joined the Graduate
School of Engineering Science at Osaka University. Since 2007, he has been with the Graduate
School of Information Science of NAIST. He studied pattern recognitionand computer vision. When he joined the Human Interface Technology
Laboratory (HIT Lab) at the University of Washington as a visiting
scholar in 1998, he started a research on Augmented Reality which
is currently his main research topic. He received the Virtual Reality
Technical Achievement Award from IEEE VGTC in 2009, and Lasting
Impact Award at the 11th IEEE International Symposium on Mixed andAugmented Reality in 2012.