International Conference on Artificial Reality and TelexistenceEurographics Symposium on Virtual Environments (2016)
D. Reiners, D. Iwai, and F. Steinicke (Editors)

Video Guides on Head-Mounted Displays:
The Effect of Misalignments on Manual Task Performance
Shalika Pathirathna1 , Christian Sandor1 , Takafumi Taketomi1 , Alexander Plopski1 and Hirokazu Kato1
1 Nara Institute of Science and Technology, Japan

Figure 1: (a) A participant of our experiment (re-enacted) performing a lego placement task following a video guide on his Video SeeThrough Head-Mounted Display (VST-HMD). (b) We investigate the impact of two different misalignments: First, between the HMD’s andthe video guide’s view-plane normals (WM); Second, between the HMD’s and the video guide’s up vectors (SM). (c), (d): Example of aparticipant’s view from our experiment; (c): W M = 0◦ , SM = 0◦ , (d): W M = 45◦ , SM = 90◦ .
Abstract
Head-Mounted Displays (HMD) are becoming widely available at affordable prices. A typical application of HMDs is supporting users when perform manual tasks. However, displaying information on a HMD without any registration with the environment
lead to misalignments and reduced task performance. The goal of our investigations is to quantify how increased misalignments
impact the performance. We conducted two experiments and we found that task completion time is significantly affected byWorld space Misalignment and a significant correlation of the absolute difference between World space Misalignment andScreen space Misalignment to the task completion time.
Categories and Subject Descriptors (according to ACM CCS): H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems—Video H.5.2 [Information Interfaces and Presentation]: User Interfaces—Training, help

1. Introduction
Video See-Through Head-Mounted Displays (VST-HMDs), e.g.,
Canon MREAL HMD, and Optical See-through HMDs (OSTHMDs), e.g., Google Glass, are becoming widely available at affordable prices. A common application of such HMDs is manualtask support. There are two fundamental ways of displaying instructions on HMDs; registered to the real environment, following
the Augmented Reality (AR) paradigm or unregistered, following
the Wearable Computing paradigm. A previous study [RMW08]
found that registered AR performs better than unregistered graphics. However, AR does not perform better when the content is notproperly aligned with the environment. Therefore, the system mustbe accurately calibrated with the environment to obtain the desired
performance. This is often a very complex, expensive and time consuming process. Instead, graphical instructions such as video andc 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.

pictures can be displayed on the HMD without any registration andtracking, which is much faster and easier to accomplish.
Video guides are one of the most common ways to present instructions for manual task support. In addition, a large number of
2D video guides already exists and are freely available in variousforms. These existing video guides can be displayed on an HMD
without further preparation or registration to the environment, thus
greatly reducing the required preparation time. However, unregistered techniques might present the information from a different perspective than the angle the user is looking from and create a misalignment. This misalignment can manifest in two ways (Figure1b). First, between the HMD’s and the video guide’s up vectors(SM: Screen space Misalignment); Second, between the HMD’s
and the video guide’s view-plane normals (WM: World space Misalignment). Because of these misalignments, users will have toShalika Pathirathna, Christian Sandor, Takafumi Taketomi, Alexander Plopski & Hirokazu Kato / Video Guides on Head-Mounted Displays

mentally rotate the guide to align the video instructions with theirworkspace. This results in reduced task performance [SM71].
In this work, we evaluate the effect of misalignment of the videoguides using HMDs on manual tasks. We conducted two experiments to identify how task completion time is affected by the misalignment of the video guide. We found a significant effect of WM
on task completion time and a significant correlation of the absolutedifference between WM and SM to the task completion time.
2. Related Work
Previous studies of misalignments can be divided into three groups,
human perception of misalignment, misalignment evaluation andmisalignment elimination.
Human Perception of Misalignment Shepard and Metzler[SM71] (1971) conducted an experiment to identify the effect ofmental rotation of three-dimensional (3D) objects. They found thatreaction time increases linearly with the rotation angle.
Misalignment Evaluation Stephen R. Ellis et al. [ETKS91] conducted two experiments where they examined the 3D pursuit tracking when operators of teleoperation simulations are faced with misalignment between the display and control frames of reference.
They combine both human perception and action to evaluate theeffect of misalignment. The results show that the error increases toa maximum at about 125◦ then, decreases to 180◦ .
Misalignment Elimination Macedo et al. [MKE∗ 98], proposed anautomated compensation method to reduce misalignments between
operator of teleoperation and controller orientation. Results show asignificant improvements in operator performance with automaticcompensation under some conditions. Goto et al. [GUS∗ 10] proposed an AR based task support system that uses instructional
videos. The proposed system transforms instructional videos according to the user’s view and overlays them onto the user’s viewto eliminate the misalignment. Their evaluation results confirmedthat displaying the instructional video according to the user’s viewis effective for improving the user’s understanding of the instructions. However, there is no previous work that quantifies the effectsof misaligned video guides on HMDs.
3. Experiments
We conducted two experiments to identify the effects of misalignments on users. In both experiments, users were asked to place
Lego blocks according to a video instructions displayed on a Canon
VH2002 VST-HMD. In the first experiment we evaluate the effectof WM on task completion time. Twelve voluntary participants (9male, 3 female) with ages ranging 23 to 43 years (mean=28.83,sd=5.27) took part in the experiment. In the second experiment weevaluate the effect of both WM and SM on task completion time.
Fifteen participants (10 male, 5 female) with ages ranging 23 to 44years (mean=29.53, sd=5.01) were took part in the experiment.
In both experiments we displayed the video guide in a fixed position in the top right corner of the HMD view (Figure 1 c,d). The
video guide was rendered on the HMD with OpenGL and the instructions were displayed step by step. Whenever the user completed a step, the supervisor switched the guide to the next stepwith a key press. Both experiments were within group evaluations.
In the first experiment each participant participated in 12 trials andin the second experiment in 25 trials.

Figure 2: Experiment 2 mean task completion time for different
WM and SM. Whiskers represent ±standard error (SE).
Results Analysis of the results of Experiment 1 with two way repeated measures ANOVA showed that task completion time wassignificantly affected by WM [F(1,132)=6.6, p<0.001]. It increases
from WM 0◦ to 180◦ and decreases from WM 180◦ to 330◦ . Peaktask completion time is at WM 180◦ . This result supports the observations by Shepard and Metzler [SM71].
Analysis of results of Experiment 2 with two-way repeated measures ANOVA showed that task completion time was significantly
affected by WM and SM interaction [F(16,308)=8.413, p<0.001].
We observe that task completion time varies with the absolutedifference between WM and SM (Figure 2). Therefore, we analyzed |WM-SM| with one-way repeated measures ANOVA andthe task completion time was significantly affected by |WM-SM|[F(4,56)=21.72, p<0.001].
4. Conclusions
We have evaluated the impact of misalignments of video guides
with the user’s workspace on user performance. Our experiments
show a significant effect of WM and |WM-SM| on the task completion time. In the future we want to use the observations made in
our experiments to improve the presentation of video guides in support systems, e.g., by determining if correction of the video-guidealignment is required, or determining how existing guides can bewarped to best match the user’s perspective.
References[ETKS91] E LLIS S. R., T YLER M., K IM W. S., S TARK L.: Threedimensional tracking with misalignment between display and controlaxes. SAE Transactions: Journal of Aerospace 100, 1 (1991), 985–989.
2[GUS∗ 10] G OTO M., U EMATSU Y., S AITO H., S ENDA S., I KETANI
A.: Task support system by displaying instructional video onto AR
workspace. In Proceedings of the IEEE ISMAR (2010), pp. 83–90. 2[MKE∗ 98] M ACEDO J. A., K ABER D. B., E NDSLEY M. R., P OWANU SORN P., M YUNG S.: The Effect of Automated Compensation Axes onTeleoperator Performance for Incongruent. Human Factors 40, 4 (1998),541–553. 2[RMW08] ROBERTSON C. M., M ACLNTYRE B., WALKER B. N.: Anevaluation of graphical context when the graphics are outside of the task
area. In Proceedings of the IEEE ISMAR (2008), pp. 73–76. 1[SM71] S HEPARD R. N., M ETZLER J.: Mental rotation of threedimensional objects. Science 171, 3972 (1971), 701–703. 2c 2016 The Author(s)Eurographics Proceedings c 2016 The Eurographics Association.