Visual Effects of Turning Point and Travel
Direction for Outdoor Navigation using
Head-Mounted Display
Yuji Makimura1 , Aya Shiraiwa1 ,Masashi Nishiyama1,2 , and Yoshio Iwai1,2
1Graduate School of Sustainability Science, Tottori University
2
Graduate School of Engineering, Tottori University
3
Cross-informatics Research Center, Tottori University
101 Minami 4-chome, Koyama-cho, Tottori, 680-8550 Japannishiyama@tottori-u.ac.jp

Abstract. We investigate the visual effects of superimposing turning
points and travel directions within the user’s field of view in a navigation system using a subjective assessment procedure. Existing methods
were developed without conducting subjective assessments of the effectsof superimposing the turning points and travel directions on the user’s
display while walking outdoors. We therefore designed a questionnairebased subjective assessment of the use of these navigation methods. We
developed an outdoor navigation system using a recently launched optical see-through head-mounted display (HMD) product that was compact
and lightweight. We demonstrated that the subjective scores in terms ofunderstanding of the turning points and the travel directions were significantly increased by the visual effects of superimposing these cues on thedisplay. We confirmed that the HMD helps to increase user likeability ofuse of the navigation system while walking outdoors.
Keywords: Visual effect, Navigation, Superimposed image, Turning point,
Travel direction

1

Introduction

There is considerable demand for an outdoor navigation method that can guideusers intuitively while they walk. Conventional navigation methods [1] guideusers by providing a route on a map to allow the user to reach their destinationsmoothly. Before the user walks, he or she sets the destination at the starting
point and selects the route that is suggested by the navigation method. While
walking, the user checks for the required turning points on the map and searches
for landmarks in the real world. Figure 1 illustrates the typical process of thistype of navigation method. If the user understands and can determine the turning points in the real world, he or she can then determine the required travel
direction. Otherwise, the user must laboriously repeat the navigation process

II
Check turning point on map

Search for landmarks in real world

Understand turning point in real world?

No

Yes
Determine travel direction and move

Fig. 1. Overview of conventional navigation method.

until he or she correctly understands both the turning points and the travel directions. Note here that a turning point is a location at which the user changes
the travel direction to start the next stage of the journey, and the travel direction
is a unit vector that is directed from the current position to the next turning
point. A navigation method that leads to frequent repetition of the navigation
process cannot guide the user intuitively. Ease of understanding of both theturning points and the travel directions for the user is therefore very important.
Many researchers [2, 3] have attempted to design suitable navigation methods.
To reduce repetition in the navigation process, the existing methods [4, 5]
often generate either a rough map or a written announcement. These existing
methods can thus become a burden because they require the user to read the mapor the announcement at least once. To eliminate the repetition from this process,existing methods [6, 7, 8, 9, 10, 11] generally overlay the navigational information
on real world images. Narzt et al. [6] overlaid routes on images that were acquired
from a camera equipped in a mobile device. Mulloni et al. [7, 8, 9] developed
hand-held indoor navigation systems using mobile phones. Oliveira et al. [10]overlaid the travel directions on images using a process based on recognition ofmarkers using a mobile device. For their car navigation application, Narzt etal. [11] overlaid the route when using a head-up display for the car’s driver. We
believe that users can understand the routes intuitively, including the turning
points and the travel directions. However, holding up a mobile device or using ahead-up display while walking can prove to be very inconvenient for the user.
In this work, we discuss a method to superimpose the turning points andthe travel directions directly using an optical see-through head-mounted display(HMD). As described in [12, 13, 14, 15], an HMD helps the user to have intuitive understanding of the navigation process. In indoor navigation studies using
HMDs, Rehman et al. [16] superimposed the travel directions, while Gerstweiler
et al. [17] superimposed the route on the HMD. However, the existing methods
assumed the case of indoor use. In addition, the existing methods did not assess
whether or not the user understood the turning points and the travel directions
intuitively. Recently, compact and lightweight commercial optical see-through
HMDs have been launched. We are now able to test outdoor navigation withthese HMDs using subjective assessment processes.

III

Fig. 2. This study investigated whether or not the visual effects of superimposing
turning points and travel directions within the user’s field of view help the user’s
understanding of the directions given while walking outdoors.

In this paper, we investigate the visual effects that are used in our method
for outdoor navigation and demonstrate that they help the user to understandthe turning points and the travel directions intuitively while walking by superimposing these directions directly into the real world using the HMD. Note that
we consider the scenario in which the user is approaching a turning point while
walking. Figure 2 illustrates the overview of the outdoor navigation process. We
conducted a questionnaire-based subjective assessment of the navigation methods used with the HMD. The experimental results show that there was significant
agreement among the participants about the visual effects of superimposing boththe turning points and the travel directions. The rest of the paper is organized
as follows. Section 2 describes the user study protocol, while Section 3 and Section 4 present the results of the subjective assessment. Our concluding remarks
are given in Section 5.

2

Design of test for evaluation of the visual effects

2.1

Overview

We aimed to evaluate whether or not the visual effects of showing the turning
point and the travel direction aid the user’s understanding of these aspects. We
therefore developed a navigation method that superimposed both the turning
point and the travel direction in the user’s field of vision using the HMD. We
tested four possible methods, as follows:
M1 : The navigation method did not provide visual effects for the turning pointor the travel direction.
M2 : The navigation method provided visual effects for the turning point only.
M3 : The navigation method provided visual effects for the travel direction only.
M4 : The navigation method provided visual effects for both the turning point
and the travel direction.
We set a walking task for each participant using each navigation method.
After walking, we used a questionnaire to ask each participant about the different
navigation methods. We showed a printed map that included the starting point,

IV
15 m

S: Starting point
T: Turning point
D: Destination point

15 m

T

D

Building

S

Fig. 3. Illustration showing the starting point, the turning point, and the destination
point.

the turning point, and the destination point to each participant before they
began walking. While the user walked, our navigation method provided the same
voice guidance for each of methods M1 to M4 to inform the user of the timingof the turning point. The details of this evaluation are described below.
2.2

Walking task

We designed a walking task in which the participant moved from a starting point
to a destination point. In the general case, there is a polarity for the turning
points that are used for navigation. In our evaluation, we set the number ofturning points to be one to simplify the navigation issue. We used a crossroads as
the turning point. When standing at the starting point, the participant was able
to see the turning point, but was unable to see the destination point, which was
hidden behind a building. The distance from the starting point to the destination
point was 30 m. We set the turning point to be 15 m forward from the starting
point. Figure 3 shows the starting point, the turning point, and the destination
point. We prepared two combinations of the starting point, the turning point,
and the destination point. We used the combinations of these points at random
for each assessment.
2.3

Representation of the turning point and the travel direction

We used a three-dimensional virtual object composed of an arrow to represent
the turning point and/or the travel direction. The arrow object is frequently
used in navigation systems, as described in [18]. Use of the arrow object was
intended to ensure that the experimental conditions remained the same for eachof methods M1 to M4.
Figure 4 shows examples of the superimposed arrow objects used for eachnavigation method. In this figure, the angle of view shown in the camera images
is smaller than the actual angle of view of the participant. The arrow object had
identical dimensions of 28 × 10 × 34 cm in each navigation system. We adjusted
the height of the arrow object above the floor to suit each participant within therange of the vertical angle of view of the HMD. In method M2, we superimposed
the arrow object at the three-dimensional location of the turning point to inform

V
M2

M3

M4

Time

M1

Fig. 4. Navigation methods M1 to M4 were compared to investigate the visual effects ofsuperimposing the various combinations of the turning point and the travel direction.
The participants walked wearing the HMD while using each navigation method. In thisfigure, we have overlaid the arrow objects on images that were acquired from a camerathat was attached close to the eye level of the user.

the user of the turning direction. The navigation system varied the size of thearrow object based on the distance from the current position to the turning
point. In method M3, we superimposed the arrow object to inform the user of
the travel direction. We set the arrow object at a distance of 2 m in front of
the user’s current position. The navigation method changed the direction of thearrow when the user reached a distance of 1 m from the turning point. The
arrow turns stepwise by 45 degrees rather than turning by 90 degrees aroundthe turning point. We assumed that each walking human moves by 1 m over a
period of 0.75 s, as described in [19]. In method M4, we superimposed arrow
objects for both the turning point and the travel direction in the user’s fieldof view by combining methods M2 and M3. Note that the arrows of methods
M2 and M3 never crossed because the arrow in M3 turns stepwise around theturning point, as described above.

VI
Prior

HMD
Reconstruct
3D mapSet starting, turning anddestination positions

Estimate user’s positionand pose
Render virtual objects of turning
point and travel direction

Fig. 5. Architecture of proposed navigation method using the HMD. We used an HMD(HoloLens Development Edition, Microsoft). Our method superimposed the arrow objects by rendering them on the display based on the position and the pose of the user.

Fig. 6. Participant wearing the HMD.

2.4

Optical see-through HMD

Using the HMD, we developed navigation methods to superimpose the arrow
objects within the user’s field of view. Figure 5 illustrates the architecture of
our navigation method. We used an HMD (HoloLens Development Edition, Microsoft) that was equipped with an optical see-through display, an inertial measurement unit, cameras and a speaker. The HMD is capable of acquiring boththe position and the pose of the user in real time. Figure 6 shows a participant
wearing the HMD. Our method superimposed the arrow objects that represented
the turning point and the travel direction by rendering them on the display basedon the position and the pose of the user. We reconstructed a three-dimensional(3D) map to represent the surroundings of the road to be navigated in advanceof the experiments. Figure 7(a) shows the reconstructed 3D map. The HMD
renders the arrow on the 3D map shown in part (b) in the participant’s field ofview, which is illustrated in part (c). We also set the locations of the starting,turning, and destination points on this map in advance. We assumed that no
obstacles were present on the road during the period in which user was walking.

3

Questionnaire-based subjective assessment of turning
point and travel direction

3.1

Design of the subjective assessment

Sixteen participants (13 males and three females, with an average age of 22.3 ±1.6 years old) participated in the study. We used Scheffe’s paired comparisons
method [20] (Ura Variation [21]) in the assessment. We set various pairs of
the navigation methods 12 (= 4 C2 × 2) times. For each pair of methods, theVII(a) Complete 3D map from bird’s-eye view(b) 3D map with first-person viewof participant(c) Camera image with first-person viewof participant

Fig. 7. Reconstructed 3D map of the surroundings of the road shown in part (a). The
HMD renders the arrow on the 3D map shown in part (b) in the participant’s field ofview, which is illustrated in part (c).

participant used first the former and then the latter navigation method. Wethen asked each participant the following questions:
Questions
Q1: Which navigation method made it easy to understand the turning
point?
Q2: Which navigation method made it easy to understand the travel direction?
Answers (four response levels)– Absolutely the former navigation method (−1.5)– Maybe the former navigation method (−0.5)– Maybe the latter navigation method (0.5)– Absolutely the latter navigation method (1.5)
We also asked the inverse questions of Q1 and Q2, i.e., to determine which
method made it more difficult to understand the travel direction and the turning
point. Each participant selected an answer for each question of each pair. We
showed the pairs of questions to the participants in random order.
3.2

Results of the subjective assessment

Figure 8 shows the subjective scores from the questionnaire. Yardstick Y indicates that there is a significant difference of 5% or 1% when the differencein subjective scores between the navigation methods is larger than Y (0.05) orY (0.01), respectively. In the answers to Q1 and Q2, there were significant differences between method M4 and the other three methods. We can therefore claimthat each user found it easier to understand the turning point and the travel
direction when using method M4, as compared with methods M1 to M3. In theanswers to Q1, no significant difference was observed between methods M2 andM3. Despite the fact that M2 superimposes the turning point in the user’s fieldof view, we cannot claim that the user found it easier to understand the turning
point when using M2 as compared with M3. In the answers to Q2, however,there was a significant difference between M2 and M3. We can thus claim thatthe user found it easier to understand the travel direction when using M3 rather
than M2 because M3 superimposes the travel direction in their field of view.

VIII
M1
-1.2

**

M3 M2

-0.6

0

**

M4
0.6

1.2

Y(0.05)=0.266( *), Y(0.01)=0.325( **)

Q1

M1
-1.2

**

M2

**

M3 *

M4

-0.6
0
0.6
Y(0.05)=0.257(*), Y(0.01)=0.314( **)

1.2

Q2

Fig. 8. Rated scores of participants’ understanding obtained via comparison of navigation methods M1 to M4.

3.3

Assessment of likeability

We also evaluated the likeability of the navigation methods that superimposed
the turning point and the travel direction in the user’s field of view. We used
the same procedure that was described in Section 3.1. We asked the participants
the following questions:
Q3: Which navigation method guided you intuitively?
Q4: Which navigation method guided you comfortably?
We also asked the inverse questions of Q3 and Q4, i.e., to establish which of thenavigation methods guided the users less intuitively or comfortably. The answers
were given with the same four response levels.
Figure 9 shows the subjective scores that were obtained from the questionnaire. In the answers to Q3 and Q4, significant differences were again found between method M4 and the other three methods. We can therefore claim that M4
increases the likeability of the navigation method for the users when comparedwith methods M1 to M3. We also observed that M3 obtained better subjectivescores than M2. We believe that the users preferred the dynamic movement of
the arrow in M3 in accordance with the movement of the users when comparedwith the static fixed arrow in M2.

4

Assessment of the visual effect of the route

4.1

Design of the route navigation method

We evaluated the visual effects of superimposing a route that contained boththe turning point and the travel direction in the user’s field of view using thefollowing method:
M5: The navigation method showed the route within the range in which theuser is looking.

IX
M1
-1.2

**

**

M2

-0.6

M3 * M4

0

0.6

1.2

Y(0.05)=0.237(*), Y(0.01)=0.290( **)

Q3

M1
-1.2

**

M2 **

-0.6

0

M3

**
0.6

M4
1.2

Y(0.05)=0.230(*), Y(0.01)=0.281( **)Q4

Fig. 9. Rated likeability scores obtained via comparison of navigation methods M1 to
M4.
M5

Time

M4

Fig. 10. Navigation method M5 for evaluation of the visual effects of a route that
contains both the turning point and the travel direction.

The method again used the arrow objects to represent the route. We set the arrow
objects at 1 m intervals over the route from the starting point to the destination
point. Figure 10 shows an example of the superimposed arrow objects used formethod M5. Each participant evaluated navigation methods M4 and M5. The
reason for the use of the arrow objects here was that we intended to maintainconsistent experimental conditions between M4 and M5.

X
M4
-1.2

-0.6

p < 0.01 : **

M5
0

0.6

1.2

0.6

1.2

Q1
M4
-1.2

-0.6

**

M5

0

Q2

Fig. 11. Rated scores of participants’ understanding obtained via comparison of methods M4 and M5.

4.2

Results of superimposing the routeTen participants (eight males and two females, with an average age of 22.9 ± 1.3
years old) participated in the study. We compared the subjective scores of theparticipants for methods M4 and M5 using Q1 and Q2 from the questionnairethat was described in Section 3.1. Each participant used methods M4 and M5
in random order and then answered the questionnaire.
Figure 11 shows the subjective scores that were obtained from the questionnaire. We used the Wilcoxon signed-rank test in this case. There were no
significant differences (p < 0.01) between the methods for Q1. We believe thatthe users felt that they were able to understand the turning point that was provided by M5 in a similar manner to that provided by M4. However, there was asignificant difference for Q2. We believe that the users felt that it was easier tounderstand the travel directions provided by M5 when compared with M4 because method M5 superimposed the future travel directions in the field of viewin addition to the current travel direction.

4.3

Likeability of superimposing the route

We also evaluated the likeability of the navigation methods in which the routeis superimposed. We used the same procedure that was described in Section 4.1
with questions Q3 and Q4.
Figure 12 shows the subjective scores that were obtained from the questionnaire. No significant differences were observed between M4 and M5. We therefore
cannot claim that M5 increased the likeability for the users when compared with
M4. After completion of the experiments, we issued a free-form questionnaire to
the participants. From this questionnaire, we obtained the opinion that a single
moving arrow was sufficient for navigation, while there was also the opinion that
it is preferable to superimpose a number of arrows simultaneously. The results
for the likeability of the route were different for each participant. We therefore
need to perform a further subjective assessment of the method of displaying theroute.

XI
M4
-1.2

-0.6

M5
0

p < 0.01 : **
0.6

1.2

0.6

1.2

Q3
M5
-1.2

-0.6

M4
0

Q4

Fig. 12. Rated likeability scores obtained via comparison of methods M4 and M5.

5Conclusions

We have demonstrated the visual effects of showing the turning point and thetravel direction in navigation systems by superimposing them in the user’s fieldof view using a commercial HMD. The combination of the turning point and thetravel direction produced higher subjective scores in terms of user understanding
when compared with use of only one of these directions. A route that contained
the turning point and the current and future travel directions increased thesubjective score in terms of both user understanding and likeability of the travel
direction when compared with the simpler combination of the turning point andthe travel direction.
In future work, we will expand our assessment of the system’s usability, and
we also intend to develop a navigation method using more complex outdoor
routes.

References[1] Blades, M., Spencer, C.: How do people use maps to navigate through the world?
Cartographica: The International Journal for Geographic Information and Geovisualization (3) (1986) 64–75[2] May, A.J., Ross, T., Bayer, S.H., Tarkiainen, M.J.: Pedestrian navigation aids:
information requirements and design implications. Personal and Ubiquitous Computing 7(6) (Dec 2003) 331–338[3] Munzer, S., Zimmer, H.D., Schwalm, M., Baus, J., Aslan, I.: Computer-assisted
navigation and the acquisition of route and survey knowledge. Journal of Environmental Psychology 26(4) (2006) 300 – 308[4] Devlin, A.S., Bernstein, J.: Interactive wayfinding:use of cues by men and women.
Environmental Psychology 15(1) (1995) 23–38[5] Monobe, K., Tanaka, S., Furuta, H., Mochinaga, D.: Fundamental research on
traffic support to the pedestrian by route giude map based on the space perception.
Applied Computing in Civil Engineering 16 (2007) 323–330[6] Narzt, W., Pomberger, G., Ferscha, A., Kolb, D., Müller, R., J. Wieghardt, H.H.,
Lindinger, C.: Pervasive information acquisition for mobile ar-navigation systems.
In: Proceedings of the Fifth IEEE Workshop on Mobile Computing Systems &Applications. WMCSA ’03 (2003) 13–20

XII[7] Mulloni, A., Seichter, H., Schmalstieg, D.: Indoor navigation with mixed reality
world-in-miniature views and sparse localization on mobile devices. In: Proceedings of the International Working Conference on Advanced Visual Interfaces. AVI’12 (2012) 212–215[8] Mulloni, A., Seichter, H., Schmalstieg, D.: User experiences with augmented reality aided navigation on phones. In: Proceedings of International Symposium on
Mixed and Augmented Reality. ISMAR ’11 (2011) 229–230[9] Mulloni, A., Seichter, H., Schmalstieg, D.: Handheld augmented reality indoor
navigation with activity-based instructions. In: Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices andServices. MobileHCI ’11 (2011) 211–220[10] Oliveira, L.C.D., Soares, A.B., Cardoso, A., Andrade, A.D.O., Júnior, E.L.: Mobile augmented reality enhances indoor navigation for wheelchair users. Research
on Biomedical Engineering 32(2) (2016) 111–122[11] Narzt, W., Pomberger, G., Ferscha, A., Kolb, D., Müller, R., Wieghardt, J.,
Hörtner, H., Lindinger, C.: Augmented reality navigation systems. Universal
Access in the Information Society 4(3) (2006) 177–187[12] Feiner, S., MacIntyre, B., Höllerer, T., Webster, A.: A touring machine: Prototyping 3d mobile augmented reality systems for exploring the urban environment.
Personal Technologies 1(4) (Dec 1997) 208–217[13] Reitmayr, G., Schmalstieg, D.: Scalable techniques for collaborative outdoor augmented reality. In: Proceedings of International Symposium on Mixed and Augmented Reality. ISMAR ’04 (2004) 1–10
[14] Grasset, R., Mulloni, A., Billinghurst, M., Schmalstieg, D.: Navigation techniques
in augmented and mixed reality: Crossing the virtuality continuum. Handbook of
Augmented Reality (2011) 379–407[15] Krevelen, D.W.F.V., Poelman, R.: A survey of augmented reality technologies,
applications and limitations. International Journal of Virtual Reality 9(2) (2010)
1–20[16] Rehman, U., Cao, S.: Augmented-reality-based indoor navigation: A comparative
analysis of handheld devices versus google glass. IEEE Transactions on HumanMachine Systems 47(1) (2017) 140–151[17] Gerstweiler, G., Platzer, K., Kaufmann, H.: Dargs: Dynamic ar guiding system
for indoor environments. Computers 7(1) (2017) 1–19
[18] Kalkusch, M., Lidy, T., Knapp, N., Reitmayr, G., Kaufmann, H., Schmalstieg,
D.: Structured visual markers for indoor pathfinding. In: Proceedings of the first
IEEE international workshop of Augmented Reality Toolkit. ART ’02 (2002) 1–8[19] Sato, H., Ishizu, K.: Gait patterns of japanese pedestrians. Journal of Human
Ergology 19(1) (1990) 13–22[20] Scheffé, H.: The analysis of variance. John Wiley & Sons (1967)[21] Ed., J.: Sensory evaluation handbook. JUSE Press (1973)