Guiding Blind Pedestrians in Public Spaces by Understanding
Walking Behavior of Nearby Pedestrians
SEITA KAYUKAWA, Waseda University, JapanTATSUYA ISHIHARA, IBM Research - Tokyo, JapanHIRONOBU TAKAGI, IBM Research - Tokyo, JapanSHIGEO MORISHIMA, Waseda Research Institute for Science and Engineering, JapanCHIEKO ASAKAWA, IBM Research, United States and Carnegie Mellon University, United States
We present a guiding system to help blind people walk in public spaces while making their walking seamless with nearby
pedestrians. Blind users carry a rolling suitcase-shaped system that has two RGBD Cameras, an inertial measurement unit
(IMU) sensor, and light detection and ranging (LiDAR) sensor. The system senses the behavior of surrounding pedestrians,
predicts risks of collisions, and alerts users to help them avoid collisions. It has two modes: the “on-path” mode that helps
users avoid collisions without changing their path by adapting their walking speed; and the “o�-path” mode that navigates analternative path to go around pedestrians standing in the way. Auditory and tactile modalities have been commonly used
for non-visual navigation systems, so we implemented two interfaces to evaluate the e�ectiveness of each modality forcollision avoidance. A user study with 14 blind participants in public spaces revealed that participants could successfully
avoid collisions with both modalities. We detail the characteristics of each modality.
CCS Concepts: • Human-centered computing ! Accessibility technologies; • Social and professional topics !
People with disabilities.
Additional Key Words and Phrases: Visual impairments; pedestrian avoidance; collision prediction; blind navigation; audio
interface; tactile interface.
ACM Reference Format:
Seita Kayukawa, Tatsuya Ishihara, Hironobu Takagi, Shigeo Morishima, and Chieko Asakawa. 2020. Guiding Blind Pedestriansin Public Spaces by Understanding Walking Behavior of Nearby Pedestrians. Proc. ACM Interact. Mob. Wearable UbiquitousTechnol. 4, 3, Article 85 (September 2020), 22 pages. https://doi.org/10.1145/3411825

1

INTRODUCTION

Blind people face signi�cant risks of collision with other pedestrians when walking through public spaces dueto their lack of vision. According to one survey, 87.8% of blind people have collided or nearly collided withpedestrians, bicycles, and other obstacles [51]. Using a white cane is the most common method for the blind
to sense obstacles and pedestrians, but it requires a user to risk their safety to physically contact the object.
Therefore, blind people report that the cane is not useful in avoiding contact with walking pedestrians in crowded
Authors’ addresses: Seita Kayukawa, Waseda University, Tokyo, Japan, k940805k@ruri.waseda.jp; Tatsuya Ishihara, IBM Research - Tokyo,
Tokyo, Japan, TISIHARA@jp.ibm.com; Hironobu Takagi, IBM Research - Tokyo, Tokyo, Japan, TAKAGIH@jp.ibm.com; Shigeo Morishima,
Waseda Research Institute for Science and Engineering, Tokyo, Japan, shigeo@waseda.jp; Chieko Asakawa, IBM Research, New York, UnitedStates, Carnegie Mellon University, Pennsylvania, United States, chiekoa@us.ibm.com.

Author Version

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85

85:2

•

Kayukawa et al.

%XࣚRQV
RGBD
Cameras1 Collision Prediction
Pedestrian Detection& Tracking

IMU

2 On-path Mode: Changing Walking Speed
Slow Down

3 2ॾ-path Mode: Generating an Alternative Path

LiDAR

Localization

3XVKWKH%XࣚRQ

Estimating the risk of collision
Suitcase

Fig. 1. Overview of the proposed system. 1) The system predicts the potential risks of collisions using two RGBD cameras,
a LiDAR sensor, and an IMU sensor. Then, when the system detects a risk of collision, it emits low-urgency alert signals.
2) Blind users who receive these alert signals can avoid a collision by changing their walking speed. In addition, 3) whenpedestrians are blocking the user’s path, the system emits high-urgency alert signals continuously. If the user pushes thebu�on equipped on the suitcase, our system generates a path around the blocking pedestrians and guides the user safely
around them.

sidewalks and corridors or in other crowded environments [74]. Due and Lange reported that blind pedestrians
rely on the collision avoidance behaviors of sighted pedestrians, such as changing trajectory or stopping [16].
That is why collision incidents happen when sighted pedestrians have di�culty noticing blind pedestrians inpublic spaces such as stations [1] and airports [34].
Meanwhile, sighted pedestrians continuously adapt their speed and direction using their sense of vision to
make their walking seamless with nearby pedestrians [49]. We characterize such walking behaviors as two typesof avoidance behavior: (1) “on-path” avoidance: adjusting walking speed without changing the path; and (2)“o�-path” avoidance: changing the path and walking through free space. For example, sighted pedestrians choose
the on-path avoidance when other pedestrians will cut across in front, but choose the o�-path avoidance whenpeople are standing still in front and talking. Our goal in this work is to enable blind people to walk seamlessly
with nearby pedestrians by using the on-path and o�-path avoidance, like sighted pedestrians. We argue that theon-path avoidance is more important for blind people because they have to walk along non-visually sensible
landmarks. Changing their path frequently may risk them losing their way and becoming disoriented.
Research using computer vision has aimed to assist blind pedestrians to avoid obstacles or hazards [5, 11, 26,
41, 42, 57, 73, 74, 81]. These systems generate an alternative path around the detected obstacles and navigate
blind users. However, these systems provide only the o�-path avoidance, rather than the on-path avoidance.
BBeep is a sonic collision warning system to alert nearby sighted pedestrians about potential risks of a collision
via beeping sounds [34]. The system assumes sighted pedestrians will give way to blind users, and a guiding
system to help blind people walk seamlessly with nearby pedestrians has not been explored.
We present a guiding system to help blind people walk in public spaces by adapting their walking speed to
avoid collision with approaching pedestrians (the on-path avoidance) and by enabling them to avoid standing
pedestrians (the o�-path avoidance). The system �rst predicts the risks of collisions using sensors, and then it
recommends that the blind user adjust his or her walking speed to avoid a collision with a walking pedestrian ortake an alternative route to avoid a collision with a standing pedestrian. For example, if a pedestrian is going to
cut across in front of a blind user, the system recommends adjusting the blind user’s walking speed. This is called
the “on-path mode.” If a group of people is blocking the blind user’s path, the system recommends an alternativeroute and navigates the user. This is called the “o�-path mode.”
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •85:3

To realize the on-path and o�-path modes, our system predicts the risks of collisions with nearby pedestrians.
Our system �rst localizes the user’s position and calculates the user’s velocity using simultaneous localizationand mapping (SLAM) with a light detection and ranging (LiDAR) sensor. The system uses two RGBD cameras tocapture a wide �eld of view and detects surrounding pedestrians by applying a convolutional neural network(CNN)-based object detector from the data on the two cameras. Then, the system can accurately track and predictthe motion of multiple pedestrians by compensating for camera motion using the SLAM results. By combining
these sensing results, the system predicts the potential risks of collision (Figure 1 (1)). When the system detects arisk of collision, the on-path modes emit low-urgency alert signals for the users. By walking slowly or stopping
while being alerted, the users can avoid collisions without changing their path. (Figure 1 (2)). When someone isblocking the blind user’s path, the system continues to emit high-urgency alert signals (Figure 1 (3)). In suchsituations, the o�-path mode can be initiated by the user, and the system generates an alternative path to avoid
the collision with the standing pedestrian. We designed the system to be attached to everyday luggage like arolling suitcase. We attached two cameras and a LiDAR sensor to the bar of the handle on a suitcase and asked
blind users to carry it. This rolling suitcase form-factor is used as a supportive system for blind people in a
recent work [34]. This suitcase-based system has several advantages such as it can capture images without
signi�cant motion-induced blur and can carry sensors and computational resources easily [34] (other advantages
are described in 3.4.).
Navigation technologies for blind people commonly use an audio interface [2, 3, 8, 13, 18, 50, 52, 57, 59, 64, 65, 77]
or tactile interface [4, 62, 74, 78]. Each interface has its own characteristics. For example, audio interfaces canconvey clear instructions, but they may block ambient sounds that blind people often use to ensure theirsafety [10]. Although tactile interfaces may not block these ambient sounds, they have di�culty conveying
detailed information. Because of these characteristics, blind users’ preferences depend on the types of tasks andenvironments (e.g.turn-by-turn navigation, collision avoidance, indoor/outdoor navigation, and crowded/empty
spaces). In this paper, we present our implementation of tactile and audio interfaces to �nd out which is moresuitable for our target situations with our guiding system. The audio interface alerts the risks of collisions byusing beep sounds and guides the user by using text-to-speech feedback through a bone conduction headset.
The tactile interface warns of the risks of collisions with a vibrating handle and navigates users with a newly
developed directional lever, which shows the correct direction. We attached the two tactile devices to the handleof the suitcase.
We conducted a user study with 14 blind people in speci�c routes and evaluated the e�ectiveness of the audio
and tactile interfaces and the overall guiding system. We obtained the following results.
• Most blind participants successfully avoided the walking and standing pedestrians in both controlled andreal-world environments by using both interfaces.
• The sound-based audio interface for the on-path mode made it easier for blind participants to recognize
alerts from the system than the vibration-based tactile interface. One reason was that the vibration wasa�ected by the �oor texture.
• Participants completed tasks using the tactile interface (the directional lever) for the o�-path mode signi�cantly faster than they did using the speech-based audio interface.
Overall, participants had signi�cantly stronger preferences for the tactile interface after the studies. The audio
interface was useful in certain situations, but its tendency to block ambient sounds was a major drawback.

2 RELATED WORK
2.1 Blind Navigation Systems
Previous research presented various types of blind navigation systems [3, 4, 13, 18, 24, 44, 50, 52, 59, 65, 80].
Most of these systems use turn-by-turn navigation to help blind users get to their destination with the help of
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:4

•

Kayukawa et al.

localization technologies, such as a global positioning system (GPS) [52, 59], radio frequency identi�cation (RFID)
tags [4, 13, 19], visible light communication [50], and Bluetooth low-energy beacons [3, 15, 35, 65]. While thesesystems focus on navigating users to their destination, other systems enable avoiding obstacles during navigation.
For example, NavCog can convey possible obstacles that are registered on a map [66]. However, it does not help
users avoid dynamic obstacles, such as pedestrians, or temporal obstacles, such as carts and signboards.

2.2

Obstacle Avoidance Systems

Obstacle avoidance systems are developed using various types of sensors, such as lasers [38], ultrasonics [32, 67],
phone speakers and microphones [72], or depth sensors [11, 21, 26, 29–31, 41, 53, 79]. Some of these systems
present all the information detected by the systems, such as the distance and the types and sizes of objects. Users
have to avoid obstacles on the basis of such information by themselves. When several obstacles are present,
understanding the situation and avoiding them may be di�cult, so the path necessary to avoid obstacles must be
generated. Some systems plan a path around detected obstacles and navigate users using di�erent interfaces, suchas sound feedback [42, 57, 81], a cane connected to a wheeled robot [73], a mobile robot [26], or a leashed aerial
robot [5]. Guerreiro et al. proposed an autonomous suitcase-shaped navigation robot, CaBot, that guides blindusers to a destination [26]. CaBot can avoid static obstacles such as a standing person by changing the user’s path.
However, these systems mainly focus on static obstacles and support avoiding obstacles only by changing a user’s
path (o�-path avoidance). Avoiding walking pedestrians just by changing walking speed (on-path avoidance) has
not been explored in any previous research.
One common assumption is that sighted people will notice the presence of blind people and avoid colliding
with them. However, sighted people who are distracted by looking at smartphones or by talking with others
are at serious risk of colliding with blind people. To overcome this problem, BBeep was created to help blind
people avoid collisions with sighted pedestrians by using a suitcase-shaped system [34]. When the system detects
risks of collisions, it plays alert sounds to surrounding people. The drawback is that the sounds may attract too
much attention from these people. It may cause a disruptive situation because they will need to clear a path
immediately.

2.3

Non-Visual Interfaces for the Blind2.3.1 Audio Interfaces. Audio interfaces are used in existing blind navigation systems for conveying both turnby-turn navigation commands and obstacle avoidance paths. Natural language is mainly used for navigationcommands [3, 18, 35, 64, 65]. Non-speech sounds are also used for navigation, for example, 3D spatialized audio
cues are used for indicating turning directions [8, 77], and sound patterns are used for determining correct
directions [2, 57]. Audio interfaces can provide clear information that is easy to learn. However, blind people
often depend on ambient sounds to ensure their safety, and the audio commands may disrupt their hearing ability.
2.3.2 Tactile Interfaces. Tactile interfaces have been developed to overcome or supplement the limitations ofaudio interfaces. Previous research used vibration patterns to indicate approximate turning directions [62, 78].
Shape-changing interfaces have been used to indicate �ne directions. Animotus [69] is a cube-shaped interface
that conveys heading directions when its top-half is rotated. Animotus was evaluated by blind participants,
and they were able to follow a route provided by using the device [70]. Audio and tactile modalities have theirown characteristics, and the e�ciency of each interface depends on the tasks and environments. Some researchshowed that blind users prefer tactile-based navigation because they can focus on ambient sounds [47], andother studies showed that blind users preferred an audio interface because they do not need additional wearable
devices [60]. On the basis of this previous work, we designed audio and tactile interfaces and evaluated theire�ectiveness.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •

3

85:5

GUIDING SYSTEM FOR A PUBLIC SPACEOur main goal is to develop a guiding system to help blind people and to make their walking seamless withnearby pedestrians in public spaces. Such public spaces are characterized by “restricted, impeded, and unstable�ow of pedestrians in the levels of services de�ned by Polus et al. [55]. In this section, we describe the design ofour guiding system speci�cally for the following typical situations: A blind pedestrian tries to walk through apublic space such as public buildings and shopping centers. He/she is familiar with the route. He/she should be able towalk seamlessly with the surrounding pedestrians.
• Situation 1) Other pedestrians often cut across the blind pedestrian’s path at close range with average
walking speed (1 – 1.5 m per second). Such pedestrians can be regarded as dynamic obstacles for him/her.
• Situation 2) A group of standing pedestrians blocks the blind pedestrian’s path unintentionally. Suchpedestrians can be regarded as static obstacles for him/her.

3.1

On-path Navigation

Blind pedestrians are usually trained to walk along speci�c familiar routes with non-visual landmarks giventhat they lack vision. We argue that a system should stick to these familiar paths as much as possible evenwhen it navigates the blind pedestrian so as to avoid collision. We call such navigation without route-changes“on-path” navigation. In Situation 1, the system should recommend adjusting the blind user’s walking speed
without changing his or her path to avoid a collision. The technical challenge is creating a method to present such
alerts in real-time. Therefore, we designed two interfaces: a sound-based audio interface and a vibration-based
tactile interface based on previous work [7, 17, 27, 40, 45, 54, 56, 63]. The system �rst senses the walking speed
and direction of the approaching pedestrian and predicts the trajectory and the risks of collisions. Then, thesystem alerts the user to adjust his or her walking speed to avoid a collision with a walking pedestrian (seeImplementation).

3.2

O�-path Navigation

In Situation 2, the system needs to help a blind pedestrian avoid obstacles by having him or her move outof the path, walk through free space, and return to his or her path. Successfully navigating a user accurately
along a target path by continuously presenting directions is a challenge, but previous studies showed that audio
and tactile interfaces may have su�cient utility for such a situation [20, 69, 70]. Therefore, we designed andcompared both interfaces. The speech-based audio navigation was designed on the basis of previous methods like
Headlock [20]. A new shape-changing device to indicate the accurate direction in real-time was designed on thebasis of previous research [69, 70]. We call the device the “directional lever” (see Implementation and Figure 3 (2)).

3.3

A�achment Design

We designed the system as a set of components attachable to a standard rolling suitcase (Figure 1). Given thefootprint of the required sensors, it will be di�cult to make the system fully wearable in the near future. Asan alternative, we expect such a system can be attached to daily mobile devices, such as a rolling suitcase, a
shopping cart, and a wheelchair. Such a rolling suitcase with attachments can move naturally alongside a blind
person, much like a guide dog or a sighted guide who walks side-by-side. Kayukawa et al. created a supportive
system for blind people [34] and argued that this rolling suitcase form has four advantages: 1) for a blind user, arolling suitcase can often act as an extended sensing mechanism for identifying changes in �oor texture or as aform of protection from collisions with obstacles; 2) as a robotic sensing system, it also provides a convenient
place to store and attach sensors, power, actuators, and computing resources; 3) users can walk with the systemeasily on �at spaces; and 4) the system can capture images without signi�cant motion-induced blur. For thesereasons, we also chose a suitcase form for our prototype system.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:6

•

Kayukawa et al.
15*%'&DPHUDV

IMU
LiDAR

Pedestrian Tracking

Pedestrian/Hࣙ&DPHUD

5LJKW&DPHUD

SLAM-Based
Localization

3

Collision Prediction(A)

Future Trajectry&11%DVHG
Detection
3D-Position
Sensing

2 Future Position Prediction

Warning
Area(B)

Emergency Area

FuturePositionsUser’ sVelocity
Suitcase

User

2.5 m
1.2 m

1.2 m

Emit Warning Signals

Emit Emergency Signals

Fig. 2. Overview of the on-path mode. 1) The system detects and tracks the pedestrians’ position using two RGBD cameras
and the SLAM-based localization method. 2) The system estimates the user’s current position and velocity using the SLAM
results and then predicts the user’s future position. Next, 3) the system estimates two levels of collision risks and emits two
types of alert signals (A: a warning signal and B: an emergency signal) via an audio or tactile interface.

3.4

Navigation Interface Overview

The overview of the navigation process is as follows (see Figure 1). A blind user usually starts walking withthe on-path mode. The user is instructed by the system to slow down when he or she perceives alert signals,
sounds, or vibrations. The user can walk at normal speed again after the alert signal stops. This means the user’s
path is clear and safe. If a pedestrian is not aware that the user is approaching and blocks the user’s path, thesystem continues to emit alert signals. In such a situation, a user can push the start button on the handle to enable
the o�-path mode to avoid the standing pedestrians. The system automatically navigates the user in speech orwith the directional lever. After returning to the user’s usual path, the system automatically changes back to theon-path mode.

4

IMPLEMENTATIONIn this section, we �rst describe our implementation of the system, which is characterized by its on-path ando�-path modes, followed by the audio and tactile interfaces, respectively.

4.1

The On-path Mode

As shown in Figure 2, we attached a LiDAR sensor, an inertial measurement unit (IMU) sensor, and two RGBD
cameras to a suitcase. The system uses these sensors to predict the risks of collisions. In what follows, we explain
how the system predicts the risks of collision on a step-by-step basis.
4.1.1 Localization. During navigation, our system estimates the current location and direction of a user using a
cartographer package1 [28] of the robot operating system (ROS) [58]. The cartographer can localize by comparing
the 3D pointcloud map previously generated and real-time scanning data from the LiDAR and IMU sensors.
On the basis of the localization results, the system estimates what the user’s velocity and position will be four
seconds in the future (Figure 2 (2)).
4.1.2 Pedestrian Detection, Tracking, and Prediction. We use two RealSense D435 cameras2 for tracking pedestrians
to obtain a wide �eld of view. Each camera has an 69.4 ⇥ 42.5 �eld of view. By arranging the two cameras
horizontally (Figure 2 (1)), we can obtain about a 135 ⇥ 42.5 �eld of view. To calibrate the relative position andorientation of each camera for LiDAR, we used an intensity-based LiDAR camera calibration tool [75]. Pedestrians
are tracked using the following steps.
1 http://wiki.ros.org/cartographer

2 https://www.intelrealsense.com/depth-camera-d435/

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •85:7(1) The system detects pedestrians using a YOLOv3 object detector [61]. The model is trained for detecting
people using the publicly available COCO dataset [43].
(2) The system calculates the positions of detected pedestrians in camera coordinates. RGB-D images are used
to calculate the 3D positions of the detected pedestrians in the camera coordinates.
(3) The system calculates the positions of detected pedestrians using map coordinates. By using localization
results, it compensates for camera motion and converts pedestrian positions into map coordinates from thecamera coordinates.
(4) The system matches detected pedestrians with tracked pedestrians. To match the detected pedestrians withthe tracked pedestrians, we �rst use a Kalman �lter [33] for each track to predict the positions in the nexttime step. We assume each person has a 1.0m circle size in the 2D map and calculate the intersection overunion (IoU) for the detected and predicted circles. To �nd the best matches of tracked circles and detectedcircles using the IoU, we use the Hungarian algorithm [37].
(5) The system estimates the velocity for each tracked pedestrian using a Kalman Filter.
These steps for detection (steps 1–3), tracking (step 4), and velocity estimation (step 5) are done using separate
processes. The detection steps are done for each camera, and the detection results for each camera are mergedin the tracking step. All these steps were done at about 4–5 FPS when we used a laptop computer (Intel Core
i7-8750H CPU @ 2.20GHz, NVIDIA GeForce GTX 1080 Mobile GPU). On the basis of the estimated surrounding
pedestrians’ velocity, the system assumes that the pedestrians move at the constant velocity and predicts theirpositions four seconds in the future (Figure 2 (2)).
4.1.3 Collision Prediction. The system predicts the risk of a future collision on the basis of all the predicted
positions of surrounding pedestrians and the blind user, then it decides whether or not to emit an alert signal. A
collision is expected when a pedestrian’s future trajectory crosses the “Warning Area” shown in Figure 2 (3) A.
The system de�nes the area as triangular, its base length is 1.2<, and the apex position is the user’s predicted
position in 4 seconds. The system can dynamically change the “Warning Area” in accordance with the user’s
velocity. For example, when the user is walking faster, the system predicts collisions with pedestrians in a larger
area. When the user is walking slower, the system considers collisions in a smaller area. If the system detects
the intersection between the pedestrian trajectory and the warning area, the system decides the user has a riskof collision and emits the low-urgency alert signals. In addition, we de�ne the “Emergency Area” shown in
Figure 2 (3) B. The system de�nes the area as a �xed-size rectangle, 1.2 ⇥ 2.5<. When a pedestrian is in the area,
the system emits the high-urgency alert signals. In this case, we expect the blind user to stop immediately. The
system estimates the risk of collisions in the “Emergency Area” by assuming that both the suitcase and the user
face the same direction. Thus, in our user study, we asked blind participants to walk while keeping the suitcasesin the direction they were heading.

4.2

The O�-path Mode

4.2.1 Path Planning. To navigate a user in the o�-path mode, the system assumes that navigation maps include
route information that is safe for blind users. In the following user studies, we assumed that the �oor map had
the positions of the tactile paving. Note that this study focused on navigating users on tactile paving, but othernon-visual sensible landmarks, such as walls, can be used to de�ne the blind users’ path.
To enable users to avoid standing pedestrians, the system sets the current user’s position as the start positionof the o�-path mode, and it sets the position �ve meters ahead of the user on tactile paving as the goal positionof the o�-path mode. Then, the system plans a path to avoid the pedestrian(s) and return to the tactile paving.
The path planning can be done with the navigation packages of the ROS, Navfn global path planner3 , and DWA
3 http://wiki.ros.org/navfn

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:8

•

Kayukawa et al.

Fig. 3. 1) In the o�-path mode, our system calculates the angle (\ ) between the correct direction and the suitcase direction. 2)
The tactile interface has two vibration motors and a directional lever driven by a servo motor indicating the correct direction.
The directional lever tells the user the correct walking direction in accordance with the generated path in the o�-path mode.

local path planner4 [22]. Navfn can generate a safe path avoiding obstacles such as pedestrians, walls, and static
obstacles using surrounding structural information from the LiDAR sensor. When the system cannot generate asafe path, for example, in cases where no space is available for pedestrian avoidance, the system continues toemit the emergency alert to stop the blind user until a path becomes available.
4.2.2 Direction Estimation. Until the user reaches the de�ned goal position, the DWA local path planner showsa local plan with the trajectory that the user should follow at that moment. Our system uses the local plan toestimate the correct walking direction (\ in Figure 3 (1)). The correct direction is de�ned by a straight path to theend of the local plan. To guide blind users in the correct direction, the system calculates the angle between thecorrect direction and the suitcase direction estimated by the SLAM.

4.3

Interface for Navigation Instruction

4.3.1 Audio Interface. Our audio interface emits beeps to alert the user about the risk of collisions. We use abone conduction headset to convey navigation information without impeding environmental sounds. Beeps
have been used as a means to alert people about urgent situations, such as in aircraft [9], nuclear power
plants [46], and hospital intensive care units [48]. Audio noti�cations can also alert drivers of an imminent riskof collision or assist in navigation [45]. The relationship between perceived urgency and sound parameters iswell documented [17, 27, 45]. We prepared two types of beeps with di�erent urgency levels denoted low andhigh. Speci�cally, we varied the pulse rate, the pitch, and the base frequency, as given in Table 1. The values weused are based on previous research addressing sound urgency [34, 54, 63].
In the o�-path mode, the audio interface instructs the user on the correct walking direction via text-to-speech
feedback such as “Right,” “Left,” and “Go straight.” These navigation commands were used in a prior navigation
system called Headlock [20], which provides information to navigate toward detected objects (e.g., direction
and distance) via audio. We chose such simple navigation commands because a previous study revealed thatblind people have di�culty adjusting their orientation slightly [66]. Our audio interface says “Right” or “Left”
depending on the angle between the correct direction and the suitcase direction. When the absolute value of theangle is within 10 degrees, the system says “Go straight.” These speech commands are emitted at one-second
intervals.
4 http://wiki.ros.org/dwa_local_planner

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •85:9

Table 1. Feedback pa�erns. PD: Pulse duration, IPI: inter-pulse interval, and BF: base frequency.

Interface

Urgency Level

PD

IPI

Audio

Low
High

0.5 s 0.5 s 400 Hz
0.1 s 0.1 s 1000 Hz

Tactile

Low
High

0.5 s
Inf

0.5 s
N/A

BF

N/A
N/A

4.3.2 Tactile Interface. To provide vibration feedback in the on-path mode, we attached two vibration motors(T.P.C., FM34F), which were connected to an Arduino Uno Rev35 on a suitcase handle (Figure 3 (2)). Studies haveused vibration to alert people of emergency situations [25, 74], and the relationship between perceived urgency
and vibration parameters has been shown [7, 40, 56]. Speci�cally, perceived urgency signi�cantly decreases as
the inter-pulse interval (IPI) increases. On the basis of previous research [7, 40, 56], we designed a low-urgency
tactile signal that has an IPI and a high-urgency tactile signal that causes continuous vibrations (Table 1).
In the o�-path mode, the system needs to convey the correct direction to avoid obstacles. The tactile interface
has also been used to show directions to destinations [14, 47, 62, 68]. We designed the “directional lever,” which
always indicates the correct direction (Figure 3). The directional lever is rotated by a servo motor (NEW TC,
SE-A410) that is connected to an Arduino Uno Rev3. We attached the directional lever under the suitcase handle.
As shown in Figure 3 (2), a user holds the suitcase handle with one hand and clamps the directional lever with the�ngers. While the suitcase is facing the correct direction, the directional lever indicates ahead. When users should
turn right (left), the lever indicates right (left) in accordance with the angle of the estimated correct walking
direction (\ in Figure 3 (1)). For comfortable use, we clipped the angle range of the directional lever from 30 to
30 .

5

USER EVALUATIONS

We conducted a user study with 14 blind people. The main goals were 1) to evaluate if participants could avoid
collisions using our guiding system, and 2) to evaluate the e�ectiveness of the tactile and audio interfaces. We
asked the participants to walk a short route in a controlled environment and to walk a long route in a real-world
environment.

5.1

Participants

As shown in Table 2, we recruited 14 blind participants (6m/8f) with ages ranging from 32 to 70 (Mean=50.43 andSD=10.04). Thirteen participants (P1–P13) regularly used a white cane, and one (P14) owned a guide dog. They
considered themselves to have good orientation and mobility skills.

5.2

Tasks

In this study, we asked the participants to walk on tactile paving located in two types of environments: 1)
controlled environments where one experimenter crossed or blocked a blind user’s path, and 2) real-world
environments where many di�erent people were walking. The participants walked through these environmentswith either the audio or tactile interface.
5 https://store.arduino.cc/usa/arduino-uno-rev3

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:10•

Kayukawa et al.

Table 2. Demographic information on our participants and the SUS Score for each interface.

ID

Demographic information
GenderNavigation Aid

Age

SUS Score (Grade)
Audio
Tactile

P01
P02
P03
P04
P05
P06
P07
P08
P09
P10
P11
P12
P13
P14

FemaleMale
Male
FemaleFemaleFemaleFemaleFemaleMale
Male
FemaleMale
Male
Female

44
56
48
47
48
57
51
43
40
70
32
55
69
46

85
82.5
90
67.5
72.5
65
87.5
45
72.5
57.5
77.5
75
90
97.5

A+
A
A+
C
C+
C
A+
F
C+
C
B
B
A+
A+

77.5
82.5
92.5
62.5
72.5
65
87.5
52.5
75
47.5
70
72.5
95
97.5

B+
A
A+
D
C+
C
A+
D
B
F
C
C+
A+
A+

Mean 50.43
SD 10.04

76.1
13.7

B

75.0
14.6

B

Cane
Cane
Cane
Cane
Cane
Cane
Cane
Cane
Cane
Cane
Cane
Cane
Cane
Dog (primary) and Cane

5.2.1 Controlled Environments. To evaluate the e�ectiveness of our system in the same conditions across all
participants, we �rst prepared a controlled environment that had a simple route of 16 meters with tactile paving.
In that environment, one experimenter interrupted participants’ walking. We prepared two conditions to evaluate
the e�ectiveness of the on-path and o�-path modes (see also Figure 4 (1)): A) one experimenter walked across theroute at two points, and B) one experimenter blocked the tactile paving at two points. We asked the participants
to walk the route four times (two interfaces ⇥ two conditions). In condition A, the experimenter started walking
at the time at which they would collide with the participants. Each participant held the suitcase handle with one
hand and used his or her white cane with the other. Their goal was to walk on the tactile paving while avoiding
collisions with the experimenter. Participants started each task without knowing how the experimenter would
behave (walk across or block their path). We asked participants to change modes on the basis of the feedback
from the system.
5.2.2 Real-world Environments. In the study, we also asked blind participants to walk a long route (approximately
180 meters) on the ground �oor of an o�ce building (Figure 4 (2)). We selected this real-world environment
because people constantly walk into or out of the o�ce, restroom, convenience store, co�ee shop, etc. Whenparticipants walked on the route, one experimenter crossed the participants’ path at two points and blocked thepath at one point (Figures 4 (2) and 5). Participants walked the long route twice using either the audio or tactileinterface.

5.3

ProcedureWe �rst provided an overview of the study and administered a questionnaire on demographics and navigation
habits. We also surveyed the participants’ opinions about using a headset to receive navigation instructions while
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •1) Controlled Environment2) Real-world Environment

8m

S

Insurance
Counter

Scale

Tactile Paving

Route0 m 10 m 20 m

Start Position

Walking Pedestrian

ce

6m

6m

4m

Entra
n

Condition A
8m
60°

85:11

Convenience
Store

&RসHH
Shop

Rest
Room

G

S

16 m

Condition B
4m

S

G

S

G Goal Position

Standing Person

Entrance4m

G

Fig. 4. The routes used in our user study. 1) We first asked the participants to walk routes in a controlled environment. The
routes included condition A, where an experimenter crossed the participants’ route at two di�erent points, and condition B,
where an experimenter blocked the route at two di�erent points. 2) Blind participants also walked a long route (around 180
meters) in a real-world environment. The route included two points where an experimenter crossed the participants’ routeand one point where an experimenter blocked the route.

walking alone. Next, we described the two modes for two types of interfaces to the participants and gave them ashort training session (10 – 20 minutes) until they became familiar with each interface. We adjusted the volume
in the audio interface to make sure it was comfortable but audible. During the training session, we explained
how to hold the suitcase as it a�ects the accuracy of collision prediction.
For the �rst task, we asked the participants to walk the short route in the controlled environment while
using either the audio or tactile interface. They walked the route four times while changing the interfaces andconditions (conditions A and B shown in Figure 4 (1)) in a counter-balanced order. For each trial, we measured
the task completion time and counted the number of collisions between the participants and the experimenter.
After completing the �rst task, participants took a post-questionnaire, which was audio recorded for further
analysis. Speci�cally, we asked the participants to rate the following sentences using 7-point Likert items (ratingfrom 1: strongly disagree to 7: strongly agree):Q1: “The on-path mode with audio interface helped me avoid collisions while changing my walking speed.6 ”
Q2: “The on-path mode with tactile interface helped me avoid collisions while changing my walking speed.”
Q3: “The o�-path mode with audio interface helped me avoid pedestrians who blocked my path.”
Q4: “The o�-path mode with tactile interface helped me avoid pedestrians who blocked my path.”
We also asked open-ended questions about the advantages and challenges of each interface.
The remaining tasks were performed in the real-world environment shown in Figures 4 (2) and 5. The
participants had a training session to walk the route twice while using each interface to grasp the overall route.
Then, they were again asked to walk the route twice while using either the audio or tactile interface. In each trial,
we did not instruct the participants to use a particular mode. Instead, we asked them to change the modes bythemselves on the basis of the alerts from the system. We informed the participants that a researcher would bewalking behind them to guarantee their safety as well as other pedestrians’ safety (Figure 5). The researcher
did not intervene unless an imminent risk or a deviation from the path occurred. We counted the number oftimes imminent risks of collisions occurred for each condition. To observe the participants’ movement and the
response of pedestrians, we mounted a GoPro camera on the top of the suitcase during the study.
6 All of the communications with participants were done in their native language. In this paper, we describe any translated content in theform of “translated content”.

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:12•

Kayukawa et al.

A) Avoiding a walking pedestrianExperimenterB) Avoiding a standing person

Walking PedestrianStanding Person
SystemParticipant

SystemExperimenter
Participant

Fig. 5. User study of a real-world environment. Participants walked on tactile paving while avoiding a walking experimenter(A) and a standing experimenter (B). One experimenter crossed the participants’ path at two points and blocked the path at
one point.

After completing all the tasks, participants took a post-questionnaire. The participants were again asked torate the four sentences (Q1–Q4) they assessed after the �rst task in the controlled environment. In addition, we
asked them to rate the following sentences about their preferred interface using 7-point Likert items (rating from1: do not prefer to 7: prefer):
Q5: “I prefer the audio interface for navigation instructions.”
Q6: “I prefer the tactile interface for navigation instructions.”
They also rated the items of the system usability scale (SUS) [12]. Finally, we asked open-ended questions togather feedback about their overall experience with each interface. The task process took around 45 minutes,
while the whole experiment took approximately 90 minutes per participant.

6 RESULTS
6.1 Audio Interface Usage
All participants commented that receiving auditory navigation commands via headsets is undesirable because
these commands may interfere with ambient sounds. For that reason, 11 participants out of 14 mentioned thatthey never use a headset while walking: A1: “When I’m walking alone, I always listen to various sounds such asfootsteps and engine sounds. A headset may block these sounds, so I always walk without using one.” (P3); A2: “When
I use a headset to get navigation instructions, I always keep in mind that I need to stand in a safe zone, such as near awall. If I receive auditory commands while walking through public spaces, I am distracted by these commands andhave di�culty hearing ambient sounds.” (P9); A3: “I sometimes use Google Maps and a headset to get instructions
to the destination. While using the app, I make sure I stop walking.” (P5); and A4: “When audio commands comefrom a headset, I tend to concentrate on listening to them. In fact, I was once nearly hit by a car when I tried walking
with a headset.” (P10). Although seven participants (P5–P7, P9, P11–P13) mentioned that they sometimes use
audio-based navigation systems, such as Google Maps, they also strongly agreed on the risk of using a headset
while walking. In particular, three participants (P5, P7, and P9) mentioned that they stop walking while listening
to audio commands from a navigation app.

6.2

Experience of Collision with Nearby Pedestrians

Thirteen participants out of 14 mentioned that they have collided or nearly collided with pedestrians in public
spaces. The only exception was P14, who commented that her dog was very well trained at avoiding collisions.
All participants except P2 and P14 also mentioned that they had experienced situations in which someone had
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •85:13

Table 3. The number of times the system emi�ed a warning or emergency alert.

Interface
Audio
TactileMean and SD
Warning Alert Emergency Alert
12.6±5.7
12.5±3.1

4.1±2.2
8.1±3.6

blocked their path even on a tactile paving. P2 commented that he could recognize empty spaces with no standing
pedestrians by listening to the ambient sounds.

6.3

Overall Performance

6.3.1 The Number of Collisions. In our controlled study, all participants reached the goal without collisions.
In our real-world study, while participants did not encounter real pedestrians who blocked their path, several
pedestrians crossed it. Table 3 reports the number of times the system emitted a warning or emergency alert.
The participants were thus at risk of collision about 10 times in each trial. Blind participants could avoid suchpedestrians by using the on-path mode. Most participants also successfully avoided the experimenter who crossed
the participants’ path at two points and blocked it at one point. However, two exceptions occurred. In these cases,
participants had an imminent risk of colliding with the standing experimenter, and another experimenter had to
ask them to stop. The reasons for these close calls were as follows: 1) P11 and P13 continued to walk without
noticing the vibration alerts from the system; and 2) although the system generated a path to the left, the system
told P7 to “go straight,” because the suitcase was facing the generated path (i.e., a di�erence occurred between thesuitcase direction and the user’s walking direction).
6.3.2 Task Completion Time. Table 4 reports the task completion time in terms of its mean and standard deviation,
as well as 95% con�dence intervals, obtained using each interface. The table also shows the ?-value of the Wilcoxon
signed-rank test. Our statistical analysis revealed that, in the o�-path mode, participants who used the directional
lever could avoid a standing person more quickly than those who used the speech-based audio interface. In theon-path mode, we observed no signi�cant di�erences in the task completion time between the audio and tactileinterfaces.
6.3.3 Video Observation. Video recordings enabled us to analyze the behavior of the blind user and sighted
pedestrians to complement our quantitative analysis. We observed that participants could switch between theon-path mode and o�-path mode on the basis of the feedback from the system. Participants mainly walked
with the on-path mode and slowed down or stopped when the system emitted alert signals. When the system
continued to emit alerts, they pushed the start button to enable the o�-path mode. In the controlled environments,
all participants could switch between the on-path and o�-path modes successfully. In the real-world environment,
many participants also could change the modes properly. However, as mentioned above, P11 and P13 who used
the tactile interface continued to walk without noticing the vibration alerts and had an imminent risk of colliding
with the standing experimenter.

6.4

Ratings of Our System

Figure 6 shows the participants’ ratings: the e�ectiveness of the on-path mode with the audio interface (Q1) andtactile interface (Q2), the e�ectiveness of the o�-path mode with the audio interface (Q3) and tactile interface (Q4),and the participants’ preferences for the audio interface (Q5) and tactile interface (Q6). In the study, we asked
Q1–Q4 after participants �nished tasks in both the controlled and real-world environments. We compared theseProc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:14•

Kayukawa et al.

Table 4. �antitative evaluations of the task completion time: Mean and SD: the mean and standard deviation of the taskcompletion time; Lower and Upper: the lower and upper bounds of the 95% confidence intervals, respectively; and the ?-valueof the Wilcoxon signed-rank test (* indicates the significance found at the levels of 0.01).

Condition
Pattern A (the on-path mode)
Pattern B (the o�-path mode)

Audio Interface

Tactile Interface?-value

Mean and SD

Lower

Upper

Mean and SD

Lower

Upper

36.3 ± 5.85

32.8

39.8

34.3

44.6

0.23

53.5

65.5

39.4 ± 8.60

48.7

58.4

0.015*

59.5 ± 9.97

53.6 ± 8.08

questions using the Wilcoxon signed-rank test. The ?-values of each test are shown in Figure 6. A comparison ofanswers to each question between the controlled and real-world environments showed a signi�cant di�erence in
Q2. The e�ectiveness of the vibration-based tactile interface signi�cantly reduced after the participants used itin the real-world environment. In addition, we also observed a signi�cant di�erence between Q5 and Q6. Our
analysis found that participants had signi�cantly stronger preferences for the tactile interface than the audio. In
Q6, 12 participants rated the tactile interface higher than neutral on its navigation instructions (over 5 points).
We observed no signi�cant di�erences in the other tests.
Table 2 reports the scores of the system usability scale (SUS) for each participant. The mean (M) and standard
deviation (SD) of the SUS score were 76.1 and 13.7 for the audio interface and 75.0 and 14.6 for the tactile interface,respectively. Participants who did not like our system mainly pointed out the di�culties with the suitcase form.
We describe the feedback from them in a later section.

6.5

User Feedback on Our System

6.5.1 Overall Experiences. Participants generally agreed that the on-path mode was e�ective: A5: “While walking
alone, I always concentrate on grasping surrounding environments via auditory sensations to avoid collisions. Using
the system alerted me of the risks of collisions, so I could walk more con�dently” (P13); A6: “The system emits noalert signals when my path is safe. Therefore, I could feel safe walking when no alerts were being emitted” (P8); A7:“The system [the on-path mode] told me my path was clear and safe. So, I could walk faster while the system was not
emitting alert signals.” (P1); and A8: “In my workplaces [a braille library and a school for the blind], I sometimes
collide with other blind pedestrians. Thus, I want to use the system in my o�ce” (P11).
We also got positive feedback on the o�-path mode: A9: “When I encounter a group of people who block mypath, I always ask them to move out of the way. By using the system, I could avoid such people by myself” (P12); andA10: “When I avoid obstacles, I sometimes lose my way and become disoriented. The system [the o�-path mode] isuseful because it can guide me back to my path” (P7).
To generate an alternative path in the o�-path mode, we used the navigation packages in ROS. The packages
are standard for controlling robots, but P7 and P8 commented that the navigation was not smooth enough forhuman navigation: A11: “When I avoided pedestrians using the o�-path mode, the system sometimes instructed me
to alternate between going toward the right and left. Thus, I felt I lost my position and direction” (P7); and A12: “In
the o�-path mode, the system provided me with the same direction repeatedly, and it made me confused” (P8).
Nine participants provided negative feedback on the suitcase form. We present our examination of this feedbackin the discussion section: A13: “This suitcase-shaped system was too large and heavy for daily use” (P10); A14:
“When I walk with a cane, I want to keep my other hand free” (P6); and A15: “This system is large and I’m afraid of
the additional risk of hitting it against other pedestrians, especially in a very crowded area” (P8). Three participants
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •85:15(ॾHFWLYHQHVVRI2QSDWKPRGH
Q1: Audio(Beep sounds)

Q2: Tactile
9LEUDWLRQ

Controlled
Real-world
Controlled
Real-world

p = 0.57
p = 0.16

p = 0.11

p = 0.003**

1
2
1RW(সHFWLYH

3

4
Neutral

5

6
7(সHFWLYH(ॾHFWLYQHVVRI2ॾSDWKPRGH
Q3: Audio
6SHHFK

Controlled
Real-world

p = 0.23
p = 0.10

Q4: Tactile

Controlled'LUHFWLRQDO/HYHU Real-world

p = 0.56
p = 0.37

1
2
1RW(সHFWLYH

3

4
Neutral

5

6
7(সHFWLYH

Preferable Modality for Navigation Instruction
Q5: Audio Interfacep = 0.013*

Q6: Tactile Interface
1
2
3
Not Preferable

4
Neutral567Preferable

Fig. 6. Box plots of users’ ratings (Q1–Q6): Controlled and Real-world are ratings a�er tasks in the controlled environment
and the real-world environment, respectively, and ?: ?-is the value of the Wilcoxon signed rank test done on each question (*and ** indicate the significance found at the levels of 0.01 and 0.001, respectively).

(P7, P13, and P14) also mentioned the need to shrink the tactile interface: A16: “If the tactile interface can be
attached to my cane, I want to use it every day” (P13); and A17: “I’d be so happy if the tactile device were made small
and lightweight enough for me to hold it in my hand” (P7).
6.5.2 Audio Interface. All participants mentioned that the audio feedback was easy to recognize: A18: “Audio
feedback is clearer than tactile feedback. Thus, I could respond to it quickly” (P5); and A19: “I could distinguish beepsounds easily because these sounds were characterized by not only the pulse rate but also the pitch” (P4). However,they also reported that the audio interface was disadvantageous for sensing ambient sounds: A20: “I walk while
getting surrounding information using my ears, so I don’t want to use a headset while walking if at all possible. In
particular, when I heard auditory instructions through the headset, I had di�culty listening to footsteps” (P9); A21:
“When I was using the audio system in the quieter environment [the controlled environment], I could easily hear bothambient sounds and audio signals. However, in the noisy environment [the real-world environment] where there
are many ambient sounds such as footsteps, it was necessary to use extra awareness to hear ambient sounds” (P5);
A22:“In the real-world environment, where many people were walking, there was a larger amount of informationfrom ambient sounds than in the controlled environment. So, I had di�culty recognizing both ambient sounds andaudio-based feedback” (P11); A23:“Text-to-speech feedback distracted me more than beeping sounds” (P10); andA24:“The instructions from the system were simple. So, I could distinguish instructions even if I used the tactile system.
I think it is excessive to use the audio system to convey such simple instructions” (P2).
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:16•

Kayukawa et al.

6.5.3 Tactile Interface. All participants appreciated that the tactile interface did not interrupt auditory sensations:A25: “I could easily grasp the surrounding environment and collision risks by simultaneously using my cane, my ears,
and tactile signals from the system” (P8); and A26: “Audio-based feedback interfered with my auditory sensations,but I could use both the auditory sensations and tactile-based instructions. The tactile interface gave me an additional
sensing modality” (P11). However, some participants mentioned that the vibration alerts were di�cult to recognize:
A27: “I had to distinguish between two vibration patterns by considering the pulse duration, so it was harder thanbeep sounds with changing pitch” (P5). In addition, eight participants commented that the e�ectiveness of thevibration depended on the surface of the �oor: A28: “The ground of the real-world environment had a rough surface,
and the handle of the suitcase also vibrated. I had di�culty distinguishing between the suitcase vibration and thevibration alerts” (P9); and A29: “[In the real-world environment,] I had to concentrate on recognizing the vibration
alerts because the suitcase vibrated due to the unevenness of the �oor” (P6).
The blind users provided positive comments on the directional lever: A30: “The directional lever helped me
adjust the walking direction because it indicated the correct direction directly” (P8); and A31: “The lever could tell me
the direction more precisely than auditory commands could. If the system were to say precise directions, like ‘turnright 32 degrees,’ it would be time-consuming and annoying”; (P3). In particular, all the participants mentioned thatthe directional lever was e�ective in both the controlled and real-world environments: A32: “The directional lever
was not a�ected by the surface of the ground and always worked e�ectively” (P7). However, three participants (P1,
P2, and P10) mentioned that the navigation lever took time to get used to: A33: “The directional lever took a while
for me to get used to because it indicated the direction too precisely” (P2).

7 DISCUSSION
7.1 E�ectiveness of The Guiding System
Both the controlled and real-world studies showed that our guiding system was e�ective for blind users to prevent
collisions with pedestrians. Most participants successfully avoided collisions with nearby pedestrians by using
the on-path and o�-path modes properly. Feedback from the participants also supported the e�ectiveness of oursystem (A5–A10). They appreciated that our collision warning system enabled them to walk more con�dently
and with a more secure feeling than they usually do in daily life (A5–A7).

7.2

Audio Interface

We observed that participants listen for ambient sounds to ensure their safety while they walk through a public
space. All participants commented that receiving audio feedback frequently while walking in the real-world
situation was not usable because it interfered with their ability to make out useful ambient sounds such as footstepsof other pedestrians and echoes from walls (A1–A4, A20–A24). In particular, two participants commented thatthey did not want to use the audio system in the real-world environment because of the rich ambient sounds
they needed to listen for (A21 and A22). Although the participants mentioned that they could recognize thesound alerts more clearly than the vibration alerts (A18 and A19), they had signi�cantly stronger preferences
for the tactile interface (Q5 and Q6). This result is understandable since the footsteps and other sounds fromother pedestrians are faint and can be easily masked by other ambient sounds or computer-generated navigationcommands. In echolocation, changes in frequency and amplitude of low-frequency sounds need to be detected torecognize changes in echoes [36]. Such recognition requires trained abilities, which vary among blind people(see 1. Introduction). From this study, audio interfaces are not a promising interaction method for navigation
tasks. We believe more studies should be done to seek better interaction methods for the diverse abilities of blind
people by combining audio, haptics, and other non-visual media.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •

7.3

85:17

Tactile Interface

7.3.1 Advantages. We observed two advantages of the tactile interface. First, the users could recognize thevibration alerts and the correct direction indicated by the lever while listening to ambient sounds (A25 and A26).
Second, the directional lever enabled the participants to avoid standing pedestrians signi�cantly faster than the
speech-based audio interface did. The participants also commented that the directional lever could indicate thecorrect direction directly (A30 and A31), and all participants commented that the lever was not a�ected by thesurface of the ground (e.g., A32). Therefore, they revealed stronger preferences for the tactile interface than for
the audio interface (Q5 and Q6).
7.3.2 Disadvantages: Vibration Alerts. Some participants expressed some concern about recognizing the vibration
alerts. In the controlled environment, all participants successfully avoided collisions using the vibration alerts.
However, in the real-world environment, two participants sometimes could not recognize the vibration alerts andhad imminent risks of collisions. In the controlled environment, the �oor was carpeted, and the vibration alerts
were clear. However, in the real-world environment, the �oor was tiled with a rough surface, and the vibration
alerts were mixed with the vibration of the suitcase. Eight participants commented that the e�ectiveness of thevibration alerts was a�ected by the �oor texture (e.g., A28 and A29). These results indicate that the vibration alerts
are a�ected by the type of �oor surface. The participants provided signi�cantly lower scores for the e�ectivenessof the vibration alerts in the real-world environments than in the controlled environments (Q2).
The one possible solution to overcome this limitation is to attach vibration motors to a user’s body, such as ona wrist or �ngers. All participants mentioned that the directional lever was always useful and e�ective in both
environments (e.g., A32). This may suggest that the directional lever in a shape-changing interface was e�ective
for the guiding system and that this interface is better than vibration signals for indicating alerts.
Participants were asked to adjust their walking speed on the basis of the pulse duration of the vibration, but it
was hard for some participants (A27). Another possible solution is to equip brakes on the wheels of the suitcase
and to control the walking speed using the physical feedback from the suitcase handle.

7.4

Form-factor of the System

We designed the system to be attached to standard luggage like a rolling suitcase. Rolling suitcases can beseamlessly used in public spaces, are well designed to walk with when holding the device, and can have allnecessary sensors and tactile devices mounted on them, and enable images to be captured without signi�cantmotion-induced blur from the mounted camera (see also 3.3 Attachment Design above). Eight out of 14 participants
mentioned that the suitcase-shaped system might be too heavy for daily usage (A13–A15). This suggests thatthe device will be accepted when they use a suitcase (or any other similar luggage) for other purposes, and thenattach the system as assistive technology. At this moment, our solution is comprised of a depth camera and alaptop, which increase the weight and reduce the available space (for luggage) in the suitcase. This situation
usually happens when a new assistive technology is developed [11, 53]. We expect that the size and weight of boththe sensors and processors will be decreased as the device and communication technologies are systematically
improved, enabling cloud-based computational power in the near future.
Another result related to the form is the possibility of mobile devices with total functionality. The directional
lever was well accepted by all participants, and some commented that they would want to use the device on adaily basis if it were made the size of a mobile device (A9 and A10). We have to overcome technical challenges toenable such a form, but our results suggest a high possibility of utilizing mobile shape-changing devices.

7.5

Integrating a Guiding System to O&M Training Methods

Some participants commented that the experiments were their �rst time they were able to avoid pedestrians byfollowing the instructions from a system (A9 and A10). Also, while all participants could learn how to use theProc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:18•

Kayukawa et al.

system after a short training session (10 – 20 minutes), three of them commented that it took time to get used tothe directional lever (A33). We asked participants to hold the suitcase handle with one hand and the white canewith the other. One participant (P6) commented that she want to keep her other hand free (A14). The current
orientation and mobility (O&M) method is based on a white cane as the primary tool, and it uses all possible
senses to understand the situations. Skill is required to recognize non-visual landmarks and to navigate safely inpublic spaces. Therefore, all blind users are strongly recommended to take O&M training [76] when they start
walking independently. The use of systems for O&M is an uncharted territory for not only an individual blinduser but also the entire community who supports O&M for the blind. In the near future, we should share theresults and our experience with the community and discuss how to build new O&M methodologies by fully
utilizing both traditional navigation aids and new technologies, including the system we introduced in this study.

7.6

Autonomous Guiding Systems

The o�-path mode requires a guiding system to navigate users accurately. A future solution for such navigationneeds can be autonomous guiding robots [5, 6, 23, 26, 39, 71, 73]. Our non-autonomous guiding system has theadvantage of allowing users to control their speed voluntarily even during an o�-path situation. The directional
lever successfully achieved su�cient accuracy for the o�-path navigation. We believe that our guiding interface
and autonomous guiding robots will complement each other to broadly satisfy blind users’ needs, such as avariety of mobility skills, familiarity with a target public space, the density of crowds, and the preferences ofusers.

7.7

Other Guiding Situations in Public Spaces

We focused on two typical situations for this study: walking and standing pedestrians. No comprehensive list forsuch situations has been reported in previous studies to the best of our knowledge, but imagining other situations
is not di�cult. For example, a blind user may have di�culty in following a queue to get on a train car at a station,
walking together with sighted surrounding people in the same direction, and walking through an extremely
crowded public space. Such situations are beyond the scope of this study, and further research is required tocover a comprehensive set of situations.

7.8

Comparing with Traditional Navigation Aids

This study was not designed to compare the proposed system with traditional navigation aids such as a cane or
guide dog, because traditional methods are very challenging in public spaces. Previous studies reported that blindpedestrians using only traditional methods had di�culty avoiding collisions with nearby pedestrians [1, 16, 34, 51,74]. In fact, 13 out of 14 participants in our study mentioned that they had collided with other pedestrians while
walking. On the other hand, we believe that it would be informative for researchers and developers to understandhow blind pedestrians behave when using traditional methods in public spaces as the baseline. Therefore, weplan to measure, evaluate, and create a model of such behaviors and compare traditional methods with other new
navigation methods, including our system.

8

CONCLUSION

We presented a guiding system equipped on a rolling suitcase to help blind people walk in public spaces seamlessly
with nearby pedestrians. The system recognizes and predicts surrounding people’s behavior and predicts therisks of collisions. The system then recommends the user to adjust his or her walking speed (the on-path mode) or
to take an alternative path around pedestrians (the o�-path mode). We implemented tactile and audio interfaces
and conducted a user study with 14 blind participants. The results revealed that blind users could successfully
avoid pedestrians using both interfaces; the tactile interface for the o�-path mode guided blind participants
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •85:19

signi�cantly faster than the audio interface could; and the sound-based audio interface was easier for recognizing
alerts than the vibration-based tactile interface. Overall, blind participants believed that a tactile interface could bee�ective because it did not block ambient sound. In future work, we will further research new tactile interfaces toeliminate the weakness of vibration alerts by focusing on shape-changing interfaces. We also plan to collaborate
with orientation and mobility (O&M) communities for building new O&M methodologies with technologies by
sharing our results and experiences.

ACKNOWLEDGMENTS
We would like to thank all participants who took part in our user study. We would also like to thank theanonymous reviewers for their helpful comments. This work was supported by JSPS KAKENHI Grant Number
JP20J23018, Grant-in-Aid for Young Scientists (Early Bird, Waseda Research Institute for Science and Engineering),
and JST-Mirai Program Grant Number JPMJMI19B2.

REFERENCES[1] Shikha Agarwal, Abir Mullick, and GG Ray. 2017. Way�nding Challenges and Strategies in Mumbai Suburban Railway. In Proc.
International Conference on Research into Design (ICoRD ’17). Springer, 161–172. https://doi.org/10.1007/978-981-10-3518-0_14[2] Dragan Ahmetovic, Federico Avanzini, Adriano Baratè, Cristian Bernareggi, Gabriele Galimberti, Luca A Ludovico, Sergio Mascetti,
and Giorgio Presti. 2019. Soni�cation of Rotation Instructions to Support Navigation of People with Visual Impairment. In Proc. IEEE
International Conference on Pervasive Computing and Communications (PerCom ’19). IEEE, 1–10. https://doi.org/10.1109/PERCOM.2019.
8767407[3] Dragan Ahmetovic, Cole Gleason, Chengxiong Ruan, Kris Kitani, Hironobu Takagi, and Chieko Asakawa. 2016. NavCog: a navigational
cognitive assistant for the blind. In Proc. ACM Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’16). ACM, 90–99. https://doi.org/10.1145/2935334.2935361[4] Tomohiro Amemiya, Jun Yamashita, Koichi Hirota, and Michitaka Hirose. 2004. Virtual leading blocks for the deaf-blind: A real-timeway-�nder by verbal-nonverbal hybrid interface and high-density RFID tag space. In Proc. IEEE Conference on Virtual Reality (VR ’04).
IEEE, 165–287. https://doi.org/10.1109/VR.2004.1310070[5] Mauro Avila Soto, Markus Funk, Matthias Hoppe, Robin Boldt, Katrin Wolf, and Niels Henze. 2017. DroneNavigator: Using Leashed andFree-Floating Quadcopters to Navigate Visually Impaired Travelers. In Proc. ACM SIGACCESS Conference on Computers and accessibility(ASSETS ’17). ACM, 300–304. https://doi.org/10.1145/3132525.3132556[6] Shiri Azenkot, Catherine Feng, and Maya Cakmak. 2016. Enabling building service robots to guide blind people: A participatory design
approach. In Proc. ACM/IEEE International Conference on Human Robot Interaction (HRI ’16). IEEE Press, 3–10. https://doi.org/10.1109/
HRI.2016.7451727[7] Carryl L Baldwin, Jesse L Eisert, Andre Garcia, Bridget Lewis, Stephanie M Pratt, and Christian Gonzalez. 2012. Multimodal urgency
coding: auditory, visual, and tactile parameters and their impact on perceived urgency. Work 41, Supplement 1 (2012), 3586–3591.
https://doi.org/10.3233/WOR-2012-0669-3586[8] Simon Blessenohl, Cecily Morrison, Antonio Criminisi, and Jamie Shotton. 2015. Improving indoor mobility of the visually impairedwith depth-based spatial sound. In Proc. IEEE International Conference on Computer Vision Workshops (ICCVW ’15). IEEE, 26–34.
https://doi.org/10.1109/ICCVW.2015.62[9] GP Boucek Jr, James E Veitengruber, and Wayne D Smith. 1977. Aircraft alerting systems criteria study. Volume II. Human factors guidelines
for aircraft alerting systems. Technical Report. BOEING COMMERCIAL AIRPLANE CO SEATTLE WA.
[10] Nicholas A Bradley and Mark D Dunlop. 2002. Investigating context-aware clues to assist navigation for visually impaired people. In
Proc. of Workshop on Building Bridges: Interdisciplinary Context-Sensitive Computing, University of Glasgow. 5–10.
[11] Michael Brock and Per Ola Kristensson. 2013. Supporting blind navigation using depth sensing and soni�cation. In Proc. ACM International
Joint Conference on Pervasive and ubiquitous computing adjunct publication (UbiComp ’13). ACM, 255–258. https://doi.org/10.1145/
2494091.2494173[12] John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4–7.
[13] Sakmongkon Chumkamon, Peranitti Tuvaphanthaphiphat, and Phongsak Keeratiwintakorn. 2008. A blind navigation system using
RFID for indoor environments. In Proc. IEEE International Conference on Electrical Engineering/Electronics, Computer, Telecommunications
and Information Technology (ECTI-CON ’08), Vol. 2. IEEE, 765–768. https://doi.org/10.1109/ECTICON.2008.4600543[14] Akansel Cosgun, E Akin Sisbot, and Henrik I Christensen. 2014. Guidance for human navigation using a vibro-tactile belt interface
and robot-like motion planning. In Proc. IEEE International Conference on Robotics and Automation (ICRA ’14). IEEE, 6350–6355.
https://doi.org/10.1109/ICRA.2014.6907796
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:20•

Kayukawa et al.

[15] Karen Duarte, José Cecílio, and Pedro Furtado. 2014. Easily guiding of blind: Providing information and navigation-smartnav. In WirelessInternet. Springer, 129–134. https://doi.org/10.1007/978-3-319-18802-7_18[16] Brian Due and Simon Bierring Lange. 2018. The Moses e�ect: The spatial hierarchy and joint accomplishment of a blind person
navigating. Space and culture 21, 2 (2018), 129–144. https://doi.org/10.1177/1206331217734541[17] Judy Edworthy, Sarah Loxley, and Ian Dennis. 1991. Improving auditory warning design: Relationship between warning sound parametersand perceived urgency. Human factors 33, 2 (1991), 205–231. https://doi.org/10.1177/001872089103300206[18] Navid Fallah, Ilias Apostolopoulos, Kostas Bekris, and Eelke Folmer. 2012. The user as a sensor: navigating users with visual impairments
in indoor spaces using tactile landmarks. In Proc. ACM CHI Conference on Human Factors in Computing Systems (CHI ’12). ACM, 425–432.
https://doi.org/10.1145/2207676.2207735[19] José Faria, Sérgio Lopes, Hugo Fernandes, Paulo Martins, and João Barroso. 2010. Electronic white cane for blind people navigation
assistance. In Proc. IEEE Conference on World Automation Congress (WAC ’10). IEEE, 1–7.
[20] Alexander Fiannaca, Ilias Apostolopoulous, and Eelke Folmer. 2014. Headlock: a wearable navigation aid that helps blind caneusers traverse large open spaces. In Proc. ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’14). ACM, 19–26.
https://doi.org/10.1145/2661334.2661453[21] Vítor Filipe, Filipe Fernandes, Hugo Fernandes, António Sousa, Hugo Paredes, and João Barroso. 2012. Blind navigation support system
based on Microsoft Kinect. Procedia Computer Science 14 (2012), 94–101. https://doi.org/10.1016/j.procs.2012.10.011[22] Dieter Fox, Wolfram Burgard, and Sebastian Thrun. 1997. The dynamic window approach to collision avoidance. IEEE Robotics &Automation Magazine 4, 1 (1997), 23–33. https://doi.org/10.1109/100.580977[23] Georgios Galatas, Christopher McMurrough, Gian Luca Mariottini, and Fillia Makedon. 2011. eyeDog: an assistive-guide robot for thevisually impaired. In Proc. ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA ’11). ACM,
58. https://doi.org/10.1145/2141622.2141691[24] Thomas Gallagher, Elyse Wise, Binghao Li, Andrew G Dempster, Chris Rizos, and Euan Ramsey-Stewart. 2012. Indoor positioning
system based on sensor fusion for the blind and visually impaired. In Proc. IEEE International Conference on Indoor Positioning and Indoor
Navigation (IPIN ’12). IEEE, 1–9. https://doi.org/10.1109/IPIN.2012.6418882[25] Giuseppe Ghiani, Barbara Leporini, and Fabio Paternò. 2009. Vibrotactile feedback to aid blind users of mobile guides. Journal of Visual
Languages & Computing 20, 5 (2009), 305–317. https://doi.org/10.1016/j.jvlc.2009.07.004[26] João Guerreiro, Daisuke Sato, Saki Asakawa, Huixu Dong, Kris M. Kitani, and Chieko Asakawa. 2019. CaBot: Designing and Evaluating
an Autonomous Navigation Robot for Blind People. In Proc. ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’19).
ACM, 68–82. https://doi.org/10.1145/3308561.3353771[27] Ellen C Haas and Judy Edworthy. 1996. Designing urgency into auditory warnings using pitch, speed and loudness. Computing &Control Engineering Journal 7, 4 (1996), 193–198. https://doi.org/10.1049/cce:19960407[28] Wolfgang Hess, Damon Kohler, Holger Rapp, and Daniel Andor. 2016. Real-time loop closure in 2D LIDAR SLAM. In Proc. IEEE Conferenceon Robotics and Automation (ICRA ’16). IEEE, 1271–1278. https://doi.org/10.1109/ICRA.2016.7487258[29] Juan David Hincapié-Ramos and Pourang Irani. 2013. CrashAlert: enhancing peripheral alertness for eyes-busy mobile interaction while
walking. In Proc. ACM CHI Conference on Human Factors in Computing Systems (CHI ’13). ACM, 3385–3388. https://doi.org/10.1145/
2470654.2466463[30] Hsieh-Chang Huang, Ching-Tang Hsieh, and Cheng-Hsiang Yeh. 2015. An indoor obstacle detection system using depth information
and region growth. Sensors 15, 10 (2015), 27116–27141. https://doi.org/10.3390/s151027116[31] Andreas Hub, Joachim Diepstraten, and Thomas Ertl. 2004. Design and development of an indoor navigation and object identi�cation
system for the blind. In Proc. ACM SIGACCESS Conference on Computers and accessibility (ASSETS ’04). ACM, 147–152. https://doi.org/
10.1145/1029014.1028657[32] Kiyohide Ito, Makoto Okamoto, Junichi Akita, Tetsuo Ono, Ikuko Gyobu, Tomohito Takagi, Takahiro Hoshi, and Yu Mishima. 2005.
CyARM: an alternative aid device for blind persons. In Proc. ACM CHI Extended Abstracts on Human Factors in Computing Systems (CHI’05). ACM, 1483–1488. https://doi.org/10.1145/1056808.1056947[33] Rudolph Emil Kalman. 1960. A new approach to linear �ltering and prediction problems. Journal of basic Engineering 82, 1 (1960), 35–45.
https://doi.org/10.1115/1.3662552[34] Seita Kayukawa, Keita Higuchi, João Guerreiro, Shigeo Morishima, Yoichi Sato, Kris Kitani, and Chieko Asakawa. 2019. BBeep: A Sonic
Collision Avoidance System for Blind Travellers and Nearby Pedestrians. In Proc. ACM CHI Conference on Human Factors in Computing
Systems (CHI ’19). ACM, 52:1–52:12. https://doi.org/10.1145/3290605.3300282[35] Jee-Eun Kim, Masahiro Bessho, Shinsuke Kobayashi, Noboru Koshizuka, and Ken Sakamura. 2016. Navigating visually impaired travelers
in a large train station using smartphone and bluetooth low energy. In Proc. ACM Annual Symposium on Applied Computing (AC ’16).
ACM, 604–611. https://doi.org/10.1145/2851613.2851716[36] Andrew J Kolarik, Silvia Cirstea, Shahina Pardhan, and Brian CJ Moore. 2014. A summary of research investigating echolocation abilitiesof blind and sighted humans. Hearing research 310 (2014), 60–68. https://doi.org/10.1016/j.heares.2014.01.010

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians •85:21[37] Harold W Kuhn. 1955. The Hungarian method for the assignment problem. Naval research logistics quarterly 2, 1-2 (1955), 83–97.
https://doi.org/10.1007/978-3-540-68279-0_2[38] Vladimir Kulyukin, Chaitanya Gharpure, John Nicholson, and Grayson Osborne. 2006. Robot-assisted way�nding for the visually
impaired in structured indoor environments. Autonomous Robots 21, 1 (2006), 29–41. https://doi.org/10.1007/s10514-006-7223-8[39] Vladimir Kulyukin, Chaitanya Gharpure, John Nicholson, and Sachin Pavithran. 2004. RFID in robot-assisted indoor navigation for
the visually impaired. In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS ’04), Vol. 2. IEEE, 1979–1984.
https://doi.org/10.1109/IROS.2004.1389688[40] Bridget A Lewis, Jesse L Eisert, and Carryl L Baldwin. 2014. E�ect of tactile location, pulse duration, and interpulse interval on perceived
urgency. Transportation research record 2423, 1 (2014), 10–14. https://doi.org/10.3141/2423-02[41] B Li, JP Muñoz, X Rong, Q Chen, J Xiao, Y Tian, A Arditi, and M Yousuf. 2019. Vision-based Mobile Indoor Assistive Navigation Aid for
Blind People. IEEE transactions on mobile computing 18, 3 (2019), 702–714. https://doi.org/10.1109/TMC.2018.2842751[42] Bing Li, J Pablo Munoz, Xuejian Rong, Jizhong Xiao, Yingli Tian, and Aries Arditi. 2016. ISANA: wearable context-aware indoor assistive
navigation with obstacle avoidance for the blind. In Proc. IEEE European Conference on Computer Vision (ECCV ’16). Springer, 448–462.
https://doi.org/10.1007/978-3-319-48881-3_31[43] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.
2014. Microsoft coco: Common objects in context. In Proc. European Conference on Computer Vision (ECCV ’14). Springer, 740–755.
https://doi.org/10.1007/978-3-319-10602-1_48[44] Roberto Manduchi, Sri Kurniawan, and Homayoun Bagherinia. 2010. Blind guidance using mobile computer vision: A usability study. In
Proc. ACM SIGACCESS Conference on Computers and accessibility (ASSETS ’10). ACM, 241–242. https://doi.org/10.1145/1878803.1878851[45] Dawn C Marshall, John D Lee, and P Albert Austria. 2007. Alerts for in-vehicle information systems: Annoyance, urgency, andappropriateness. Human factors 49, 1 (2007), 145–157. https://doi.org/10.1518/001872007779598145[46] Edward Marshall and Sue Baker. 1995. Alarms in nuclear power plant control rooms: current approaches and future design. In Human
factors in alarm design. Taylor & Francis, Inc., 183–191. https://doi.org/10.1201/9780203481714-22[47] Manuel Martinez, Angela Constantinescu, Boris Schauerte, Daniel Koester, and Rainer Stiefelhagen. 2014. Cognitive evaluation of haptic
and audio feedback in short range navigation tasks. In Proc. International Conference on Computers for Handicapped Persons (ICCHP ’14).
Springer, 128–135. https://doi.org/10.1007/978-3-319-08599-9_20[48] Christina Meredith and Judy Edworthy. 1995. Are there too many alarms in the intensive care unit? An overview of the problems.
Journal of advanced nursing 21, 1 (1995), 15–20. https://doi.org/10.1177/001872089303500408[49] Mehdi Moussaïd, Dirk Helbing, and Guy Theraulaz. 2011. How simple rules determine pedestrian behavior and crowd disasters. Proc. of
the National Academy of Sciences 108, 17 (2011), 6884–6888. https://doi.org/10.1073/pnas.1016507108[50] Madoka Nakajima and Shinichiro Haruyama. 2012. Indoor navigation system for visually impaired people using visible light communication and compensated geomagnetic sensing. In Proc. IEEE International Conference on Communications in China (ICCC ’12). IEEE,
524–529. https://doi.org/10.1109/ICCChina.2012.6356940[51] Japan Federation of the Visually Impaired. 2013. Research Project Report for Developing Assistive Systems that Ensure theSafety of the Visually Impaired People in Out-door Spaces (in Japanese).
http://nichimou.org/wp-content/uploads/2014/02/
9a�1483649e8747de10bfbd5c8bca55.pdf[52] Helen Petrie, Valerie Johnson, Thomas Strothotte, Andreas Raab, Ste� Fritz, and Rainer Michel. 1996. MoBIC: Designing a travel aid forblind and elderly people. The Journal of Navigation 49, 1 (1996), 45–52. https://doi.org/10.1145/191028.191051[53] Huy-Hieu Pham, Thi-Lan Le, and Nicolas Vuillerme. 2016. Real-time obstacle detection system in indoor environment for the visually
impaired using microsoft kinect sensor. Journal of Sensors 2016 (2016), 1–13. https://doi.org/10.1155/2016/3754918[54] Ioannis Politis, Stephen A Brewster, and Frank Pollick. 2014. Evaluating multimodal driver displays under varying situational urgency. In
Proc. ACM CHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, 4067–4076. https://doi.org/10.1145/2556288.2556988[55] Abishai Polus, Joseph L Schofer, and Ariela Ushpiz. 1983. Pedestrian �ow and level of service. Journal of transportation engineering 109,
1 (1983), 46–56. https://doi.org/10.1061/(ASCE)0733-947X(1983)109:1(46)[56] Stephanie M Pratt, Bridget A Lewis, BN Penaranda, Daniel M Roberts, Christian Gonzalez, and Carryl L Baldwin. 2012. Perceived
urgency scaling in tactile alerts. In Proc. of the Human Factors and Ergonomics Society Annual Meeting. SAGE Publications Sage CA: Los
Angeles, CA, 1303–1306. https://doi.org/10.1177/1071181312561378[57] Giorgio Presti, Dragan Ahmetovic, Mattia Ducci, Cristian Bernareggi, Luca Ludovico, Adriano Baratè, Federico Avanzini, and Sergio
Mascetti. 2019. WatchOut: Obstacle soni�cation for people with visual impairment or blindness. In Proc. ACM SIGACCESS Conference on
Computers and Accessibility (ASSETS ’19). ACM, 402–413. https://doi.org/10.1145/3308561.3353779[58] Morgan Quigley, Ken Conley, Brian P. Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y. Ng. 2009. ROS: an
open-source Robot Operating System. In ICRA Workshop on Open Source Software. IEEE, 5.
[59] Lisa Ran, Sumi Helal, and Steve Moore. 2004. Drishti: an integrated indoor/outdoor blind navigation system and service. In Proc.
the Second IEEE Annual Conference on Pervasive Computing and Communications (PerCom ’04). IEEE, 23–30. https://doi.org/10.1109/
PERCOM.2004.1276842
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.

85:22•

Kayukawa et al.

[60] Kyle Rector, Rachel Bartlett, and Sean Mullan. 2018. Exploring Aural and Haptic Feedback for Visually Impaired People on a Track:
A Wizard of Oz Study. In Proc. ACM SIGACCESS Conference on Computers and accessibility (ASSETS ’18). ACM, 295–306. https://doi.org/10.1145/3234695.3236345[61] Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018), 1–6.
[62] David A Ross and Bruce B Blasch. 2000. Wearable interfaces for orientation and way�nding. In Proc. ACM SIGACCESS Conference on
Computers and accessibility (ASSETS ’00). ACM, 193–200. https://doi.org/10.1145/354324.354380[63] Shadan Sadeghian Borojeni, Susanne CJ Boll, Wilko Heuten, Heinrich H Bültho�, and Lewis Chuang. 2018. Feel the Movement: RealMotion In�uences Responses to Take-over Requests in Highly Automated Vehicles. In Proc. ACM CHI Conference on Human Factors in
Computing Systems (CHI ’18). ACM, 246. https://doi.org/10.1145/3173574.3173820[64] Manaswi Saha, Alexander J. Fiannaca, Melanie Kneisel, Edward Cutrell, and Meredith Ringel Morris. 2019. Closing the Gap: Designing
for the Last-Few-Meters Way�nding Problem for People with Visual Impairments. In Proc. ACM SIGACCESS Conference on Computers
and Accessibility (ASSETS ’19). ACM, 222–235. https://doi.org/10.1145/3308561.3353776[65] Daisuke Sato, Uran Oh, João Guerreiro, Dragan Ahmetovic, Kakuya Naito, Hironobu Takagi, Kris M Kitani, and Chieko Asakawa. 2019.
NavCog3 in the wild: Large-scale blind indoor navigation assistant with semantic features. ACM Transactions on Accessible Computing(TACCESS) 12, 3 (2019), 14. https://doi.org/10.1145/3340319[66] Daisuke Sato, Uran Oh, Kakuya Naito, Hironobu Takagi, Kris Kitani, and Chieko Asakawa. 2017. Navcog3: An evaluation of a smartphonebased blind indoor navigation assistant with semantic features in a large-scale environment. In Proc. ACM SIGACCESS Conference on
Computers and Accessibility (ASSETS ’17). ACM, 270–279. https://doi.org/10.1145/3132525.3132535[67] Shraga Shoval, Johann Borenstein, and Yoram Koren. 1998. The Navbelt-A computerized travel aid for the blind based on mobile robotics
technology. IEEE Transactions on Biomedical Engineering 45, 11 (1998), 1376–1386. https://doi.org/10.1109/10.725334[68] Shraga Shoval, Johann Borenstein, and Yoram Koren. 1998. The Navbelt-A computerized travel aid for the blind based on mobile robotics
technology. IEEE Transactions on Biomedical Engineering 45, 11 (1998), 1376–1386. https://doi.org/10.1109/10.725334[69] Adam J Spiers and Aaron M Dollar. 2016. Design and evaluation of shape-changing haptic interfaces for pedestrian navigation assistance.
IEEE transactions on haptics 10, 1 (2016), 17–28. https://doi.org/10.1109/TOH.2016.2582481[70] Adam J Spiers, Janet Van Der Linden, Sarah Wiseman, and Maria Oshodi. 2018. Testing a Shape-Changing Haptic Navigation Device
With Vision-Impaired and Sighted Audiences in an Immersive Theater Setting. IEEE Transactions on Human-Machine Systems 48, 6(2018), 614–625. https://doi.org/10.1109/THMS.2018.2868466[71] Susumu Tachi, Kazuo Tanie, Kiyoshi Komoriya, and Minoru Abe. 1985. Electrocutaneous communication in a guide dog robot (MELDOG).
IEEE Transactions on Biomedical Engineering BME-32, 7 (1985), 461–469. https://doi.org/10.1109/TBME.1985.325561[72] Yu-Chih Tung and Kang G Shin. 2018. Use of Phone Sensors to Enhance Distracted Pedestrians’ Safety. IEEE Transactions on Mobile
Computing 17, 6 (2018), 1469–1482. https://doi.org/10.1109/TMC.2017.2764909[73] Iwan Ulrich and Johann Borenstein. 2001. The GuideCane-applying mobile robot technologies to assist the visually impaired. IEEE
Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans 31, 2 (2001), 131–136. https://doi.org/10.1109/3468.911370[74] Hsueh-Cheng Wang, Robert K Katzschmann, Santani Teng, Brandon Araki, Laura Giarré, and Daniela Rus. 2017. Enabling independentnavigation for visually impaired people through a wearable vision-based feedback system. In Proc. IEEE International Conference on
Robotics and Automation (ICRA ’17). IEEE, 6533–6540. https://doi.org/10.1109/ICRA.2017.7989772[75] Weimin Wang, Ken Sakurada, and Nobuo Kawaguchi. 2017. Re�ectance Intensity Assisted Automatic and Accurate Extrinsic Calibrationof 3D LiDAR and Panoramic Camera Using a Printed Chessboard. Remote Sensing 9, 8 (2017), 851. https://doi.org/10.3390/rs9080851[76] William R Wiener, Richard L Welsh, and Bruce B Blasch. 2010. Foundations of orientation and mobility. Vol. 1. American Foundation for
the Blind.
[77] Je� Wilson, Bruce N Walker, Je�rey Lindsay, Craig Cambias, and Frank Dellaert. 2007. Swan: System for wearable audio navigation. In
IEEE international symposium on wearable computers (ISWC ’07). IEEE, 91–98. https://doi.org/10.1109/ISWC.2007.4373786[78] Koji Yatani, Nikola Banovic, and Khai Truong. 2012. SpaceSense: representing geographical information to visually impaired peopleusing spatial tactile feedback. In Proc. ACM CHI Conference on Human Factors in Computing Systems (CHI ’12). ACM, 415–424. https://doi.org/10.1145/2207676.2207734[79] Cang Ye, Soonhac Hong, Xiangfei Qian, and Wei Wu. 2016. Co-robotic cane: A new robotic navigation aid for the visually impaired.
IEEE Systems, Man, and Cybernetics Magazine 2, 2 (2016), 33–42. https://doi.org/10.1109/MSMC.2015.2501167[80] Chris Yoon, Ryan Louie, Jeremy Ryan, MinhKhang Vu, Hyegi Bang, William Derksen, and Paul Ruvolo. 2019. Leveraging Augmented
Reality to Create Apps for People with Visual Disabilities: A Case Study in Indoor Navigation. In Proc. ACM SIGACCESS Conference on
Computers and Accessibility (ASSETS ’19). ACM, 210–221. https://doi.org/10.1145/3308561.3353788[81] Limin Zeng, Markus Simros, and Gerhard Weber. 2017. Camera-based mobile electronic travel aids support for cognitive mapping of
unknown spaces. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services(MobileHCI ’17). ACM, 8. https://doi.org/10.1145/3098279.3098563

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 3, Article 85. Publication date: September 2020.