CHI 2018 Paper

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

Enabling People with Visual Impairments to Navigate
Virtual Reality with a Haptic and Auditory Cane Simulation
Yuhang Zhao1, 2, Cynthia L. Bennett1, 3, Hrvoje Benko1, Edward Cutrell1, Christian Holz1,
Meredith Ringel Morris1, Mike Sinclair1
1
Microsoft Research, Redmond 2Information Science, Cornell 3Computer Science & Engineering,{benko, cutrell, cholz, merrie,Tech, Cornell University
University of Washington
sinclair}@microsoft.com
yz769@cornell.edu
bennec3@uw.edu

Figure 1. (A) A blind user wearing the gear for our VR evaluation, including a VR headset and Canetroller, our haptic VR controller.
(B) The mechanical elements of Canetroller. (C) Overlays of the virtual scene atop the real scene show how the virtual cane extends
past the tip of the Canetroller device and can interact with the virtual trash bin. (D) The use of Canetroller to navigate a virtual street
crossing: the inset shows the physical environment, while the rendered image shows the corresponding virtual scene.
Note that users did not have any visual feedback when using our VR system. The renderings are shown here for clarity.
ABSTRACT

Author Keywords

Traditional virtual reality (VR) mainly focuses on visualfeedback, which is not accessible for people with visual
impairments. We created Canetroller, a haptic cane
controller that simulates white cane interactions, enabling
people with visual impairments to navigate a virtualenvironment by transferring their cane skills into the virtualworld. Canetroller provides three types of feedback: (1)physical resistance generated by a wearable programmablebrake mechanism that physically impedes the controller
when the virtual cane comes in contact with a virtual object;(2) vibrotactile feedback that simulates the vibrations whena cane hits an object or touches and drags across various
surfaces; and (3) spatial 3D auditory feedback simulating
the sound of real-world cane interactions. We designed
indoor and outdoor VR scenes to evaluate the effectivenessof our controller. Our study showed that Canetroller was apromising tool that enabled visually impaired participants
to navigate different virtual spaces. We discuss potential
applications supported by Canetroller ranging fromentertainment to mobility training.

Virtual reality; white cane; blindness; visual impairments;
haptic feedback; auditory feedback; mobility.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights forcomponents of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, topost on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
CHI 2018, April 21–26, 2018, Montreal, QC, Canada© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-5620-6/18/04…$15.00
https://doi.org/10.1145/3173574.3173690

Paper 116

ACM Classification Keywords

H.5.1. Information interfaces and presentation: Multimedia
Information Systems. K.4.2. Computers and Society:
Assistive technologies for persons with disabilities.
INTRODUCTION

Virtual reality (VR) technologies are maturing quickly andhave been widely applied to different fields by researchersand designers, such as entertainment [6,61], education[32,54], and social activities [62]. Researchers haverecognized VR’s potential in accessibility and developed
various VR systems for training and rehabilitation forpeople with different disabilities [25], such as dyslexia[17,21], stroke [14,33], and ADHD [1,34].
While presenting great potential in a variety of use cases,
current VR solutions rely mainly on realistic visualfeedback to provide an immersive experience to sighted
people, for whom the visual sense is dominant [38]. Most
VR applications are not accessible for people with visual
impairments (VIPs), preventing them from benefiting fromthis important class of emerging technologies [28].
Some prior research has explored VR experiences for VIPs.
However, they either generated pure auditory virtual realitythat has limited capability in describing the shape or
specific layout of a virtual space [28,36], or provided

Page 1CHI 2018 Paper

simple haptic feedback with a joystick or a controller
grounded by a robot arm that can only be used in astationary situation (e.g., [10,26,35]). To our knowledge, no
VR experience has been created for VIPs that enables themto walk around the VR environment and achieve a goodcomprehension of the 3D virtual space.
To address this problem, we created Canetroller (see Figure1), a wearable haptic controller that enables VIPs tophysically navigate a VR world, thus providing them amore immersive VR experience. Since the white cane is themost common tool for blind people to navigate the realworld, our controller simulates many white caneinteractions, which allows users to transfer their cane skill
from the real world into the virtual world. We describe thedesign of Canetroller, including a formative study withwhite cane users and orientation and mobility (O&M)
instructors that inspired the design, a discussion of ourhaptic controller and VR system, and an evaluation with
VIPs. Our results indicated that VIPs could successfully
navigate and understand virtual scenes using Canetroller.
We close by discussing potential application scenarios.
RELATED WORK
Auditory VR for People who are Blind

Most prior research in this area has focused on creating
auditory VR for VIPs. Audio-based environment simulator(AbES) games allow blind users to interact with the virtualworld through auditory feedback, e.g., AudioDOOM [41],
AudioMetro [40], and AudioZelda [37]. Studies showedthat AbES games enhanced people’s mobility skills whennavigating the real-world space that was mapped by avirtual game [31]. Trewin et al. [50] further enhanced theaccessibility of VR games by providing descriptions of the
VR scenes and generating extra sound effects like footsteps.
They also supported special commands such as auto-aim
and auto-walk, where the system automatically completed
an aiming and walking task and provided audio feedback
indicating the current game status.
Purely acoustic VR solutions [11,12,44,48,49] induce
presence and render virtual obstacles with sound rendering.
For example, González-Mora et al. [12] created an acoustic
VR space with a portable system that generated spatial
audio to create the illusion that surrounding objects werecovered with speakers emitting sounds depending on their
physical characteristics (e.g., color, texture).
Several audio VR systems also incorporated VIPs’ realworld navigation strategies. For example, Picinali et al. [36]created an acoustic VR system that simulated the use ofecholocation (the use of sound reflecting of objects tolocalize them) to help users better understand the virtual
space. Virtual-EyeCane [28] used an acoustic-only whitecane for interacting in VR. A user pointed the virtual cane
to different objects and received audio feedback withdifferent frequencies representing the distance to the virtual
objects. In contrast to these audio-only solutions, our work

Paper 116

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

combines both haptic and audio feedback to generate amore realistic VR experience.
Haptic VR for VIPs

Haptic sensation via white canes or body touch is the main
channel for blind people to understand a physical space[13], with audio serving as a channel for complementary
information. When navigating the real world, the whitecane is commonly used for low-resolution scanning of theenvironment and to detect obstacles and identify distances.
Similarly, legs and feet are used to feel the ground’s
surface, and the palm or fingers are used to precisely
recognize the shape and texture of objects [13,22].
Prior research explored using haptic interfaces to help blind
people navigate virtual environments. Although someresearchers presented the virtual world in 2D and usedpassive tactile representations to provide the thirddimension information of a virtual object or space (e.g.,[18,20,27,56,60]), recent research has focused on creating
3D haptic virtual spaces using various desktop-mounted
force feedback devices [9,16,22,35,45,57], such as
PHANToM [63], Novint Falcon [64], and Geomagic Touch[65]. For example, MoVE [45] enables a blind user to toucha scaled virtual version of a space by using two PHANToM
force-feedback devices with her thumb and index finger.
Lahav and Mioduser [22,23] created a virtual environmentthat allowed blind users to get vibrations andattraction/rejection force feedback when navigating thespace with a joystick. Jansson et al. [16] also renderedvirtual textures and simple 3D objects with PHANToM andgrooved surfaces with Impulse Engine 3000TM. However,none of these systems allowed for free movement in spaceas all of them rendered a scaled model that needed to be
explored via a probe or stylus. These scaled representations
made it difficult for users to understand the virtual space[45] and build a consistent mental model [9].
To enable a more intuitive exploration of the virtual space,other researchers have simulated aspects of white caneinteraction in virtual reality [26,42,51,52]. For example,
BlindAid [42] used PHANToM to simulate the force
feedback generated by a white cane during navigation.
HOMERE [26] is a multimodal VR system in which a cane
controller connected to a robot arm was used to control thevirtual cane and provide haptics of collision and texture.
Although they simulated haptic aspects of cane interaction,these systems used a grounded controller, which prevented
mobile navigation of the VR space.
The closest solution to ours is the work by Tzovaras et al.
[51], which enabled a mobile VR experience by using a
CyberGrasp haptic glove to generate force feedback to ablind user’s fingers, providing the illusion that they arenavigating the virtual space with a cane. The researchers
further improved the system by asking the user to hold areal cane [52], with which the system can detect the user’s
grasp posture and provide corresponding force feedback
through CyberGrasp. The system also provided 3D audio

Page 2

CHI 2018 Paper

feedback. Although an evaluation showed that the system
was useful for VIPs to navigate the virtual space, thecurrent design could not prevent a user from penetrating
virtual objects, which left much room for improvement.
Moreover, no thorough analyses of the users’ feedback
were reported, making it difficult to assess the benefits andweaknesses of the system design.
Our work addresses the problems involved in prior work,by creating an immersive VR experience that enables
mobile VIPs to navigate a real-world-scale virtual space
using their cane skills coupled with spatial audio. Further, abrake mechanism on our cane controller addresses theobject-penetration problem present in prior work.
“Smart Canes” in (non-virtual) Reality

Researchers have also augmented a physical white cane toimprove users’ navigation experience in the physical (nonvirtual) world. Some have explored equipping real caneswith sensors to detect far-away obstacles (out of the reachof a normal cane) and providing audio or vibration
feedback to the user [29,30,46,55]. Others have added
motors to the cane, providing navigation cues on the canethrough force feedback to assist a visually impaired user toavoid obstacles and navigate more safely and efficiently[8,53]. Though some smart canes are commerciallyavailable for purchase, they have not been widely adopted,
primarily because people need to know their cane will work
100% of the time and will never run out of batteries,
experience a software crash etc. In contrast to this prior
work, we do not enhance or replace the regular white caneused in the physical world, but focus on making VR
accessible to VIPs using a novel controller that mimics
white cane interactions.
The smart cane that is most related to our work is thecontactless haptic cane [2–5], a short version of a whitecane with ultrasound sensors and actuators. This cane did
not physically contact objects but still provided users ahaptic sensation similar to a real cane. This design could
reduce the possible inconvenient environmental interactions
caused by cane contact, which was valuable from a social
perspective. Our VR system adopted the same “contactless”
concept, which simulated the haptic feedback of the caneinteraction, but used a short cane controller to avoid contact
between the controller and the real environment reducing
the effect of the real world on users’ virtual experience.
FORMATIVE STUDY

To guide our design, we relied on two sources: 1) priorpublished work [58,59] that explored the navigation
challenges and strategies for people who are blind, and 2)
our own observations from a formative study with VIPs
who rely on a white cane and with O&M instructors.
Formative Study: Method

We recruited seven VIP participants (3 female) whose ages
ranged from 25 to 40 (mean=32.4) and five O&M
instructors (all females) whose ages ranged from 26 to 49

Paper 116

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

ID
Age/ Sex
Visual Condition
V1
29/F
Ultra-low vision for ~10 years.
1,2
V2
40/F
Blind since birth.
1,2
V3
31/F
Blind since birth.
1,2
V4
34/M
Blind since birth.
1
V5
30/M
Legally blind since birth.
1,2
V6
25/M
Ultra-low vision since birth.
1,2
V7
38/M
Blind. He was legally blind since birth.
2
V8
63/F
Ultra-low vision for 50 years.
2
V9
51/F
Ultra-low vision, for ~5 years.
2
V10
37/M
Low vision, for ~8 years.
Table 1. Demographic information of VIPs. Participantslabeled with “1” participated in the formative study, and thosewith “2” participated in the evaluation study.
1,2

ID
O1
1
O2
1,2
O3
1
O4

VisionYears teaching O&M
Ultra-low vision.
1.5 years
Sighted
2.5 years
Sighted
5.5 years
Mild low vision.
8 years
Not legally blind.
1,2
O5
26/F
Sighted2 years2
O6
38/F
Sighted
14 years
2
O7
26/F
Sighted3 years
Table 2. Demographic information of O&M instructors.
Instructors with “1” participated in the formative study, andthose with “2” participated in the discussion on Canetroller.
1,2Age/ Sex
29/F
28/F
49/F
39/F

(mean=34.2), as shown in Table 1-2. All VIP participants
were cane users. Since V1 (also labeled as O1) was both a
VIP and an O&M instructor, we included the interview
questions for both VIP and O&M instructors in her study.
With the VIPs, we first conducted an interview. We asked
about their understanding of virtual reality and gave them adefinition of VR in case they were not familiar with theconcept. We then asked about their prior VR experienceand their expectations of VR. We also asked about their useof the cane in real life. Then, we asked participants tonavigate with their canes while we observed their canestrategies in five different scenarios. Participants wererequired to think aloud during the navigation. The scenarios
include: (1) the experiment room with desks, chairs, andtrashcans; (2) a corridor from the experiment room to anelevator; (3) an outdoor area surrounded by circular cement
seating platforms; (4) a street crossing area with tactile
domes; (5) a square sand sports pit surrounded by curbs,
with two benches and a metal trashcan outside of the area.
With the O&M instructors, we conducted an interview,
asking them about their prior knowledge of VR applications
for VIPs, the potential of VR, and to describe standard caneskills, training techniques, and the main training scenarios.
We also asked them to demonstrate different cane skills
with a white cane we provided.
Formative Study: Findings
Experience with VR

We found most of our visually impaired participants had
little or no prior experience with virtual reality and none of
the O&M instructors had heard of any VR applications that

Page 3

CHI 2018 Paper

were designed for VIPs. However, participants all showed
great interest in VR and hoped they could be able toexperience VR as sighted people could. As V3 said, “Whenlistening to some VR examples, I wanted to know ‘why
cannot this be for us?’”
Expectations of VR

When asked about their expectations of VR, both VIPparticipants and O&M instructors envisioned VR to be amobility-training platform, where users could simulateroutes or unfamiliar environments to learn cane skills,
prepare for travel, and further build confidence. Some VIPs
wanted to use VR as an educational tool, and some even
hoped that VR would be able to bring them equal access toall visual information that they could never perceive before.
“If VR has a way to describe or make me feel sculptures, or
if it is able to translate whatever everyone can see to tactile,[it would be great]” (V5). Interestingly, V7 wanted to use
VR to build a new identity in which other co-users in VR
would not know that he had any disabilities. Although notwanting his virtual avatar to appear different than others,
V7 believed that a cane-like controller would be useful for
him to navigate the virtual world because it could give him
the simulation of what he was used to.
Cane Strategies

The O&M instructors identified four main cane use styles:
Two-point touch is swinging the cane from side to side andtapping the edges of one’s walking path on either side in anarc slightly wider than one’s shoulder [15]. It is used toprotect both sides of the walker’s body. Most participants
used this strategy when they were walking fast in a
relatively safe or familiar place.
Constant contact is sweeping the cane from side to side andkeeping the cane tip in contact with the surface at all times[19]. This technique provides the most reliable detection ofsubtle surface changes at every point of the arc. Participants
used it to conduct a thorough scanning of the environment.
For example, we found that V1 primarily used two-point
touch when navigating the corridor, but she changed toconstant contact when she was getting close to the stairs.
Trailing is used to search for openings, objects, and surfacechanges along an edge, especially vertical surfaces such asa wall or curbs [66]. A VIP can “keep the cane at base of
the wall, maintain the position, and travel along” (O5). In
the outdoor circular area, V5 trailed along the cement
seating platform and determined the area to be a circle.
Shorelining is another technique that uses touch and drag totravel along an edge. A person can repeatedly tap the edge
she wants travel along, move the cane from the edge to the
opposite side, and then drag the cane along the ground backtoward the edge [66]. Compared with trailing, O&M
instructors recommended shorelining because “it canprotect the other side of your body” (O5). For example, we
observed that V7 explored the square sand sports pit area by
shorelining along the curbs around it.

Paper 116

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

Perceiving the Environment with the Cane

Compared with other navigation tools, such as GPS or
guide dogs, a cane provided VIPs “more detailed
information on what is immediately in front of them in theenvironment” (V7). During our observation, participants
could perceive the shape, texture, material, and even theinterior structure of an object (e.g., hollowness) by using acane. They used both the tactile and auditory feedback fromthe cane to perceive the environment. As V2 described, “weget the information from all the senses.”
We found that participants used the tactile feedback tounderstand the position and shape of an object. For
example, V5 could tell the object in front of him was achair because he felt the shape of the wheels at the base byusing the tactile feedback from the cane. Participants could
also perceive the height of an object according to thecontact point where the cane hit it. Some participants could
perceive the texture by using a cane. V6 described how heperceived the texture of the carpet in the experiment room,“I know the floor is carpet because it feels like there’s
pattern to the carpet. I can feel the lines [on the carpet] withthe cane.” Besides tactile feedback, participants also used
auditory feedback from the cane to determine the materialof an object and even whether it was hollow.
Although some participants used their hands to feel details(e.g., locating elevator buttons) or confirm the information
they perceived from the cane, most participants indicatedthat they avoided using their hands, especially in unfamiliar
places. For example, when V2 explored the sand sports pit
area, she only used her cane to explore the surrounding
objects (benches and trashcan), “See, I would never reachand touch that [trashcan].” The O&M instructors also
recommended using the cane to explore an unknown space
before touching it. This emphasized the importance of acane in perceiving objects in an unfamiliar place.
Target Scenarios for Cane Training

There were four main scenarios that most O&M instructors
emphasized for training students: rooms, hallways, stairs,
and street crossings with curbs and tactile domes. Although
VIPs need to get training in many important scenarios, theinstructors indicated that “funding does not always allow
ideal training amounts” (O4). This emphasized the potentialof VR simulation in O&M training, which could generate
different virtual scenarios to provide users sufficienttraining with a lower demand on resources.
Formative Study: Discussion

The formative study demonstrated VIPs’ strong interest in
virtual reality and its potential in orientation and mobility
training. According to the VIPs and the O&M instructors,
there were different cane skills that people could use tonavigate different environments. For example, a cane user
can use two-point touch to conduct a fast navigation, while
using constant contact to have a thorough scanning of theenvironment. She can also use trailing or shorelining towalk along a wall or a hallway. Our VR system should

Page 4

CHI 2018 Paper

Figure 2. The mechanical parts of Canetroller, including thecane controller users interact with (bottom) and the haptic
mechanism users have to wear to experience feedback (top).

support all these standard cane techniques to enable avisually impaired user to easily navigate the virtual spacewith their existing cane skills. Moreover, VIPs use bothauditory and tactile feedback to perceive objects in theenvironment, suggesting the importance of multimodal
feedback. The sensations our participants described
included: the physical resistance when a cane touches anobject, the tactile feedback when the cane sweeps on
different textures, and the audio feedback when the caneinteracts with the environment. Since VIPs have different
preferences for cane types, we consider customizing thevirtual cane (e.g., the length) based on users’ preferences.
CANETROLLER

Guided by the findings from the formative study, we
designed Canetroller, a haptic cane controller coupled withspatial audio feedback that allows VIPs to use their realworld cane skills to explore different virtual spaces.
Interaction Design

The design of Canetroller was inspired by real-worldinteractions with a white cane. To experience VR, a user
wears a VR headset with earphones to hear 3D audio andholds Canetroller to feel haptic feedback as shown in Figure
1A. Canetroller is a wearable controller that consists of thefive main parts shown in Figure 2: a brake mechanism thata user wears around the waist, the cane controller that theuser holds and sweeps or taps as they would a real cane, aslider between the brake and the controller that imparts ahorizontal physical resistance generated by the brake, avoice coil that generates vibrations to the controller fortactile feedback, and a tracker to monitor the cane
controller’s 3D poses. We designed Canetroller to be
shorter (25 inches) than real canes (36–60 inches) to reduce
the effect of the physical space on the user’s virtual
experience by inadvertently hitting real objects. Although

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

Figure 3. (A) Canetroller in use. (B) The overlaid image of thevirtual cane (brown). Note that the virtual cane can be longer
than Canetroller, usually as long as the VIP’s real cane.

Canetroller is short, the virtual cane it represents in VR canbe arbitrarily long as shown in Figure 3. We adjust thelength of the virtual cane based on the user’s height andhow they hold the cane.
Canetroller provides three types of feedback: (1) Braking:
When the virtual cane hits a virtual object horizontally inthe left-right direction, the brake mechanism generates
physical resistance that stops the controller moving towards
the object. With this feedback, the virtual cane would not
penetrate virtual objects, providing users precise perception
on the boundaries of virtual objects. (2) Vibrotactile: Whenany part of the virtual cane contacts a virtual object (hits anobject or sweeps on a surface), Canetroller generates
vibrotactile feedback to simulate the corresponding hit or
texture vibration that the user would feel in the real world.
(3) Auditory: Our VR system generates spatial 3D audio
feedback that simulates the sound of a real cane interactingwith the real world. Our sound rendering depends on thesurface type and collision speed. With this multi-modal
feedback, our system supports different cane strategies:
Shorelining. When the virtual cane hits a vertical surface,e.g., a virtual wall, in the horizontal left-right direction, theuser can feel the physical resistance that enables them toperceive the boundary of the wall. They can also feel thecontact vibration and hear the 3D sound, which simulate thecollision between the cane and a wall. With this interaction,a blind user can walk along a virtual wall with Canetroller
by using the shorelining technique (Figure 4A).
Two-point Touch. When the virtual cane taps on a surface,
Canetroller generates corresponding vibration to simulate
the tactile sensation. The user can also hear 3D tapping
sound from the VR system. With this interaction, a user canconduct two-point touch on different virtual surfaces suchas carpet, concrete, or tactile domes (Figure 4B).
Constant Contact. We also support constant contact

Figure 4. (A) A user walks along a virtual wall with shorelining. (B) The user uses two-point touch on a piece of virtual carpet.
(C) The user uses constant contact on a piece of virtual tactile dome. (D) The user penetrates the virtual cane into a virtual wall.

Paper 116

Page 5

CHI 2018 Paper

rendered by generating tactile and auditory feedback thatsimulates textures of different surfaces (Figure 4C).
Moreover, since tactile domes found at sidewalk crossings
have bumps that would generate resistance when the canesweeps across, we use the brake to generate shortresistances at each bump, thus providing the user with theillusion that her cane hits the bumps. To make theexperience the most realistic, we modulate the frequency ofthe braking resistance, and the speed and amplitude of thevibrotactile and the 3D audio feedback based on the virtual
texture characteristics and the sweeping speed.
Our current prototype only provides physical resistance inthe horizontal left-right direction. Although the ideal device
would support resistance in all three dimensions,mechanical engineering limitations forced us to only choose
one, or else the device would have been too heavy andbulky to allow ergonomic use. We chose the horizontal
braking based on the formative study, which indicated thehorizontal feedback is the most common in identifying
obstacles and spatial boundaries (e.g., walls) when using
different cane strategies, such as shorelining and constantcontact. We did not leverage the physical floor to provide
haptic feedback in the up-down direction because it would
restrict the topology of simulated environments to only
those that match the room’s properties.
Since our brake cannot control the up-down or forwardbackward direction, and our system cannot limit the user’s
body motions, Canetroller may still penetrate virtual objects
in some circumstances (e.g., when the user pokes
Canetroller forward, or physically moves or rotates herbody), making it difficult for users to understand the exact
boundary of virtual objects or planes. Thus, we generate a“beeping” sound to warn the user if the cane penetrates avirtual object too deeply (e.g., wall, carpet), or if the user
moves outside the virtual space (Figure 4D).
Implementation

To build Canetroller, we combined two pieces of a folding
cane to create a 24-inch cane controller. We used a Placidindustries magnetic particle brake (Model B35) to generate
the physical resistance and a Dayton Audio DAEX25FHE-4
voice coil on the controller to generate the vibration. A
Vive tracker was attached to the controller to track its
position. Our VR system used an HTC Vive headset totrack the user’s head position and orientation, thus generate
3D audio feedback. In typical operation, we do not renderanything to the headset screen, but we can do so fordebugging and demonstration to individuals without visual
impairments. The system was implemented in the Unity
game engine (v. 2017).
Our brake mechanism regards the user’s body as theanchor, allowing a 360-degree rotation. We connect thecenter of the brake and the tip of the cane controller with a
35-inch slider. When a user sweeps the cane controller, thetip of the controller slides along the slider and drags theslider to rotate around the brake. We used a Pololu simple

Paper 116

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

motor controller 18v7 to control the brake. We can makefine-grained adjustments to the flexibility of the brake’s
shaft rotation, thus controlling how easily the user cansweep the cane controller. If the brake is set to 100%, thebrake shaft cannot rotate, so that the user cannot sweep thecane controller without rotating her body. In our VR
system, this is used to render hard body collisions, e.g.,when the virtual cane hits a virtual object horizontally in theleft-right direction. We release the brake when the tracker
detects that the user starts to move the controller back.
When the virtual cane sweeps on the virtual tactile domes,
we set the brake to 30% in a short interval at each bump tosimulate the resistance of hitting the bump.
To generate realistic vibrations upon collision, we attacheda Pololu MinIMU-9 v5 embedded with a LSM6DS33 3-axis
gyro and accelerometer to a white cane and recorded theacceleration data when this real cane came in contact withvarious objects. We used the equipped cane to hit different
objects (e.g., walls, metal tables, plastic trashcans) andsweep on different surfaces (e.g., carpet, concrete, tactile
domes) to record the corresponding accelerations. Whensweeping the cane on different surfaces, we tried to sweep
with a relatively constant speed of 200mm/s to reduce theeffect of hand motions on the recorded acceleration data.
We recoded the data at a sample rate of 1.4kHz.
We processed the acceleration data to generate an audio
signal for input to the voice coil, using a simplified versionof Romano et al.’s method [39]. We used a 10Hz high-passfilter to remove the effect of unwanted forces, such as
gravity and hand motions during data recording. Since thehuman sensitivity to high-frequency vibrations is largely
independent of directions [7], we combined the three-axis
acceleration data into one-dimensional signal using a rootsum square calculation, so our system could work with asingle voice coil. We multiplied the signal with a constantfactor (~0.005) to fit the voice coil’s input range andupsampled it to 44.1kHz, which is a standard audio sample
rate, for final use on the voice coil. We amplified this signal
with a Velleman MK190 amplifier to drive the voice coil,making the vibrations even stronger.
To generate realistic 3D sounds, we attached a microphone
to the cane tip to record the sound when hitting or sweeping
on different surfaces. We removed recording noises withaudio-editing software and then played back the sounds
when the virtual cane produces corresponding interactionsin the virtual world, modulating the sound depending on thevirtual cane’s speed. Unity rendered the 3D sound using theMicrosoft Spatializer Plugin with a generic HRTF.
EVALUATION

We evaluated Canetroller with VIPs. Our goal was toevaluate the effectiveness of Canetroller and investigate
how VIPs perceive different virtual environments by using
the audio-haptic feedback from Canetroller.

Page 6

CHI 2018 Paper

Figure 5. (A) Indoor scenario with a metal table, a trashcan,
and a door. (B) Outdoor scenario of a street crossing withsidewalks, curbs, tactile domes, and a traffic pole.
MethodWe recruited nine participants with visual impairments (5females and 4 males) whose ages ranged from 25 to 63(mean=38.7). All VIPs were cane users. Six of them
participated in both the formative and system evaluation
studies, as shown in Table 1.
Apparatus: Two VR Scenarios

The formative study indicated that both indoor and outdoor
navigation are important scenarios for cane skill training.
Thus, we designed two VR scenarios to evaluate oursystem: an indoor scenario with four walls as the boundary,a carpet on the ground, a closed door, a metal table, and aplastic trashcan (Figure 5A); and an outdoor scenario thatsimulated a street crossing with two cement sidewalks,
curbs, two curb-cut ramps covered in tactile domes,
concrete street, and a metal traffic pole (Figure 5B).
In the outdoor scene, we played 3D environmental sound tosimulate the traffic, enabling participants to distinguishstreet directions. In the real world, VIPs usually push the
button on the traffic pole and listen for the traffic light todecide when to cross. To increase the realism of our VR
scenario, participants pushed the virtual traffic button usinga Vive controller’s trigger button when standing on the
curb-cut ramp. The system then generated a 3D traffic lightsound to inform them when to cross the virtual street.
Procedure

We conducted a one-hour evaluation in a 4.5m x 5m room.
With VIP participants, we first had a brief interview about
their demographic information and their prior experience
with VR. For safety reasons, we asked participants to walkaround the real-world space of the experiment room tofamiliarize them with the real environment. We thenintroduced the hardware setup of our VR system, including
the VR headset and Canetroller, and assisted participants in
putting on the devices. Before experiencing the VR world,
we customized the length of the virtual cane for eachparticipant. We first set the virtual cane to the length of theparticipant’s real cane, and then adjusted it based on how
she held Canetroller. We continued with a tutorial session,
an indoor scenario exploration session, and an outdoor
scenario exploration session.
Tutorial: We started with a 15-minute tutorial session, in
which we introduced all the features that Canetroller
supported, including the three types of feedback, thedifferent cane strategies the system supports, as well as thewarning sound that indicates the virtual boundary. We

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

guided the participants to interact with several
representative virtual elements, including a wall with a door
and two floor materials (i.e., carpet, concrete). We also
asked them to penetrate the virtual cane into the virtual
floor deeply to hear the warning sound. We explained thedifferent feedback and asked them to confirm whether theyfelt the feedback. Participants repeatedly practiced with
Canetroller until they felt familiar with the system.
Indoor Scenario: Participants freely explored the virtualindoor environment with Canetroller. Before they started,
we told them which virtual objects were in the virtual
space, so that they would not struggle guessing what an
object was when it was discovered. Participants were asked
to think aloud, talking about what they felt, what they
heard, what virtual objects they thought they had
discovered, which parts confused them, and the reasons fortheir perception and/or confusion. When they had located
all the virtual objects and felt confident about each object’s
position, we asked them to stand still and point to eachobject relative to their current position with their finger. We
recorded whether they correctly pointed out each object.
Outdoor Scenario: Participants needed to cross the virtualstreet using Canetroller. All participants started their
exploration at the same location on the left sidewalk, facing
the traffic pole (Figure 5B). We first informed them thatthey were on a sidewalk and asked them to identify the
position of the street and the direction of the traffic basedon the 3D traffic sound. If they did not identify it correctly,
we told them that the street was on their right side, so they
could understand in which direction they should cross thestreet. We then asked them to walk forward along the
sidewalk, locate the tactile domes and the traffic pole,trigger the traffic light sound with the Vive controller, andfinally cross the virtual street. They also needed to identify
the sidewalk on the other side to confirm that they had
successfully crossed the street. Participants thought aloud
during the whole process, describing how they identified
different virtual objects and crossed the street.
At the end of both the indoor and outdoor scenarios, we
asked participants to score eight questions from a presencequestionnaire from 1 to 7 (Table 3), evaluating how realistic
the VR experience was. We derived this questionnaire fromthe Igroup Presence Questionnaire (IPQ) [43] and removed
vision-related questions. Since participants needed to think
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8

In the VR world I had a sense of “being there”.
Somehow I felt that the virtual world surrounded me (1 is fully
disagree, 7 is fully agree).
I felt present in the virtual space.
I was completely captivated by the virtual world.
How real did the virtual world seem to you (1 is not real at all, 7
is completely real)?
How much did your experience in the virtual environment seemconsistent with your real-world experience (1 is not consistent at
all, 7 is very consistent)?
How real did the virtual world seem to you (1 is about as real as
imagined world, 7 is indistinguishable from the real world)?
The virtual world seemed more realistic than the real world.

Table 3. Modified presence questionnaire (adapted from [43]).

Paper 116

Page 7

CHI 2018 Paper

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

aloud and answer our questions about their VR experienceduring the exploration, we removed questions on awarenessof the real world from the questionnaire as well.
We finished the user study with an interview, asking
participants about their impressions of each type of
feedback, the most important feedback for them whenperceiving the virtual space, how they would improve thesystem, and the potential applications they thought
Canetroller may enable.
Results

Our study demonstrated that Canetroller enabled VIPs to
explore and navigate the VR world.
Effectiveness of Canetroller

In the indoor scenario, all participants could locate andidentify all the virtual objects (the door, the metal table, andthe trashcan) using Canetroller. Eight out of nine
participants correctly pointed out the locations of all theobjects at the end of the indoor session, indicating thatparticipants could build accurate mental models of the 3D
virtual space with Canetroller. V6 said, “This is cool,because it does let you create a good physical mapping on
your space.” V3 was the only one who did not fully
understand the virtual room layout. Although she pointed
out the virtual door and table correctly, she built a linear
mental model, saying the trashcan was in the same line withthe door and table.
In the outdoor scenario, six participants could distinguishthe traffic direction, locate the tactile domes and traffic
light, and successfully cross the virtual street with little
assistance from the experimenter. However, the other threeparticipants (V2, V4, V8) had difficulty distinguishing thetraffic direction through the 3D traffic sound. Although
they correctly identified most virtual elements (e.g., thetactile domes, the traffic light), they could not decide inwhich direction to cross the street without assistance.
Sense of Presence in the Virtual WorldIn accordance with the standard presence data analysis for
IPQ [67], we evaluated the average Spatial Presence (SP:Q2, Q3), Involvement (INV: Q4), Experienced Realism(REAL: Q5, Q6, Q7, Q8), and a general item that assesses
the general “sense of being there” (G: Q1) for both indoor
and outdoor VR scenarios based on participants’ scores in
the presence questionnaire. Figure 6 shows the averagescores to SP, INV, and REAL on three axes, and the generalitem G as a bow on the left. We found that participants had
a high spatial presence in both the indoor and outdoor VR
scenarios, indicating that VIP users could use Canetroller tophysically experience the 3D virtual space.
We compared participants’ presence data between theindoor and outdoor scenarios, finding no significant
differences for any variables with Wilcoxon Signed-Rank
Tests. Participants’ interview data echoed this result,showing that their preferences varied when comparing thetwo VR scenarios. Four participants (V1, V2, V8, V10) felt

Paper 116

Figure 6. Radar diagram that shows the participants’ average
presence data in the indoor and outdoor VR scenarios.

the indoor scene was more realistic, four participants (V3,V4, V6, V9) had a more immersive experience in theoutdoor scene, while V7 did not have a preference.
Multi-Modal Feedback from Canetroller

Braking Feedback. All participants felt that the physicalresistance helped them understand the boundary of virtual
objects. V9 felt excited about feeling the physical resistancewhen hitting the virtual wall: “I feel [the virtual cane] hit
the wall! It feels like you are hitting up against something. I
can tell the direction of this wall. Neat!” With the physicalresistance, V7 could even distinguish the shape of thevirtual trashcan by tapping around it with Canetroller:“Maybe this is the trashcan. It seems like a squarish type.”
Vibrotactile Feedback. Most participants liked thevibrotactile feedback and used it to perceive texture. As V2
mentioned, “I feel the vibration different. I think [thesimulation of] the texture is really neat, to distinguishbetween carpet and concrete.” V1 and V6 specifically liked
the feeling of sweeping on the virtual tactile domes, whichwere simulated by both the vibrotactile and the physicalresistance. As V6 explained, “The feeling is reallyconvincing. [I felt] I was just touching it. As you sweep
through them you feel each individual bump.” Someparticipants also noticed the change of vibration when theychanged the speed of sweeping Canetroller. “When I swept
it really fast one time, it vibrated much more” (V10).
Auditory Feedback. Participants relied on the auditory
feedback to distinguish the materials, thus helping themdecide what a virtual object could be. For example, V4 usedthe 3D audio to identify different virtual elements in theoutdoor scene: “This is the traffic light because it’s metallic
sounding. And the street is on the right because I hear theconcrete.” However, some participants had difficulty
distinguishing some specific sounds, which resulted introuble identifying corresponding virtual objects. For
example, in the indoor scene, V3 could not distinguish thesound difference between the wall and the door, so it took
her a long time to finally identify the virtual door. As she
mentioned, “[The sound] really did not feel that muchdifferent. That [difference] should be like stronger.”
Dominant Modality. We asked participants which feedback
was most critical for them to perceive different information

Page 8

CHI 2018 Paperin the virtual space. Participants’ preferences varied. Twoparticipants (V6, V9) felt that the physical resistance was
the most important because it enabled them to understandthe boundaries and build a good sense of the virtual space.
“[The physical resistance] was the way to create theboundaries between various virtual objects. When
perceiving minor details, the sound cannot convey that, itcannot convey that you are in that boundary at that specificmoment. The force made it realistic.” Three participants(V4, V8, V10) said they relied more on the audio feedback
when exploring the virtual world. The other four VIPs
indicated that the different feedback modalities wereequally important and the combination created the best VR
experience. “I think it’s really a combination. The metal
legs of the desk and the door proved very realistic
sounding. The force is very important for perceiving thewall. The vibration of the domes was also good that I could
feel a larger bump versus the roughness of the road” (V1).
Exploration of the Virtual World

Three participants (V1, V6, V8) used Canetroller tounderstand the size of virtual objects. For example, after
locating the three table legs, V1 reported that the size of thetable should be big: “It’s a big desk, because I found twolegs and then a while later I found that third leg.”
V6 also used Canetroller to perceive the height of different
virtual objects. When he located the table leg, he moved thecane controller up along the table leg and discerned theheight with the cane: “So it could actually have aperceivable height!” He was also able to distinguish theheight difference between virtual objects. When he foundthe trashcan, he mentioned, “That has some height, lessheight than the table, though.”
When exploring the virtual world with Canetroller,
participants used the same cane strategies as in the realworld. They usually used shorelining along the virtual wall
to locate the virtual door, and used constant contact to scan
the virtual space to discover virtual objects. Someparticipants wanted Canetroller to support more canestrategies; for example, V1 tried to use a pencil grasp (i.e.,holding the cane vertically like a pencil [66]) in the indoor
scenario because she thought it was a small and crowded
environment: “I would probably be more likely to hold it inmore of a pencil grasp in the inside. People tend to use theircanes in a lot of different ways depending on theenvironment. So, for a small room, I would not typicallyhave my cane straight out in front of me [like what I did
with Canetroller].”
Environmental Sounds

We found that the simulated environmental sounds had astrong effect on participants’ experience in the VR world.
Participants who preferred the outdoor scene all indicatedthat the environmental traffic sound made the VR world
more realistic. As V4 mentioned, “It feels like [real], it wasnerve-racking in terms of ‘Oh, am I going to get run overby this car while I’m crossing the [virtual] street?’”Paper 116

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

Participants suggested adding more environmental sounds
to the VR experience. When navigating the VR space, V1
and V4 wanted to hear the echo from the environment,
which is a common strategy blind people use to estimate thesize of space and the distance to an object (i.e.,echolocation [47]). As V4 explained, “I didn’t have a good
sense of direction where I was at [in the real world]. I canhear roughly where the wall is at, by the way it blocks off
sound in the real world. I didn’t have that in the VR world.”
Potential of Canetroller for VIPs

All VIPs believed that Canetroller had strong potential.
They were excited about Canetroller making the VR world
inclusive. As V9 mentioned, “It was exciting to … be able
to do [VR] as a blind person using [Canetroller]... it was
inclusive. I enjoyed getting to participate in something…
that sighted people get to do.”When asked about desired VR applications, mostparticipants mentioned mobility training and learning newenvironments. As V2 described, “I think it has potential. If I
can learn about a room in a certain building through virtualworld before I go over there ... It may provide me with asummary of the room, it will provide much value to me ...
in place of me just go there and explore it myself.” Someparticipants also wanted to use Canetroller for fun,experiencing a fantasy world. V6 was excited about VR
games: “I definitely can see a lot in gaming environments.
If you could touch a car with your cane when it’s going,
like a superman, that is cool!”
DISCUSSION

The user study demonstrated that Canetroller is a promising
tool for VIPs to explore and navigate virtual environments.
With only a few minutes’ tutorial, in the indoor scenario,
eight participants built an accurate mental model of theroom and correctly located all virtual objects; in the outdoor
scenario, six participants successfully navigated along thevirtual sidewalk and crossed the street without assistance.
O&M instructors also saw promise in Canetroller. We
demonstrated and discussed the potential of Canetroller
with five O&M instructors (Table 2), and received positive
feedback: “It is pretty cool. I would have been very
skeptical before, but now I see that [Canetroller] provides[a] really good experience” (O3). All instructors agreed that
Canetroller could be used in O&M training, especiallysimulating some scenarios that may have risks, such as thestreet. “It can be used with students who might be nervous
about crossing a particular busy street. Let’s try it this way
first… without having the risk of hitting by [a] car” (O1).
Moreover, O3 felt Canetroller would be good for outreachfor blindness awareness, by simulating blindness in VR.
The most novel part of Canetroller is the use of thewearable brake mechanism to generate physical resistancefor preventing virtual cane penetration in a mobile situation.
Prior VR haptic controllers for VIPs either allowed only
stationary use by grounding the controller on the desk orPage 9

CHI 2018 Paper

floor, or providing force feedback to the user’s handthrough haptic gloves, which could not physically stop thevirtual cane penetrating virtual objects. Our wearable brakemechanism addressed this by using the user’s torso as ananchor to physically halt the user’s hand movement. This
design allowed users to physically walk in the VR space,and enabled them to have a precise perception on theboundaries of different virtual objects, thus providing amore immersive VR experience.
However, our current prototype only provided physicalresistance in the horizontal left-right direction. According to
the user study, it would be helpful to provide physicalresistance in other directions. For example, participants
encountered difficulty perceiving the change of surface
height (e.g., the slope of the domes) because of the lack ofphysical resistance in the vertical direction. V6 suggested,“If it’s a little bit higher up on the ground, I hope I would
actually have to lift the cane to feel the rise and change inheight.” Moreover, without physical resistance in theforward-backward direction, participants could not receive
accurate location information by poking Canetroller at a
virtual object. For example, V9 was able to approximately
triangulate the corner by hitting the two virtual walls;
however, she could not locate the specific position of thecorner. V9 mentioned, “I know that was a corner, but it did
not feel like a corner exactly. It felt like there was an actual
space between the two walls. With a real cane, you canactually poke into the corner.” In the future, we should
improve the brake mechanism, providing threedimensional, or at least two axis braking feedback to enable
users to perceive the virtual boundaries with more flexible
cane movement.
The limited size of our physical VR space (~22 m2) was
challenging, especially for the outdoor scenario. Since thevirtual street and sidewalk were much narrower than thosein the real world, some participants reached the other end of
the room quickly without having enough time to locate thedomes and sidewalk with Canetroller. This diminished theimmersive experience. V1 mentioned, “The indoor was a
lot more realistic than the outdoor and I think part of that is
because of size. The street is very narrow so it’s very easy
to [ignore the curbs].” The O&M instructors emphasized
the importance of creating a big enough space for mobility
training. “The space is kind of small. It needs to bephysically larger so that people will have more space topractice and learn the dimensions of the street that is moresimilar to real life” (O3). Interestingly, O7 suggested using
a button to trigger a virtual translation in the VR world.
When the user released the button, she could still use
Canetroller to physically explore the nearby virtual space.
Although the virtual space can be infinitely large, thephysical room will always be limited. In the future, we will
design and evaluate different interaction techniques thatenable the user to explore a big VR space within a small
real-world room.

Paper 116

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

The hardware setup of Canetroller also had limitations.
Most participants noted that it was too heavy (the cane
controller held by the user weighed 0.44 kg and the brakevest-slider apparatus worn on the waist weighed 2.1 kg) andhad a different feeling than the real cane. Moreover, theshort length of the cane controller changed participants’holding posture. Since the tip of Canetroller did not rest on
the ground, participants needed to spend energy to hold thecane controller up, which sometimes led to overcompensation, resulting in the tip of the virtual cane
floating over the virtual ground. However, we envision that
this problem could be addressed by creating a brakemechanism in the up-down dimension. If we are able toimpart a vertical physical resistance, we would also be able
to add a constant force spring to reduce the perceived
weight of the Canetroller to that of a real cane.
We evaluated Canetroller with an indoor exploration and an
outdoor navigation task, demonstrating the basic usabilityof our system. There are many other possible approaches
that can be used to investigate people’s VR experience, forexample, a quantitative analysis on the users’ mental modelby tracking their exploration path and asking them toreconstruct the VR space [24].
We evaluated the immersion of Canetroller with the Igroup
Presence Questionnaire. Because we are the first to use thisscale for VIPs (and had to modify some questions to suit
this audience), we cannot easily compare these scores toprior work. Instead, we collected these scores to serve as ananchor for comparison with future work. We hope that
others creating VR for VIPs will find our modifications to
the questionnaire useful, and will begin to build up a set ofscores for comparison for this user group. Moreover, since
Canetroller is such a new experience, the novelty effects
could impact our participants’ scores. Future work should
conduct a long-term evaluation to reduce this impact.
CONCLUSION

In this paper, we presented Canetroller, a haptic controllerthat simulates white cane interactions, enabling people withvisual impairments to navigate a virtual environment bytransferring their cane skills into the virtual world. We
started with a formative study to understand VIPs’ canestrategies, and distilled implications to guide the design ofCanetroller. We also evaluated the effectiveness ofCanetroller with indoor and outdoor virtual scenarios,
showing it to be a promising tool to enable visually
impaired users to explore and navigate the virtual world.
Canetroller has potential in different areas, such asentertainment, O&M training, and environment preparation.
We hope our work can inspire researchers and designers todesign more effective tools to make VR more inclusive.
ACKNOWLEDGMENTS

We thank all our participants. We also thank Andrew D.
Wilson, Eyal Ofek, Mar Gonzalez Franco, Eric Whitmire,
Evan Strasnick, and Inrak Choi for their helpful feedback.

Page 10

CHI 2018 Paper

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

REFERENCES

11. J. L. González-Mora, A. Rodríguez-Hernández, L. F.
Rodríguez-Ramos, L. Díaz-Saco, and N. Sosa. 1999.
Development of a new space perception system forblind people, based on the creation of a virtual acoustic
space. . Springer, Berlin, Heidelberg , 321–330.
https://doi.org/10.1007/BFb0100499

1.

Rebecca Adams, Paul Finn, Elisabeth Moes, Kathleen
Flannery, and Albert “Skip” Rizzo. 2009.
Distractibility in Attention/Deficit/ Hyperactivity
Disorder (ADHD): The Virtual Reality Classroom.
Child Neuropsychology 15, 2: 120–135.
https://doi.org/10.1080/09297040802169077

2.

B. Ando, S. Baglio, V. Marletta, and A. Valastro. 2015.
A Haptic Solution to Assist Visually Impaired in
Mobility Tasks. IEEE Transactions on HumanMachine Systems 45, 5: 641–646.
https://doi.org/10.1109/THMS.2015.2419256

3.

Bruno Andò, Salvatore Baglio, Cristian Orazio
Lombardo, Vincenzo Marletta, E. A. Pergolizzi,
Antonio Pistorio, and Angelo Emanuele Valastro.
2015. An Electronic Cane with a Haptic Interface for
Mobility Tasks. . Springer, Cham, 189–200.
https://doi.org/10.1007/978-3-319-18374-9_18

4.

Bruno Andò, Salvatore Baglio, Vincenzo Marletta, andAngelo Valastro. 2017. A Tilt Compensated Haptic
Cane for Obstacle Detection. . Springer, Cham, 141–151. https://doi.org/10.1007/978-3-319-54283-6_11

5.

Bruno Andò, Salvatore Baglio, and Nicola Pitrone.
2008. A Contactless Haptic Cane for Blind People. In
12th IMEKO TC1 & TC7 Joint Symposium on Man
Science & Measurement, 147–152.

6.

Joseph Bates. 1992. Virtual Reality, Art, andEntertainment. Presence: Teleoperators and Virtual
Environments 1, 1: 133–138.
https://doi.org/10.1162/pres.1992.1.1.133

7.

J Bell, S Bolanowski, and M H Holmes. 1994. The
structure and function of Pacinian corpuscles: a review.
Progress in neurobiology 42, 1: 79–128. Retrieved
September 5, 2017 fromhttp://www.ncbi.nlm.nih.gov/pubmed/7480788

8.

J. Borenstein and I. Ulrich. 1997. The GuideCane-acomputerized travel aid for the active guidance of blindpedestrians. In Proceedings of International
Conference on Robotics and Automation, 1283–1288.
https://doi.org/10.1109/ROBOT.1997.614314

9.

Chetz Colwell, Helen Petrie, Andrew Hardwick, andMartlesham Heath. 1998. Use of a haptic device byblind and sighted people : perception of virtual texturesand objects. Improving the Quality of Life for theEuropean Citizen : Technology for Inclusive Design
and Equality, JULY: 1–8.
https://doi.org/10.1.1.35.2245

10. Chetz Colwell, Helen Petrie, Diana Kornbrot, Andrew
Hardwick, and Stephen Furner. 1998. Haptic virtual
reality for blind computer users. In Proceedings of thethird international ACM conference on Assistive
technologies - Assets ’98, 92–99.
https://doi.org/10.1145/274497.274515

Paper 116

12. José Luis González-Mora, Antonio Francisco
Rodríguez-Hernández, Enrique Burunat, F. Martin, and
M A Castellano. 2006. Seeing the world by hearing:
Virtual Acoustic Space (VAS) a new space perception
system for blind people. In 2nd International
Conference on Information & Communication
Technologies, 837–842.
https://doi.org/10.1109/ICTTA.2006.1684482
13. E W Hill, J. J. Rieser, M. Hill, J Halpin, and R Halpin.
1993. How persons with visual impairments explore
novel spaces? A study of strategies used by
exceptionally good and exceptionally poor performers.
J Vis Impair Blind 87: 295–301. Retrieved August 31,
2017 from http://psycnet.apa.org/record/1994-18302001
14. D. Jack, R. Boian, A.S. Merians, M. Tremaine, G.C.
Burdea, S.V. Adamovich, M. Recce, and H. Poizner.
2001. Virtual reality-enhanced stroke rehabilitation.
IEEE Transactions on Neural Systems andRehabilitation Engineering 9, 3: 308–318.
https://doi.org/10.1109/7333.948460
15. W. H. Jacobson. 1993. The Art and Science ofTeaching Orientation and Mobility to Persons with
Visual Impairments. AFB Press.
16. G Jansson, H Petrie, C Colwell, and D Kornbrot. 1999.
Haptic virtual environments for blind people:
Exploratory experiments with two devices. The
International Journal of Virtual Reality 3, 4: 8–17.
Retrieved August 31, 2017 fromhttps://pdfs.semanticscholar.org/348e/45107167a03250
51e60c883c153572a127e4.pdf
17. Katerina Kalyvioti and Tassos A. Mikropoulos. 2014.
Virtual Environments and Dyslexia: A Literature
Review. Procedia Computer Science 27: 138–147.
https://doi.org/10.1016/j.procs.2014.02.017
18. Yoshihiro Kawai and Fumiaki Tomita. Evaluation ofInteractive Tactile Display System. Retrieved August
31, 2017 fromhttps://pdfs.semanticscholar.org/c8da/db1e1caa0b4e31
7db3bb8f22c6a45a52f8c6.pdf
19. Dae Shik Kim, Robert Wall Emerson, and Amy Curtis.
2009. Drop-off Detection with the Long Cane: Effectsof Different Cane Techniques on Performance. Journalof visual impairment & blindness 103, 9: 519–530.
Retrieved September 2, 2017 fromhttp://www.ncbi.nlm.nih.gov/pubmed/21209791
20. Henry König, Jochen Schneider, and Thomas
Strothotte. Haptic Exploration of Virtual Buildings

Page 11

CHI 2018 Paper

Using Non-Realistic Haptic Rendering. Retrieved
August 31, 2017 fromhttps://pdfs.semanticscholar.org/5f54/198fc98e01be1b73eb1d31b4ef6b95070792.pdf
21. Michael Kotlyar, Christopher Donahue, Paul Thuras,
Matt G. Kushner, Natalie O’Gorman, Erin A Smith,and David E. Adson. 2008. Physiological response to aspeech stressor presented in a virtual reality
environment. Psychophysiology 45, 6: 1034–1037.
https://doi.org/10.1111/j.1469-8986.2008.00690.x
22. O Lahav and D Mioduser. Multisensory virtualenvironment for supporting blind persons’ acquisitionof spatial cognitive mapping, orientation, and mobility
skills. Retrieved August 31, 2017 fromhttp://playpen.icomtek.csir.co.za/~acdc/assistive
devices/Artabilitation2008/archive/2002/papers/2002_
28.pdf
23. Orly Lahav and David Mioduser. 2004. Exploration of
Unknown Spaces by People Who Are Blind Using a
Multi-sensory Virtual Environment. Journal of Special
Education Technology 19, 3: 15–23. Retrieved August
31, 2017 fromhttp://journals.sagepub.com/doi/pdf/10.1177/01626434
0401900302
24. Orly Lahav and David Mioduser. 2008. Construction of
cognitive maps of unknown spaces using a multisensory virtual environment for people who are blind.
Computers in Human Behavior 24, 3: 1139–1155.
https://doi.org/10.1016/j.chb.2007.04.003
25. Mark L. Latash, EA Atree, BM Brooks, DA Johnson,
EA Attree, A Bellner, M Rydmark, H Poizner, L
Hughey, CL Richards, Y Bisson, A Rovetta, S
Rushton, C Selis, and J Wann. 1998. Virtual reality: A
fascinating tool for motor rehabilitation (to be usedwith caution). Disability and Rehabilitation 20, 3: 104–105. https://doi.org/10.3109/09638289809166065
26. A. Lecuyer, P. Mobuchon, C. Megard, J. Perret, C.
Andriot, and J.-P. Colinot. HOMERE: a multimodal
system for visually impaired people to explore virtualenvironments. In IEEE Virtual Reality, 2003.
Proceedings., 251–258.
https://doi.org/10.1109/VR.2003.1191147
27. Fabrizio Leo, Elena Cocchi, and Luca Brayda. 2017.
The Effect of Programmable Tactile Displays on
Spatial Learning Skills in Children and Adolescents ofDifferent Visual Disability. IEEE Transactions on
Neural Systems and Rehabilitation Engineering 25, 7:
861–872.
https://doi.org/10.1109/TNSRE.2016.2619742
28. Shachar Maidenbaum, Shelly Levy-Tzedek, Daniel
Robert Chebat, and Amir Amedi. 2013. Increasing
accessibility to the blind of virtual environments, using
a virtual mobility aid based on the “EyeCane”:
Feasibility study. PLoS ONE 8, 8: e72555.

Paper 116

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

https://doi.org/10.1371/journal.pone.0072555
29. Willie Martin, Kory Dancer, Kevin Rock, Christopher
Zeleney, and Kumar Yelamarthi. 2009. The Smart
Cane : An Electrical Engineering Design Project. 1–9.
Retrieved August 31, 2017 fromhttp://people.cst.cmich.edu/yelam1k/CASE/Publications_files/Yelamarthi_ASEE_NCS_2009.pdf
30. M. P. Menikdiwela, K.M.I.S. Dharmasena, and A.M.
Harsha S. Abeykoon. 2013. Haptic based walking stickfor visually impaired people. In 2013 International
conference on Circuits, Controls and Communications(CCUBE), 1–6.
https://doi.org/10.1109/CCUBE.2013.6718549
31. Lotfi B Merabet and Jaime Sanchez. 2009. AudioBased Navigation Using Virtual Environments:
Combining Technology and Neuroscience. AER
Journal: Research and Practice in Visual Impairment
and Blindness 2: 128–137.
32. Zahira Merchant, Ernest T. Goetz, Lauren Cifuentes,
Wendy Keeney-Kennicutt, and Trina J. Davis. 2014.
Effectiveness of virtual reality-based instruction on
students’ learning outcomes in K-12 and higher
education: A meta-analysis. Computers & Education
70: 29–40.
https://doi.org/10.1016/j.compedu.2013.07.033
33. Alma S Merians, David Jack, Rares Boian, Marilyn
Tremaine, Grigore C Burdea, Sergei V Adamovich,
Michael Recce, and Howard Poizner. 2002. Virtual
Reality–Augmented Rehabilitation for Patients
Following Stroke. Physical Therapy 37, 1: 975–987.
https://doi.org/10.1093/ptj/82.9.898
34. Thomas D. Parsons, Todd Bowerly, J. Galen
Buckwalter, and Albert A. Rizzo. 2007. A Controlled
Clinical Comparison of Attention Performance inChildren with ADHD in a Virtual Reality Classroom
Compared to Standard Neuropsychological Methods.
Child Neuropsychology 13, 4: 363–381.
https://doi.org/10.1080/13825580600943473
35. Maurizio de Pascale, Sara Mulatto, and Domenico
Prattichizzo. 2008. Bringing Haptics to Second Life for
Visually Impaired People. . Springer, Berlin,
Heidelberg, 896–905. https://doi.org/10.1007/978-3540-69057-3_112
36. Lorenzo Picinali, Amandine Afonso, Michel Denis,
and Brian F.G. Katz. 2014. Exploration of architectural
spaces by blind people using auditory virtual reality forthe construction of spatial knowledge. International
Journal of Human Computer Studies 72, 4: 393–407.
https://doi.org/10.1016/j.ijhcs.2013.12.008
37. S. Reardon. 2011. Playing by Ear. Science 333, 6051:
1816–1818.
https://doi.org/10.1126/science.333.6051.1816
38. Irvin Rock and Jack Victor. 1964. Vision and Touch:

Page 12

CHI 2018 Paper

An Experimentally Created Conflict between the Two
Senses. Science 143, 3606. Retrieved August 28, 2017fromhttp://science.sciencemag.org/content/143/3606/594
39. Joseph M. Romano and Katherine J. Kuchenbecker.
2012. Creating Realistic Virtual Textures from Contact
Acceleration Data. IEEE Transactions on Haptics 5, 2:
109–119. https://doi.org/10.1109/TOH.2011.38
40. J H Sánchez and M A Sáenz. 2006. Assisting themobilization through subway networks by users withvisual disabilities. Virtual Reality & Assoc. Tech.
Retrieved August 30, 2017 fromwww.dcc.uchile.cl/~jsanchez
41. JAIME SÁNCHEZ and MAURICIO LUMBRERAS.
1999. Virtual Environment Interaction Through 3D
Audio by Blind Children. CyberPsychology &Behavior 2, 2: 101–111.
https://doi.org/10.1089/cpb.1999.2.101
42. David W. Schloerb, Orly Lahav, Joseph G. Desloge,
and Mandayam A. Srinivasan. 2010. BlindAid: Virtualenvironment system for self-reliant trip planning andorientation and mobility training. In 2010 IEEE
Haptics Symposium, 363–370.
https://doi.org/10.1109/HAPTIC.2010.5444631
43. Thomas Schubert, Frank Friedmann, and Holger
Regenbrecht. 2001. The Experience of Presence:
Factor Analytic Insights. Presence: Teleoperators andVirtual Environments 10, 3: 266–281.
https://doi.org/10.1162/105474601300343603
44. Yoshikazu Seki and Tetsuji Sato. 2011. A training
system of orientation and mobility for blind peopleusing acoustic virtual reality. IEEE Transactions on
Neural Systems and Rehabilitation Engineering 19, 1:
95–104. https://doi.org/10.1109/TNSRE.2010.2064791
45. SK Semwal. 2001. MoVE: Mobiltiy training in haptic
virtual environment. piglet. uccs. edu/~
semwal/NSF2001PS. pdf: 1–18.
46. Vaibhav Singh, Rohan Paul, Dheeraj Mehra, Anuraag
Gupta, Vasu Dev Sharma, Saumya Jain, Chinmay
Agarwal, Ankush Garg, Sandeep Singh Gujral, M
Balakrishnan, and K Paul. 2010. “Smart”Cane for the
Visually Impaired: Design and Controlled Field
Testing of an Affordable Obstacle Detection System.
TRANSED 2010: 12th International Conference on
Mobility and Transport for Elderly and Disabled
Persons 53, 9: 1689–1699.
https://doi.org/10.1017/CBO9781107415324.004
47. Lore Thaler, Galen M. Reich, Xinyu Zhang, Dinghe
Wang, Graeme E. Smith, Zeng Tao, Raja Syamsul
Azmir Bin. Raja Abdullah, Mikhail Cherniakov,
Christopher J. Baker, Daniel Kish, and Michail
Antoniou. 2017. Mouth-clicks used by blind expert
human echolocators – signal description and model
based signal synthesis. PLOS Computational Biology

Paper 116

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

13, 8: e1005670.
https://doi.org/10.1371/journal.pcbi.1005670
48. Virgil Tiponut, Zoltan Haraszy, Daniel Ianchis, andIoan Lie. 2008. Acoustic virtual reality performing
man-machine interfacing of the blind. In Proceedingsof the 12th WSEAS international conference on
Systems, 345–349.
49. M A Torres-Gil, O Casanova-Gonzalez, and José Luis
González-Mora. 2010. Applications of virtual reality
for visually impaired people. WSEAS Transactions on
Computers 9, 2: 184–193. Retrieved fromhttp://dl.acm.org/citation.cfm?id=1852403.1852412
50. Shari Trewin, Mark Laff, Vicki Hanson, and Anna
Cavender. 2009. Exploring Visual and Motor
Accessibility in Navigating a Virtual World. ACM
Transactions on Accessible Computing 2, 2: 1–35.
https://doi.org/10.1145/1530064.1530069
51. D. Tzovaras, G. Nikolakis, G. Fergadis, S. Malasiotis,
and M. Stavrakis. 2002. Design and implementation ofvirtual environments training of the visually impaire.
Proceedings of the fifth international ACM conference
on Assistive technologies - Assets ’02 12, 2: 41.
https://doi.org/10.1145/638249.638259
52. Dimitrios Tzovaras, Konstantinos Moustakas, Georgios
Nikolakis, and Michael G. Strintzis. 2009. Interactive
mixed reality white cane simulation for the training of
the blind and the visually impaired. Personal andUbiquitous Computing 13, 1: 51–58.
https://doi.org/10.1007/s00779-007-0171-2
53. Iwan Ulrich and Johann Borenstein. 2001. The
GuideCane — Applying Mobile Robot Technologies toAssist the Visually Impaired. IEEE Transactions on
Systems, Man, and Cybernetics, —Part A: Systems andHumans 31, 2: 131–136. Retrieved August 31, 2017fromhttp://www.cs.unc.edu/~welch/class/mobility/papers/G
uideCane.pdf
54. Maria Virvou and George Katsionis. 2008. On theusability and likeability of virtual reality games foreducation: The case of VR-ENGAGE. Computers andEducation 50, 1: 154–178.
https://doi.org/10.1016/j.compedu.2006.04.004
55. M Helmy Abd Wahab, Aa Talib, and Ha Kadir. 2011.
Smart Cane: Assistive Cane for Visually-impaired
People. IJCSI International Journal of Computer
Science Issues 8, 4: 21–27. https://doi.org/1694-0814
56. Steven Wall and Stephen Brewster. 2006. Feeling
What You Hear : Tactile Feedback for Navigation of
Audio Graphs. Conference on Human Factors in
Computing Systems, April: 1123–1132.
https://doi.org/10.1145/1124772.1124941
57. Gareth R. White, Geraldine Fitzpatrick, and Graham
McAllister. 2008. Toward accessible 3D virtual

Page 13

CHI 2018 Paper

environments for the blind and visually impaired. In
Proceedings of the 3rd international conference on
Digital Interactive Media in Entertainment and Arts DIMEA ’08, 134.
https://doi.org/10.1145/1413634.1413663

CHI 2018, April 21–26, 2018, Montréal, QC, Canada

Reality to Games. Computer 38, 9: 25–32.
https://doi.org/10.1109/MC.2005.297
62. How Virtual Reality Facilitates Social Connection |Facebook IQ. Retrieved August 28, 2017 fromhttps://www.facebook.com/iq/articles/how-virtualreality-facilitates-social-connection

58. Michele A Williams, Caroline Galbraith, Shaun K
Kane, and Amy Hurst. “Just Let the Cane Hit It”: How
the Blind and Sighted See Navigation Differently. In
Proceedings of the 16th interal ACM SIGACCESS
conference on Computers & Accessibility - ASSETS'14.
271-224. https://doi.org/10.1145/2661334.2661380

63. Phantom Force Feedback. Retrieved August 31, 2017fromhttp://www.cgl.ucsf.edu/chimera/1.2470/docs/Contribu
tedSoftware/phantom/phantom.html

59. Michele a Williams, Amy Hurst, and Shaun K Kane.
2013. “Pray Before You Step out”: Describing
Personal and Situational Blind Navigation Behaviors.
Proceedings of the 15th International ACM
SIGACCESS Conference on Computers andAccessibility: 28:1----28:8.
https://doi.org/10.1145/2513383.2513449

65. Geomagic Touch (formerly Geomagic Phantom Omni)
Overview. Retrieved August 31, 2017 fromhttp://www.geomagic.com/en/products/phantomomni/overview

60. Limin Zeng, Mei Miao, and Gerhard Weber. 2012.
Interactive Audio-haptic Map Explorer on a Tactile
Display. Interacting with Computers 27, 4: 413–429.
https://doi.org/10.1093/iwc/iwu006
61. M. Zyda. 2005. From Visual Simulation to Virtual

Paper 116

64. Novint - Products. Retrieved August 31, 2017 fromhttp://www.novint.com/index.php/products

66. Long Cane Techniques, Study Guide: APH Step-byStep. Retrieved September 2, 2017 fromhttp://tech.aph.org/sbs/04_sbs_lc_study.html
67. igroup presence questionnaire (IPQ) Database |igroup.org – project consortium. Retrieved September
11, 2017 from http://www.igroup.org/pq/ipq/data.php

Page 14