Smart Head Mounted Display (smartHMD) for Improved Mobility
Phase: NA
Conditions: Low Vision, Orientation, Mobility Limitation, Navigation, Spatial, Visual Impairment
Sponsor: James Weiland

 
SmartHMD for improved mobility  
NCT # 03781583 
IRB approval date 5/29/2021 
 
 
 
 
 

SmartHMD for improved mobility  
 
PROTOCOL 
 
I. 
BACKGROUND 
In 2015, 1.02 million individuals were blind and 3.22 million were visually impaired in the US, with this 
prevalence expected to double by 2050.(Varma 2016) The economic cost of chronic care for individuals 
with vision impairment is significant, expected to increase from $145 billion to $375 billion during this period 
(www.rpbusa.org).  Many patients with blindness or vision impairment are considered as low vision (LV), 
which is defined functionally as chronic uncorrectable vision loss that impacts daily living or clinically by 
visual acuity and visual field (VF) criteria (www.who.int/blindness).  
 
Numerous studies show that reduced orientation and mobility (O&M) results in the loss of independence, 
increased depression, and an overall reduced quality of life (Salive 1994; Owsley 2004; Montarzino 2007; 
Paz 2001). LV limits mobility, and therefore, the independence and health of afflicted individuals. Studies 
have shown that people with peripheral VF loss are especially impacted by impaired O&M (Geruschat 1998; 
Haymes 1996). The role of peripheral vision in detecting and avoiding nearby obstacles is a possible 
explanation for why people with peripheral vision loss experience greater O&M deficits than people with 
central vision loss (Kuyk 1996)  
 
The prevalence of LV due to peripheral vision loss is not fully known, but one study showed that 21% of 
individuals with LV had a diagnosis of glaucoma or early-onset retinal degeneration, which commonly lead 
to peripheral vision loss (Owsley 2009). Another study showed that 14% of patients with LV had one of 
these diagnoses and stated the possible under-sampling of patients with peripheral VF loss (Goldstein 
2012; Brown 2014). These studies underestimate the burden of peripheral VF loss since conditions such 
as non-glaucomatous optic neuropathies or a history of panretinal photocoagulation treatment for retinal 
ischemia may cause peripheral field loss, but do so less consistently or were not sufficiently captured in 
past studies.  
 
Up to 85% of patients with LV may improve their vision-dependent function through low vision rehabilitation. 
Despite potential benefits of rehabilitation, there has been limited research and few assistive devices for 
LV due to peripheral vision loss. These devices do not address functional impairments, such as reduced 
O&M, from peripheral vision loss. Head-mounted display (HMD) technology has potential to benefit 
individuals with severe peripheral vision loss (Ehrlich 2016). Our team has developed a prototype 
smartHMD system that functions as a personalized navigation system to enhance vision-dependent 
functions, in particular O&M, for patients with LV.  
 
II. 
OBJECTIVES/ HYPOTHESIS 
Our objective is to assess the impact of smartHMD in several navigation scenarios that people with low 
vision find difficult to complete. We expect the smartHMD to increase the completion time for navigation 
task, vs. baseline mobility. 
 
III. 
STUDY DESIGN 
a. Patient Recruitment 
Participants will be enrolled at the Kellogg Eye Center (KEC) and participants will be recruited via mailings 
or will be identified by a vision care provider. Patients recruited via mailings will be given two weeks to opt-
out of the study prior to receiving a phone call. To evaluate eligibility and clinical status, medical records of 
patients with conditions or treatments likely to result in PFL (glaucoma, retinal dystrophy including Argus 2 
implants, optic neuropathy, pan-retinal photocoagulation) will be reviewed. Inclusion and exclusion criteria 
are summarized in Table 1. Participants will be reimbursed $100 for their time.  
 
 
 
 

Table 1.  Summary of inclusion and exclusion study criteria for participants  
INCLUSION CRITERIA 
EXCLUSION CRITERIA 
 
• Diagnosis of low vision 
• Self-reported difficulty with 
mobility and finding doors (either 
indoors or outdoors) and using 
signalized crosswalks. 
• Either sex 
• Any self-declared ethnoracial 
category 
• > 14 years’ old 
• Ability to cooperate for tests 
• Able to participate in all visits 
 
• Unable to use head mounted display  
• Unstable age-related macular degeneration within the past 
3 months 
• Unstable diabetic retinopathy within the past 3 months 
• Unstable diabetes within the past 3 months 
• Ocular infection or ocular inflammation in the past 3 
months 
• Ocular trauma within the past 6 months 
• Intraocular surgery within 6 months 
• Optical coherence tomography retinal findings of concern 
to investigator for unstable vision during the study 
• Women who are pregnant (due to risk of falls and change 
in gait). The study team will ask if a potential subject is 
pregnant and those who self-report will be excluded. 
• Uncontrolled seizure disorder in the past 6 months 
• Cerebrovascular accident occurring in the past 6 months 
• Parkinson disease or neurological condition that limits 
mobility 
• Alzheimer disease or other forms of dementia 
• Score of less than 18 on MoCA-Blind test 
• Conditions of concern to investigator that would confound 
orientation and mobility, such as severe arthritis, pain that 
limits ambulatory activities, or orthopedic surgery (e.g., 
hand, arm, shoulder, knee, or hip surgery within 12 
months) 
 
b. Experimental Methods and Design 
SmartHMD (described in section d) will be evaluated in participants (n=30) with severe VF constriction.  
The study visit should take up to three hours depending on which experiments are performed. Typically, 
we will focus on one or two of the five tests described below. For all experiments, the participant’s baseline 
mobility skills will serve as the control. We will first train the participant to ensure they can effectively use 
the smartHMD. Prior to each task, we will describe the task to them. If the participant does not think they 
can perform the task with their baseline mobility, they will not be forced to do so. For some experiments, 
we will blindfold the participants, to remove the variable of residual vision. This has been done in other 
studies of blind mobility (Rizzo 2018). 
 
1. Outdoor testing: crosswalk detection 
We will use crosswalks with signals near the North Campus Research Center. These crosswalks are 
highly used by pedestrians, thus they have well-maintained signals, crosswalk striping, and pavement.  
Testing will be done in with verbal, vibrational, and (for Argus II participants) modified video. 
 
Subjects will stand 2 meters from and with their backs facing the intersection. Subjects will then be 
instructed to walk the intersection. They will then have to align themselves with the crosswalk and point to 
the pedestrian signal upon locating it.  Upon locating the pedestrian signal, subjects will either cross or not 
cross the street based on their interpretation of the pedestrian signal display (without smartHMD) or by 
following the cues from the smartHMD (smartHMD).. After an orientation period with the smartHMD (which 
will include practice mobility trials), subjects will repeat the crosswalk navigation task using smartHMD and, 
if possible, without smartHMD. 
 
Approximately 12 crosswalk navigation trials will be collected for each condition. To reduce the effects of 
familiarity with the intersection with repeated visits and trials, four different signalized intersections will be 

used and subjects will negotiate each intersection in the forward and reverse direction. The order of walking 
direction and which signalized intersection will be used will be counter-balanced across subjects and 
condition. The order of testing with the Baseline or Sham condition first will also be counter-balanced across 
subjects and signalized intersections. 
 
2. Outdoor navigation 
The participant will start at a location approximately 50 m or less from a building entry point. The smartHMD 
will guide the user towards an entry door. The participants starting location will vary. We will conduct 10 
trials for each participant. 
 
3. Hallway navigation 
The participant will be guided by the smartHMD from one location in NCRC to another location. The route 
will be on the same floor (no stairs or elevators), but may include ramps. A participant may complete up to 
10 routes, each about 50 m, with 2-3 turns. The order of routes and smartHMD configurations will be varied 
across participants. 
 
4. Indoor testing: Door detection  
Subjects’ will use smartHMD technology guided computer vision to navigate toward a door sign located 
approximately 3 to 4 meters away. This testing will be done in a conference room. The participant’s starting 
point will be at different orientations from the door. The subjects will be stopped to ensure that they are not 
hit by door opening. We will conduct 8 trials, each with a unique combination of starting point and orientation 
with respect to the door. 
 
Three experimenters will be present during all navigation trials that are done in common areas, including 
the outdoor spaces, crosswalks, and hallways. Two will act as a safety officers and flank the subject at all 
times and the other experimenter will record the navigation outcome measures (defined below). For testing 
done in closed space, such as a conference room, one safety officer will walk with the participant and one 
experimenter will record data. After every trial for the experiments listed above, the participant will be offered 
a seated break.  
 
Prior to the navigation trials, each subject’s preferred walking speed (PWS) will be determined by timing 
how long it takes the subject to walk at their normal pace, along a ~20 meter straight, unobstructed, sidewalk 
or hallway.  Subjects will wear their habitual spectacle correction, if normally worn for mobility activities and 
no assistive mobility devices (including the smartHMD) will be used when measuring subjects’ PWS.  
Subjects’ PWS will be measured four times but only the last three PWS trials will be averaged to represent 
each subject’s PWS due to familiarity effects between the first and second PWS trial.  
 
The smartHMD will include automatic data logging to record video and user command and cue, all time-
stamped for later analysis.  
 
For all experiments, time to the complete the route is the primary outcome measure. In contrast to reaction 
time in speed-critical tasks, time to completion is used here as a continuous measure of task difficulty that 
encompasses wrong responses and resulting error correction, slow walking due to confusion, and overall 
efficacy of travel. Our statistical analysis will account for differences in walking speed between participants. 
Secondary measures include Percentage of Preferred Walking Speed (Patel 2006), # of unintended 
contacts with obstacles and walls, # of incorrect turns, # of requests for assistance, type of assistance 
requested,  and # of interventions by the experimenter. Data collection will be facilitated by a wearable test 
system with redundant data collection via hard copy datasheets and a handheld (by an experimenter) video 
camera. Post-test data processing will refine the measures.  
 
c. Subject Interviews and Surveys 
We will administer standard and custom surveys to the participant to obtain more information about their 
condition, difficulties they face due to visual impairment, and preference for the smartHMD design. A copy 
of all 5 surveys are attached in the Additional Supporting Documents section. The interviews will be 
recorded. 

 
MoCA-Blind (Montreal Cognitive Assessment) is an adapted version of the original MoCA 
(www.mocatest.org), a rapid screening instrument for mild cognitive impairment.  We will use the 
recommended cutoff value of ≥18 to screen participants for suspected cognitive impairment or dementia. 
Data will be scored using the recommended summary scoring algorithm that includes scores on items 
corresponding to memory, attention, sentence repetition, verbal fluency, abstraction, and delayed recall 
and orientation. We will conduct this survey on the pre-screening phone call. 
 
IVI-VLV (Impact of Visual Impairment – Very Low Vision) is a standard survey to assess vision-related 
quality of life in persons with severe vision loss (Finger 2014). This survey will be administered prior to 
testing and may be administered over the phone as part of the pre-screening phone call. 
 
SmartHMD in-test questions. We will ask the participants to rate their trust in the system and to rate the 
mental energy required to use the system.  
 
SmartHMD post-test questions.  After testing, we will ask the user follow up questions to obtain their 
impressions from testing.  
 
SmartHMD follow up phone call. We will contact the participant 1-2 weeks after testing to ask them a series 
of follow up questions, since they may have more thoughts on the system design.  
 
d. Data Analysis 
Data analysis will be done to determine which configuration of the smartHMD device improves O&M in 
study participants who have LV, and if improvements are significantly better than baseline mobility. 
 
The peer-reviewed literature provides estimates of mean PPWS and variability within groups of LV subjects. 
PPWS can be viewed as a surrogate measure for completion time, and its reported variability among LV 
subjects is useful for sample size estimation. Other groups (Soong, 2004) have reported on two visually 
impaired groups (n=19 & n=18); the mean (SD) PPWS values for the two groups were 40.1% (12.7%) and 
40.8% (13.1%). Thus, we estimate the SD to be 33% (~13/41) of the mean completion times for both 
baseline and smartHMD conditions. Our prior work (Adebiyi 2017) compared guided mobility (with verbal 
or vibrational feedback) versus no guidance and found significant differences in completion time with n=11 
participants. Thus our subject pool of 30 should be adequate to yield rigorous results. 
 
Given the small sample size, an informative way to present the data will be through listings of the raw data. 
Listings will show subject disposition, demographic and baseline characteristics, extent of exposure and 
study termination/withdrawal information, as well as efficacy information. Descriptive statistics (number and 
percentage for categorical data; mean, median, standard deviation, inter-quartile range, min/max, and 
number for continuous data) will be presented for each evaluable for differences in outcomes control vs. 
smartHMD condition. For discrete variables, descriptive analyses will be based on numbers of subjects and 
related percentages.  
 
Interviews will be audio-recorded, transcribed verbatim, and analyzed using inductive qualitative analysis. 
Additionally, mixed methods analyses, such as joint display tables, will be used to integrate qualitative 
findings with vision-related quality of life data from the IVI-VLV. These analytic approaches will provide 
insights on how user preferences vary based on vision-related quality of life across various domains. 
 
Data from the IVI-VLV correspond to two domains: Activities of Daily Living, Mobility and Safety and 
Emotional Wellbeing. Survey response data will be scored using classical approaches (e.g. Likert scoring). 
Modern psychometric approaches are preferred, however. Thus, if a Rasch-analysis produces stable items 
calibrations (e.g. +/- ½ logit with 95% confidence), IVI-VLV response data will be Rasch-adjusted. 
 
A score of 18 (out of 22) on MoCA-Blind is considered normal. This is not used for statistical analysis but 
as a screening tool. 

 
All data listings, summaries, figures, and statistical analyses will be generated using SAS Version 9.3 or 
higher (SAS Institute, Cary, NC), R (R Foundation for Statistical Computing, Vienna, Austria), or other 
validated statistical software.  
 
e. HMD descriptions  
i). ODG R-7 Smartglasses 
The ODG R-7 Smartglasses (ODG, San Francisco) uses the camera on the R-7 to obtain scene information 
for the detection and guidance modules. The detection module identifies places or objects of interest in a 
scene while tracking and detecting any changes. The guidance module plans an optimal path, based on 
the detected objects, and guides the user to the destination using audio and/ or vibration feedback.  
 
ii). Manually operated auditory feedback and tactile stimulator array 
This device allows an operator to control the timing, type, and frequency of feedback provided to the 
participant. It consists of a custom app, a mobile device (such as a smart phone), an electronics module to 
communicate with the mobile device and control the user interface, and the user interface, which includes 
headphones for verbal feedback and a tactile stimulator array for vibrational feedback. The tactile stimulator 
array will be worn on the head, like a headband, around the waist like a belt, or on the handle of a long 
cane (used for obstacle detection by visually impaired people). The tactile stimulator array will have 3-5 
vibration motors. The entire system is battery powered.   
 
iii). Computer Vision Navigation prototyping system 
This system will provide the flexibility and capability needed to develop computer vision algorithms for 
navigation assistance for the visually impaired. The system consists of two components: The Intel 
RealSense D435i camera and a computer (either an Alienware M15 laptop or a Jetson Mobile Computer). 
Novel software for detection, mapping, guidance, and recording will be developed and run on the computer. 
A separate camera for patient tracking may be included. The cameras are connected to the computer via 
USB cables. The computer is in a backpack or satchel worn by the user. All devices are powered through 
a battery making this system fully portable. The camera is mounted on the user’s head with a head strap. 
The camera’s video is sent to the computer where the computer vision algorithms analyze the scene and 
decide on a path for the user to follow. The guidance algorithm will send commands to one of several 
outputs, depending on the experiment. The output devices are 1) the tactile stimulator array 2) audio output 
of the computer, for verbal guidance commands 3) video output of the device for display on a head-mounted 
display or as input to the Argus II Video Processing Unit. In the case of the output device being the VPU, 
the VPU will accept standard video format of NTSC and will stimulate the retina of the patient via the 
implanted retinal stimulator, according to the received video signal.  
 
 
References 
(Adebiyi 2017) - Adebiyi, Aminat, Paige Sorrentino, Shadi Bohlool, Carey Zhang, Mort Arditti, Gregory 
Goodrich, and James D. Weiland. "Assessment of feedback modalities for wearable visual aids in 
blind mobility." PloS one 12, no. 2 (2017): e0170531. 
(Brown 2014) - Brown JC, Goldstein JE, Chan TL, Massof R, Ramulu P. Characterizing functional 
complaints in patients seeking outpatient low-vision services in the United States. Ophthalmology. 
2014;121(8):1655-1662.e1651. 
(Ehrlich 2016) - Ehrlich JR, Ojeda LV, Wicker D, Day S, Howson A, Lakshminarayanan V, Moroi SE. 
Head-Mounted Display Technology for Low Vision Rehabilitation and Vision Enhancement. Am J 
Ophthalmol. 2016 Dec 31; PMID: 28048975 
(Finger 2014) - Finger RP, Tellis B, Crewe J, Keeffe JE, Ayton LN, Guymer RH. Developing the 
impact of Vision Impairment-Very Low Vision (IVI-VLV) questionnaire as part of the LoVADA protocol. 
Investigative Ophthalmology & Visual Science. 2014 Sep 4;55(10):6150-8 
(Geruschat 1998) - Geruschat DR, Turano KA, Stahl JW. Traditional measures of mobility 
performance and retinitis pigmentosa. Optom Vis Sci. 1998;75(7):525-537. 

(Goldstein 2012) - Goldstein JE, Massof RW, Deremeik JT, et al. Baseline traits of low vision patients 
served by private outpatient clinical centers in the United States. Archives of ophthalmology. 
2012;130(8):1028-1037. 
(Kuyk 1996) - Kuyk T, Elliott JL, Biehl J, Fuhr PS. Environmental variables and mobility performance 
in adults with low vision. Journal of the American Optometric Association. 1996;67(7):403-409. 
(Montarzino 2007) Montarzino A, Robertson B, Aspinall P, et al. The impact of mobility and public 
transport on the independence of visually impaired people. Visual Impairment Research. 2007;9:67-
82. 
(Owsley 2004) - Owsley C, McGwin G, Jr. Depression and the 25-item National Eye Institute Visual 
Function Questionnaire in older adults. Ophthalmology. 2004;111(12):2259-2264. 
(Patel 2006) - Patel I, Turano KA, Broman AT, Bandeen-Roche K, Munoz B, West SK. Measures of 
visual function and percentage of preferred walking speed in older adults: the Salisbury Eye 
Evaluation Project. Investigative ophthalmology & visual science. 2006;47(1):65-71. 
(Paz 2003) Paz SH, Globe DR, Wu J, Azen SP, Varma R. Relationship between self-reported 
depression and self-reported visual function in Latinos. Archives of Ophthalmology. 
2003;121(7):1021-1027.  
(Rizzo 2018) - Rizzo J-R, Conti K, Thomas T, Hudson TE, Wall Emerson R, Kim DS. A new primary 
mobility tool for the visually impaired: A white cane—adaptive mobility device hybrid. Assistive 
Technology. 2018;30(5):219-225. 
 (Salive 1994) - Salive ME, Guralnik J, Glynn RJ, Christen W, Wallace RB, Ostfeld AM. Association of 
Visual Impairment with Mobility and Physical Function. Journal of the American Geriatrics Society. 
1994;42(3):287-292. 
 (Soong 2004) - Soong GP, Lovie‐Kitchin JE, Brown B. Measurements of preferred walking speed in 
subjects with central and peripheral vision loss. Ophthalmic and Physiological Optics. 
2004;24(4):291-295. 
 (Varma 2016) - Varma R, Vajaranant TS, Burkemper B, et al. Visual Impairment and Blindness in 
Adults in the United States: Demographic and Geographic Variations From 2015 to 2050. JAMA 
ophthalmology. 2016;134(7):802-809.  
(www.rpbusa.org) - Blindness RtP. Vision Loss Impact https://www.rpbusa.org/rpb/resources-and-
advocacy/advocacy/vision-loss-impact/. 2017. Accessed March 19, 2017, 2017.  
(www.who.org) - Organization WH. Vision 2020 The Right to Sight. Global Initiative for the elimination 
of avoidable blindness. http://www.who.int/blindness/Vision2020_report.pdf WHO 2007:19. 
(www.mocatest.org) – www.mocatest.org/paper-tests/moca-test-blind