Environmental Localization Mapping and Guidance for Visual Prosthesis Users
Phase: NA
Conditions: Retinitis Pigmentosa, Visual Impairment, Visual Prosthesis
Sponsor: Johns Hopkins University

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
1
ENVIRONMENTAL LOCALIZATION MAPPING AND 
GUIDANCE FOR VISUAL PROSTHESIS USERS 
NCT04359108 
 
11/1/2023 
 
 
SINGLE IRB PROTOCOL 
 
 
IRB OF RECORD: JOHNS HOPKINS IRB 
 
 
 
 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
2
TABLE OF CONTENTS 
A 
INTRODUCTION ............................................................................................................................... 3 
A1 
STUDY ABSTRACT ......................................................................................................................... 3 
A2 
PRIMARY HYPOTHESIS ............................................................ERROR! BOOKMARK NOT DEFINED. 
A3 
PURPOSE OF THE STUDY PROTOCOL .............................................................................................. 4 
B 
BACKGROUND .................................................................................................................................. 5 
B1 
PRIOR LITERATURE AND STUDIES .................................................................................................. 5 
B2 
RATIONALE FOR THIS STUDY ......................................................................................................... 6 
C 
STUDY OBJECTIVES ....................................................................................................................... 7 
C1 
PRIMARY AIM ................................................................................................................................ 7 
C2 
RATIONALE FOR THE SELECTION OF OUTCOME MEASURES ........................................................... 7 
D 
STUDY DESIGN ................................................................................................................................. 7 
D1 
OVERVIEW OR DESIGN SUMMARY ................................................................................................. 7 
D2 
SUBJECT SELECTION AND WITHDRAWAL ..................................................................................... 10 
2.a 
Inclusion Criteria ................................................................................................................... 10 
2.a 
Exclusion Criteria ................................................................................................................... 11 
2.b 
Subject Recruitment Plans and Consent Process ................................................................... 11 
2.c 
Risks and Benefits ................................................................................................................... 12 
2.d 
Early Withdrawal of Subjects ................................................................................................. 14 
2.e 
When and How to Withdraw Subjects ..................................................................................... 14 
2.f 
Data Collection and Follow-up for Withdrawn Subjects ............................................................ 14 
D3 
STUDY DEVICE ............................................................................................................................ 14 
3.a 
Description ............................................................................................................................. 14 
3.b 
Treatment Regimen ................................................................................................................. 16 
E 
STUDY PROCEDURES ................................................................................................................... 16 
E1 
SCREENING FOR ELIGIBILITY ....................................................................................................... 16 
E2 
SCHEDULE OF MEASUREMENTS ................................................................................................... 16 
E3 
VISIT 1 ......................................................................................................................................... 16 
E4 
FOLLOWING VISITS ...................................................................................................................... 16 
E5 
SAFETY AND ADVERSE EVENTS ................................................................................................... 16 
5.a 
Medical Monitoring ................................................................................................................ 18 
i 
Investigator only ................................................................................................................................... 18 
5.b 
Definitions of Adverse Events ................................................................................................. 18 
5.c 
Classification of Events .......................................................................................................... 18 
5.d 
Data Collection Procedures for Adverse Events .................................................................... 18 
5.e 
Reporting Procedures ............................................................................................................. 18 
5.f 
Adverse Event Reporting Period ................................................................................................. 19 
5.g 
Post-study Adverse Event ....................................................................................................... 19 
E6 
STUDY OUTCOME MEASUREMENTS AND ASCERTAINMENT ......................................................... 19 
F 
STATISTICAL PLAN ....................................................................................................................... 19 
F1 
SAMPLE SIZE DETERMINATION AND POWER .................................................................................... 19 
F2 
ANALYSIS PLAN ............................................................................................................................... 20 
F3 
STATISTICAL METHODS ................................................................................................................... 20 
G 
DATA HANDLING AND RECORD KEEPING ............................................................................ 20 
G1 
CONFIDENTIALITY AND SECURITY ............................................................................................... 20 
G2 
TRAINING..................................................................................................................................... 21 
G3 
RECORDS RETENTION .................................................................................................................. 21 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
3
G4 
PERFORMANCE MONITORING ...................................................................................................... 21 
H 
STUDY MONITORING, AUDITING, AND INSPECTING ......................................................... 21 
H1 
STUDY MONITORING PLAN .......................................................................................................... 21 
H2 
AUDITING AND INSPECTING ......................................................................................................... 21 
I 
STUDY ADMINISTRATION .......................................................................................................... 21 
I1 
ORGANIZATION AND PARTICIPATING CENTERS ................................................................................ 21 
I2 
FUNDING SOURCE AND CONFLICTS OF INTEREST ............................................................................. 22 
I3 
SUBJECT STIPENDS OR PAYMENTS ................................................................................................... 22 
I4 
STUDY TIMETABLE .......................................................................................................................... 22 
J 
PUBLICATION PLAN ..................................................................................................................... 22 
K 
ATTACHMENTS .............................................................................................................................. 22 
K1 
INFORMED CONSENT DOCUMENTS ................................................................................................ 22 
L 
REFERENCES .................................................................................................................................. 22 
M 
SAFETY PROTOCOLS FOR COVID-19 ....................................................................................... 23 
 
 
A Introduction 
A1 Study Abstract 
 
About 1.3 million Americans aged 40 and older are legally blind, a majority 
because of diseases with onset later in life, such as glaucoma and age-related 
macular degeneration [1]. Second Sight has developed the world’s first FDA 
approved retinal implant, Argus II, intended to restore some functional vision for 
people suffering from retinitis pigmentosa (RP). 
 
The Argus II Retinal Prosthesis System developed by Second Sight is a medical 
device regulated by the FDA’s Humanitarian Device Exemption H110002. The 
Argus II system comprises three sub-system components: the internal Implant, the 
external Video Processing Unit (VPU) and Glasses, and the supporting Clinical 
Fitting System. 
 
In this era of smart devices, generic navigation technology, such as GPS mapping 
apps for smartphones, can provide directions to help guide a blind user from point 
A to point B. However, these navigational aids do little to enable blind users to form 
an egocentric understanding of their surroundings, they are not suited to navigation 
indoors, and do nothing to assist in avoiding obstacles to mobility. The Argus II, on 
the other hand, provides blind users with a form of artificial vision that provides 
limited visual representation of their surroundings that improves a user’s ability to 
orient themselves and traverse obstacles, yet the user’s vision remains 
significantly limited by the low resolution and dynamic range of the system which 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
4
is thus limited in its ability to provide high-level navigation and semantic 
interpretation of the surroundings. The proposed study aims to address these 
limitations of the Argus II through a synergy of artificial vision and state-of-the-art 
simultaneous localization and mapping (SLAM) and object recognition 
technologies.  
 
Users of the Argus II device will participate in the study while using their Argus II 
device according to its intended use. Other non-medical devices that will be used 
as part of this study include the following (see Section D3): 
• Headphones for receiving auditory information about the environment; 
• Vibrotactile stimulator to perceive, through vibrations on a participant’s skin, 
information about the environment; 
• Virtual reality headset (used only by normal-sighted participants in order to 
simulate the visual experience of a prosthetic vision user) 
• Portable central processing unit to integrate all environmental information 
and relay the information in informative ways back to the user using the 
various forms of sensory feedback, including the Argus II system or a head 
mounted display (HMD), the headphones, and a vibrotactile stimulator. 
 
This study is driven by the hypothesis that orientation and navigation performance 
for users of retinal prosthetics can be greatly improved by incorporating SLAM, 
stereo/depth based and object recognition technologies conveying spatial 
environmental information via a combination of artificial vision and auditory and/or 
haptic feedback. SLAM enables the visual prosthesis system to construct a map 
of the user’s environment and locate the user within that map. Ultimately, we 
expect that this technology will allow users to create an effective spatial image of 
the environment, that is, a continuously updated representation of locations in 3D 
space that supports volitional action for independent navigation with minimal 
cognitive load.  
A2 Purpose of the Study Protocol 
 
This is a prospective observational case series study. We will develop and test a 
navigational aid system which 1) constructs a map of unfamiliar environments and 
localizes the user in that environment using SLAM technology 2) automatically 
identifies navigationally-relevant objects and landmarks in that environment using 
object recognition and 3) provides sensory feedback via multiple modalities for 
navigation, obstacle avoidance, and object/landmark identification. 
 
We will test the system’s potential with both Argus II users and with normally 
sighted participants (by simulating a retinal prosthesis via a head-mounted display 
that emulates the capabilities of the Argus II device).  
 
The human subject tests fall into three groups: in Group 1, target identification and 
localization tasks test the basic capabilities of the system for conveying 
characteristics (e.g. shapes, sizes, movement direction) and location information 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
5
about objects that would be targets for navigation or action. In this group, 
participants will be stationary and only be asked to rotate/pan their head to acquire 
camera-generated targets. In Group 2, egocentric localization and spatial image 
updating tasks test how well the system provides spatial parameters, particularly 
distance, from optical flow and non-visual sources, leading to the formation of the 
spatial image. Finally, Group 3 tasks test the system in its function of guiding 
navigation and assess learning effects.  
 
In all three groups, various instantiations of the system are contrasted as follows. 
The Basic system is the standard Argus II device. The Augmented system 
enhances the imagery for visual feedback compared to the standard Argus II 
system by decluttering the scene and adding discriminative features to convey 
object identities. The features to be utilized will be determined after tests in Group 
1, but the general goal is to add spatial and/or temporal uniqueness without 
cluttering the display. The third system, the Augmented + Modal system, adds 
non-visual cues. Initially, we are targeting two non-visual augmentation modalities 
including (i) verbal and non-verbal auditory feedback along with spatial sound for 
purposes such as to convey the identity, distance, and direction of objects relative 
the user and (ii) haptic feedback for purposes such as to direct users how to adjust 
their head azimuth and elevation in order to acquire a landmark or waypoint within 
their field of view. 
 
The enhanced imagery information for visual feedback used in the Augmented 
and Augmented + Modal modes of the system will be computed by the navigation 
system developed in this study and communicated to the Argus II using a built-in 
video input on the Argus II that enables visual information provided from an 
external source to be displayed by the user’s implant. No alterations, including 
software or hardware changes, will be made to a participant’s Argus II system 
during this study. 
 
B Background 
B1 Prior Literature and Studies 
 
The Argus II system is a fully portable medical device approved by the FDA for 
commercial sale in the United States under the Humanitarian Device Exemption 
(HDE) program. It consists of implanted and external components (see Figure 1). 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
6
The implant is an epiretinal prosthesis that includes a 
receiver, electronics, and an electrode array that are 
surgically implanted in and around the eye. The array 
has 60 electrodes arranged in a 10x6 rectangular grid 
[2], [3]. It is attached to the retina over the macula with 
a retinal tack. The external equipment includes 
glasses, a video processing unit (VPU) and a cable. 
The glasses include a miniature video camera, which 
captures video images, and a coil that transmits data 
and stimulation commands to the implant. The VPU 
converts the video images into stimulation commands 
and is body-worn. The cable connects the glasses to 
the VPU. The Argus II system operates by converting 
video images into electrical energy that activates 
retinal cells, delivering the signal through the optic 
nerve to the brain where it is perceived as light.  
 
Retinal implant technologies are just emerging, and the Argus II currently serves 
as a complementary device to the long cane and guide dog for complex tasks, 
such as O&M. Due to the low resolution and highly limited field of view of the retinal 
implant, visual prosthesis users must use continuous head scanning, both 
horizontal and vertical, to observe the visual field of normal human vision. This skill 
has been trained and demonstrated in previous studies [4], [5] which were based 
on “radar vision” [6], a scheme using head scanning to locate a high-contrast object 
in an uncluttered background. 
 
Our proposed efforts focus on technology that will help develop the Argus II, and 
other prosthetic vision devices, into stand-alone aids capable of enabling people 
with profound vision loss to regain their independence, just as cochlear implants 
developed from lip-reading aids into fully functional devices capable of restoring 
speech understanding in as little as two decades [7]. 
B2 Rationale for this Study 
 
To enhance the Argus II capabilities, we will be adding recognition-enabled 
navigational technologies for these visual prosthesis users. This will consist in 
delivering enhanced visual cues optimized for user recognition and to provide 
auditory / verbal cues as well as haptic information through vibrotactile stimulators. 
 
We will recruit approximately 25 to 35 normally sighted adults, and approximately 
5 to 15 retinal prosthesis users. We will test sighted participants in order to 
establish baseline levels for processing the information delivered when the optic 
nerve is healthy and intact. We do not claim that Argus II users will be comparable 
in performance; in fact, testing will be directed at the implanted users to 
characterize a range of performance compared to baseline.  
 
Figure 1 - Argus II Retinal 
Prosthesis System 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
7
Our Johns Hopkins University Applied Physics Laboratory (JHU/APL), Johns 
Hopkins Medicine Wilmer Eye Institute (JHM/Wilmer), and Second Sight teams 
have been collaborating for the past several years to advance the capabilities of 
the Argus II system and improve performance of Argus II recipients, including 
developing capabilities for object recognition and stereo vision, and have extensive 
experience in designing and developing SLAM software and algorithms in the field 
of robotics. In addition to the investigators at Wilmer Eye Institute and Second 
Sight who have extensive experience in working with retinal prosthesis users, our 
team includes Dr. Roberta Klatzky, who has a long history of research working with 
visually impaired individuals and studying the impact of human neurophysiology 
and psychology on the design of instruments and devices that can be used as 
navigational aids for the blind. 
 
C Study Objectives 
C1 Primary Aim 
 
The primary aim of this study is to ascertain whether our system allows Argus II recipients 
to form a “spatial image” of their environment, that is, a continuously updated 
representation of locations in three dimensional space that supports volitional action for 
independent navigation.  
C2 Rationale for the Selection of Outcome Measures 
 
The outcome measures derive directly from the goals of this study, namely the ability to 
identify and localize targets, with and without movements, and navigate through a field 
comprised of these targets.  
 
D Study Design  
D1 Overview or Design Summary 
 
Testing will be conducted at JHU/APL, JHM/Wilmer and at JHU Homewood 
campus. 
 
This study will be conducted with three different sets of participants.  
• One set of eligible participants will consist of Argus II users, with a total of 
approximately 5 to 15 such participants expected to be recruited for this 
study. These tests may take place across all test sites. 
• A separate set of eligible participants will consist of normally sighted 
individuals from JHU/APL’s staff of over 7,000. To simulate the visual 
perception of the Argus II users, these participants will be equipped with a 
head-mounted display, such as an Oculus Rift, that simulates the field of 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
8
view and pixelated array of the retinal prosthesis. A total of approximately 
25 to 35 such participants are expected to be recruited for this study. These 
tests will take place at JHU/APL. 
• Another set of eligible participants will consist of normally sighted individuals 
from interested students and staff from JHU Homewood campus. Similar to 
the JHU/APL study group, these participants will be equipped with a head-
mounted display that simulates the field of view and pixelated array of the 
retinal prosthesis. A total of approximately 10 such participants are 
expected to be recruited for this study. These tests are expected to take 
place at JHU Homewood campus, but could utilize JHU/APL campus if 
needed. 
 
For information regarding recruitment and informed consent for these participant 
groups, refer to Section D2.b. 
 
The study itself will consist of two phases which are expected to be conducted over 
the course of 1-5 testing days (longer for Argus II users, due to setup time 
requirements, shorter for normally sighted subjects). In the first phase (Group 1 
and Group 2 tasks, see Section A2 and below), subjects will be trained to identify 
and localize objects within an unfamiliar environment by providing them 
information either via the Argus II device or the head mounted display that 
simulates the perception of a visual prosthesis. These tests will be conducted to 
assess the capabilities of the system for conveying identity and location 
information about objects that could be targets for navigation or action. Compared 
with the Basic system, we will then assess how well the augmented systems 
(Augmented and Augmented + Modal systems, refer to Section A2) function in 
providing spatial parameters, particularly distance, using visual but also non-visual 
sources of information including haptic/vibrotactile and auditory feedback. 
 
The identification and localization tests (Group 1) use common psychophysical 
judgments such as the following: discriminating between two alternatives by saying 
which is present in the field of view, touching a tablet computer to indicate the 
apparent location of a target in the frontal plane, and looking for targets by rotating 
the head and pointing when one is found. In the egocentric localization and spatial 
updating tasks (Group 2), the participants will make similar psychophysical 
judgments, but now in the context of movement, for example requiring the 
participants to walk several meters towards a visible target. For example, in some 
trials participants may be asked to move backwards to the origin (with aid of a 
guide), then verbally estimate the distance to the target or point with a joystick 
toward it. Similarly, in other trials, after walking toward the target with vision 
present, they may be asked to move forward a short distance without vision and 
make the same types of judgments.  
 
The second phase of testing (Group 3 tasks), will involve using the information 
gleaned from the first phase to assess participants’ ability to navigate through an 
unfamiliar environment. This phase will again compare user performance using the 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
9
Basic system with user performance using the Augmented and Augmented + 
Modal systems where contextual information associated with the task is conveyed 
to users via enhanced visual, auditory, and haptic modalities. 
 
Our algorithms for mapping the environment and localizing the user will provide 
the core autonomous navigational capability of the system. As the user traverses 
their environment, object recognition will be used in concert with the spatial 
localization and mapping technology to automatically detect key navigational 
features (e.g., doors, signs, etc.) and store these locations in the map. Once stored 
in the map, these semantic attributes may then be verbally queried by the user for 
navigation, e.g., “lead me to the nearest restroom”.  
 
The enhanced imagery generated by the Augmented and Augmented + Modal 
systems will be achieved without modifying the Argus II device. The imagery 
content for these modes will be computed by algorithms of the navigation system 
and then transmitted via video link to the Argus II for display to the Argus II user.  
 
Figure 2 represents an example scenario where an Argus II user has requested 
guidance to the nearest door. The door has already been identified by object 
recognition during algorithm testing and stored in the system’s map of the 
environment. The path to the door is impeded by a couch, which is also stored in 
the map as an obstacle labeled by object recognition. Upon receiving the user 
request, the system computes a path around the couch to the door and initiates 
navigational cues to guide the user along the planned path. As the user progresses 
along this path, the door is enhanced (brightened) in artificial vision as being the 
target destination. Nearby obstacles (such as the couch) are also shown through 
the prosthesis (or the simulated prosthesis for normally sighted participants), while 
the system’s stereo-vision-based obstacle detection feature continually monitors 
the scene for both mapped and previously undetected obstacles along the way. 
 
In addition to the navigation itself, participants will provide numerical ratings on 
scales to indicate the mental (cognitive) load of the task, perceived effort, and ease 
of travel. Participants will also provide qualitative answers to questions pertaining 
to their experience using the system, such as what elements of the system were 
more or less helpful and how they believe the system could be improved. 
 
During the consent process, participants will be given the opportunity to consent 
to audio-video recordings being taken of them while performing the tasks of this 
study.  Consenting to audio-video recording is not mandatory for subjects to 
participate in the study.  For those participants who consent to audio-video 
recording, the recordings may be used to help analyze the participant’s 
performance in the study and may be used in presentations of the methods and 
outcomes of the study before a general audience.  Audio-video recordings will not 
be used for advertising or non-study related purposes. 
 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
10
 
Figure 2 - Example navigation scenario for a blind user; although not shown in the figure, 
information communicated by the system to the user also enables the user to mentally form 
an egocentric spatial image of their surroundings. 
 
D2 Subject Selection and Withdrawal  
2.a 
Inclusion Criteria  
 
Criteria for inclusion of normally sighted individuals: 
• Subject speaks English;  
• Subject is an adult (at least 18 years of age); 
• Subject has the cognitive and communication ability to participate in the 
study (i.e., follow spoken directions, perform tests, and give feedback);  
• Subject is willing to conduct psychophysics testing up to 4-6 hours per day 
of testing on 3-5 consecutive days;  
• Subject has visual acuity of 20/40 or better (corrected); 
• Subject is capable of understanding participant information materials and 
giving written informed consent. 
• Subject is able to walk unassisted 
 
Criteria for inclusion of Argus II users: 
The inclusion criteria for the study are the following: 
• Subject is at least 25 years of age; 
• Subject has been implanted with the Argus II system;  
• Subject’s eye has healed from surgery and the surgeon has cleared the 
subject for programming; 
• Subject has the cognitive and communication ability to participate in the 
study (i.e., follow spoken directions, perform tests, and give feedback);  
• Subject is willing to conduct psychophysics testing up to 4-6 hours per day 
of testing on 3-5 consecutive days;  

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
11
• Subject is capable of understanding patient information materials and giving 
written informed consent; 
• 
Subject is able to walk unassisted (for participation in mobility tasks). 
 
The aim for the normally sighted group is to avoid any visual acuity issues that 
could impact the participant’s performance in the study. Therefore, we require a 
visual acuity of 20/40 or better with correction, which is sufficient to perceive the 
low-resolution visual feedback that will be displayed to sighted users using the VR 
headset. We also require sighted participants to be adults, as experiments with 
this group simulate a medical device that is used by only adults. For the Argus II 
group, the minimum FDA-approved age for use of the Argus II device is 25. We do 
not require a maximum age limit for this study group, but do require that 
participants are able to walk unassisted for participation in mobility tasks. 
Participants in the Argus II group who are not able to walk unassisted may still 
participate in non-mobility tasks, such as stationary tasks that evaluate the 
participant’s ability to perceive and understand various forms of visual feedback 
provided to their retinal implant. 
2.a 
Exclusion Criteria  
 
Exclusion criteria for all subjects is the following: 
• 
Subject is unwilling or unable to travel to testing facility for at least 3 days 
of testing within a one-week timeframe; 
• Subject does not speak English; 
• Subject has language or hearing impairment. 
 
The requirement that subjects speak English and do not have a language or 
hearing impairment is due to the fact that this study involves use of a voice-enabled 
navigational aid that subjects will interact with via a speech interface both to control 
the system and to receive specific forms of information feedback from the system 
regarding the subjects’ surroundings. The voice-enabled device leverages 
autonomous speech recognition and speech synthesis capabilities with support for 
only the English language. Therefore, it is required for subjects to both speak and 
hear English in order to fully utilize the capabilities of the navigational aid being 
developed for this pilot study; however, future accommodations may be made for 
non-English speakers or hearing impairments. 
 
2.b 
Subject Recruitment Plans and Consent Process 
 
As described in Section D1, this study will be conducted with three different sets 
of participants.  
• The Argus II user group of participants will be recruited from Argus II users 
who have given their consent to be reached for human subject tests in 
previous non-significant risk studies. Subjects will learn about the study via 
phone call from a member of our study team. 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
12
• A normally sighted group of participants will be recruited from JHU/APL’s 
staff of over 7,000. Interested candidates will be informed of the study 
through IRB-approved lab-wide human subject testing email distributions. 
Interested 
candidates 
will 
contact 
the 
PI 
and, 
if 
they 
meet 
inclusion/exclusion criteria, they will coordinate a time with the study team 
that would allow for experiments to take place. 
• Another normally sighted group of participants will be recruited from JHU 
Homewood’s students and staff. For this group of JHU Homewood 
participants, all direct recruitment will be conducted only by Nicolas Norena 
Acosta, a member of the study team who is both a full-time staff member of 
APL and a graduate student within the Whiting School of Engineering and 
who is not involved in the academic supervision of students at JHU 
Homewood. This recruitment by Nicolas Norena Acosta will be conducted 
by word-of-mouth and/or through IRB-approved human subject testing 
email distributions. Interested candidates will contact Nicolas Norena 
Acosta and, if they meet the inclusion/exclusion criteria, they will coordinate 
a time with the study team that would allow for experiments to take place. 
Prior to beginning the study, the PI or study team members qualified to consent, 
will walk each interested candidate through the consent form, providing additional 
details about the study, and if a candidate is still interested, consents and has 
time, they will be able to begin the first testing day immediately after providing 
informed consent. Study participants are also free to leave the study at any time. 
 
2.c 
Risks and Benefits 
 
This is a non-significant risk study in which no medical procedures will be 
performed. It is not a treatment, and it has no impact on healthcare or health itself. 
There will be no exchange of information with physicians. 
 
Elements of minimal risk are described herein. We divide the risks into three 
groups: common to all participants, Argus II users only, and normally sighted users 
only.  
 
As described in Section D1, the tasks associated with this study require a 
participant to walk slowly or stand in place while making simple judgments such as 
pointing to a target in space, judging distance, and discriminating identities or 
direction of motion of target objects. 
 
Risks for All Participants: 
Navigating through an unfamiliar environment with limited visibility (delivered 
through the HMD for normally sighted participants) involves a risk of tripping or 
stumbling over obstacles deliberately placed in the environment. Our team has 
worked in this kind of testing and to-date this risk has never materialized. This is 
due to a combination of ensuring the participants understand the requirement to 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
13
walk slowly toward their intended targets, as well as team attentiveness and 
experience surrounding the execution of these tasks while standing in close 
proximity to assist as necessary.  
  
The non-medical devices used in this study, including headphones and vibrotactile 
stimulators, have insignificant risk similar to use of headphones and vibration-
enabled devices such as a smart watch, a now very common device in everyday 
life. 
 
Additionally, subjects may become tired or fatigued during the testing. Experiments 
require focused concentration of the participants which may lead to fatigue. For 
similar reasons, minor discomforts such as headaches or backaches may occur. 
Participants may take as many breaks as they need and may discontinue testing 
at any time. 
 
Participants may also become frustrated or disappointed if they feel they are not 
performing well. We will remind participants that this technology is novel and has 
potential benefits for other Argus II users and others with very limited visual acuity. 
As mentioned above, we will ask participants whether they want to interrupt testing 
to discuss this in more detail or if they wish to withdraw altogether. 
 
The research information obtained from human subjects will consist of eligibility 
data, user-specific configuration settings for the Argus II device, sensor data 
required to create maps of the testing environment, and behavioral data (includes 
video). As a result, there is a risk of breach of confidentiality. This is mitigated by 
ensuring that all PII is stored on password protected computers at all sites.  
 
Risks for Argus II Users: 
For Argus II users there are no risks associated with using the Argus II device that 
are greater than the risks that participants face using their Argus II device in their 
everyday lives. In accordance with the device’s Safety and Probable benefit 
document (https://www.accessdata.fda.gov/cdrh_docs/pdf11/H110002B.pdf), the 
following precautions will be followed: 
• In the event of any undesirable sensation when using the device, all 
operations will be halted, and Argus II glasses will be removed or the 
Argus II Visual Processing Unit (VPU) will be turned off. 
• Argus II users will only use a VPU that has been specifically 
programmed for them by their clinician or Second Sight personnel. 
• Argus II users will continue to use their other mobility aides, such as a 
cane, at all times. 
 
Risks for JHU/APL Participants: 
Normally sighted study participants at JHU/APL will be asked to don a head 
mounted display for extensive periods of time. In addition to the common risks 
mentioned above, there is a risk of dizziness and fatigue caused by extensive use 
of the HMD. As above, this risk is mitigated by taking as many breaks as 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
14
participants request as well as interrupting testing and potentially resuming on a 
different day if requested by a participant. Here too, participants are free to 
withdraw from the study at any time. 
 
Benefits: 
Potential benefits that individual participants may experience from taking part in 
this research stems from the new sensory information afforded to prosthetic vision 
users by providing them with object recognition and navigational capabilities, as 
well as the addition of haptic and auditory sensory feedback to supplement 
prosthetic vision. This may help current and future prosthetic vision users to 
independently form mental maps of their surroundings using their prosthetic vision 
device and allow users to gain confidence in using the device for navigating new 
environments.  
2.d 
Early Withdrawal of Subjects  
 
In all cases, expected time commitments will be clearly communicated to 
participants ahead of time. It will also be clearly communicated to participants that 
they may choose to withdraw from the research at any time for any reason.  
2.e 
When and How to Withdraw Subjects  
 
Subjects will be withdrawn early from the study anytime upon subject request. It is also 
possible that other factors may contribute to the need for early withdrawal. For 
example, a malfunction of the system may cause us to have to stop testing. No 
special procedures are associated with withdrawing a subject early from this study 
other than to discontinue testing. Participants will be compensated for their time 
spent participating in the study prior to early withdrawal as described in Section I3. 
2.f 
Data Collection and Follow-up for Withdrawn Subjects  
 
In case of early withdrawal, no further data gathering nor follow-up will be conducted with 
the subjects.  As for all subjects, if a subject consents to be contacted for future 
research, they may be contacted later for future studies. 
D3 Study Device 
3.a 
Description  
 
The following devices are used in this study. No device will be used with any study 
participant prior to being reviewed and approved by Johns Hopkins’ clinical 
engineering services (CES).  
• Argus II Retinal Prosthesis System: used in this study by Argus II users 
who have a pre-existing implanted retinal prosthesis. This is an FDA 
approved device and will be used according to its intended use. 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
15
• Head Mounted Display: used in this study by normally sighted users to 
simulate the field of view and visual acuity provided by the Argus II 
device. For this device we will use a virtual reality headset, such as the 
Oculus Quest or HTC Vive or equivalent. Use of this device in the study 
will require review and approval by Johns Hopkins Clinical Engineering 
Services (CES) prior to being used in the study. 
• Embedded Processing Platform and Sensors: an embedded processing 
platform will be used to process information acquired by the various 
environmental sensors and to run the algorithms for image processing 
and navigation. Imagery displayed to users will be computed by this 
platform. For Argus II users, the computed images will be transmitted to 
the Argus II system for display via the retinal prosthesis, whereas, for 
sighted users, the computed images will be transmitted to the HMD 
device. The embedded processing platform and associated sensors will 
be battery powered. The system will be carried by the user via a beltpack 
or other suitable means of mobility, along with head gear for carrying the 
head-mounted sensors. This system will incorporate various sensors 
such as an RGB camera, depth sensor, and inertial measurement unit 
(IMU) for sensing objects and tracking the environment. Example 
environmental sensors that may be used include the Intel RealSense 
Depth Camera D435i and Intel RealSense LiDAR Camera L515 or 
similar. Use of the embedded platform and sensors will require review 
and approval by Johns Hopkins Clinical Engineering Services (CES) 
prior to being used in the study. 
• Wearable Haptic Device: used in this study to augment sensory 
feedback delivered back to the user, such as to guide the user in a 
direction to turn their head in order to visually locate an object of interest. 
This device will be battery-powered and designed to be worn and used 
by a human with insignificant risk. Specifically, we will embed haptic 
actuators (such as the Zorb vibrotactile actuators from Somatic Labs) 
within a body-worn device that when donned provides localized haptic 
feedback to signify directionality. For example, a neckband device may 
provide localized vibration points behind the neck, over each shoulder, 
and in front of the chest, with each vibration point signifying one of the 
four cardinal look directions (up, down, left, and right). This device will 
also require review and approval by Johns Hopkins Clinical Engineering 
Services (CES) prior to being used in the study. 
• Wearable Audio Device: used in the study to augment sensory feedback 
delivered back to the user by using voice and/or audible tones to 
communicate, such as communicating the identity or locations of objects 
in the scene or communicating which direction to turn in order to 
advance towards a destination. For this device we will use either 
conventional or bone-conduction headphones such as the Aeropex from 
Aftershokz or equivalent or other form of wearable speakers such as the 
neck-worn Soundwear Companion from Bose or equivalent. This device 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
16
will also require review and approval by Johns Hopkins Clinical 
Engineering Services (CES) prior to being used in the study. 
3.b 
Treatment Regimen  
 
The Argus II device will be used in accordance with its intended use. All other 
devices will require IRB CES approval for use in human testing. 
 
E Study Procedures  
E1 Screening for Eligibility 
 
As described in Section D2.b, subjects will be recruited from normally sighted 
populations at the Johns Hopkins University Applied Physics Laboratory and JHU 
Homewood campus as well as from lists of users of the Argus II device who have 
participated in prior studies associated with their device and who have granted 
consent to be contacted for future studies. As long as interested candidates meet 
the inclusion criteria and are not rejected by the exclusion criteria they will be 
eligible to participate in this study. 
E2 Schedule of Measurements 
 
Testing is expected to be conducted over a 1-5 day period for each participant. We 
anticipate that testing will require less time for the normally sighted participants 
than for the Argus II users.  
E3 Visit 1  
 
Apart from the consent process that occurs at the beginning of the first visit, the 
testing performed during the study visits will be similar to each other. For the 
consent process, the PI or authorized study team member will guide interested 
subjects through the informed consent form. Following consent, participants will 
be asked to perform three groups of tasks, as described in Section D1. Participants 
will be informed during consent that while breaks will be provided during the course 
of testing, they are also free to interrupt testing at any time to take breaks. 
E4 Following Visits  
 
See Section E3. 
E5 Safety and Adverse Events  
 
Study team members will complete all IRB training required to conduct the experiments 
described in this application prior to being involved in any human subject testing. 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
17
 
Communication between sites is primarily facilitated via working conference meetings held 
at least once per month between key team members from each site.  Planning for 
significant protocol events regarding the study, such as amendments to study protocols, 
annual approvals, etc., will be discussed among all sites in these meetings, as well as 
communicated between sites via email where appropriate.  A Box share folder has also 
been set up to host and distribute information among all sites and will serve as a single 
point of reference for obtaining the most up-to-date information regarding the study, such 
as the most current version of the study protocol. 
 
Both the PI (Dr. Billings) and the site-PI’s at each study location will be responsible for 
monitoring participant safety, evaluating the progress of the study, reviewing procedures 
for maintaining the confidentiality of data, the quality of data collection, management, and 
analyses, in accordance with guidance provided by the IRB of record.  In case of protocol 
events or deviations at participating sites, any team member aware of the issue will 
promptly notify the PI (Dr. Billings) and site-PI, and a report will be promptly submitted to 
the IRB (and sponsoring agency if appropriate) per the JHM IRB’s prompt reporting 
requirements, as described further below. 
 
The research data will be reviewed every day that data are collected, and the accrued 
data reviewed again at the end of the study to ensure its validity and integrity. The PI will 
conduct study team meetings on at least a monthly basis to discuss study progress, data 
quality and any unanticipated problems or findings will be reported upon occurrence. 
Reports concerning the progress of the study and subject status will be provided in the 
study closure report or when requested by the IRB. 
 
If any protocol changes are needed, they will not be implemented prior to IRB approval, 
with possible exception of minor or administrative deviations which do not affect the 
scientific soundness of the research plan or the rights, safety, or welfare of human 
subjects. If a protocol deviation occurs which meets this definition for a minor deviation, 
the deviation will be reported to the JHM IRB at the time the continuing review application 
is submitted in eIRB using the Protocol Deviation Summary Sheet (R.F. 4). 
 
In the event of an emergency protocol deviation taken without prior IRB approval to protect 
the life or physical well-being of a participant, it will be reported to the sponsor and the IRB 
of record as soon as possible, but not later than 5 days after the emergency situation 
occurred. 
 
The research teams at all site locations will monitor subjects for adverse events throughout 
the study. Adverse events that fit the following criteria will be reported to JHM IRB within 
10 working days from the time when the principal investigators become aware of them per 
JHM IRB policy: 
 
• 
Event is UNEXPECTED (in terms of nature, severity, or frequency) given (a) 
the research procedures that are described in the protocol-related documents, 
such as the IRB-approved research protocol and informed consent document, 
and (b) the characteristics of the subject population being studied; 
• 
RELATED or POSSIBLY RELATED to participation in the study (possibly 
related means there is a reasonable possibility that the incident, experience, 
or outcome may have been caused by the procedures involved in the study); 
and 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
18
• 
Suggests that the study PLACES SUBJECTS OR OTHERS AT A GREATER 
RISK OF HARM (including physical, psychological, economic, or social harm) 
than was previously known or recognized. 
 
The following unanticipated problems that are not adverse events will also be reported to 
the IRB within 10 days of becoming aware of them: 
 
• 
Unanticipated problems that do not fit the definition of an adverse event, but 
which may, in the opinion of the investigator, involve risk to the subject, affect 
others in the study, or significantly impact the integrity of study data. An 
example of this might be a report of accidental destruction of a study record; 
• 
Unplanned protocol deviations/violations that have already occurred, that may 
adversely affect the rights, safety or welfare of subjects or the integrity of the 
study data, AND for which IRB approval was not obtained. 
 
In reporting any deviations or adverse events to the IRB, the PI will submit a report to the 
IRB of record in eIRB via a Further Study Action for Protocol Event Report. 
 
In addition to reports issued to the IRB, any unanticipated problem involving risks to 
subjects or others or any serious or continuing noncompliance with 45 CFR 46 or IRB 
requirements must be reported to the sponsoring agency (NIH) and to the Office for 
Human Research Protections (OHRP), HHS, per NIH policy. 
5.a 
Medical Monitoring  
i 
Investigator only 
N/A. This is a non-significant risk study and interested candidates will not be undergoing 
medical procedures as part of this study. 
5.b 
Definitions of Adverse Events 
 
An adverse event is any undesirable experience associated with the use of a 
medical product in a patient. Adverse events may occur during the course of this 
study. The medical device that will be used in these studies is the FDA HDE 
approved Argus II Retinal Prosthesis System.  
5.c 
Classification of Events 
 
See Section E5. 
5.d 
Data Collection Procedures for Adverse Events 
 
See Section E5. 
5.e 
Reporting Procedures 
 
See Section E5. 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
19
5.f 
Adverse Event Reporting Period 
 
See Section E5. 
5.g 
Post-study Adverse Event 
 
This is a non-significant risk study, no post-study adverse events are anticipated. 
E6 Study Outcome Measurements and Ascertainment 
 
Our study will determine whether the following hypotheses are correct:  
1. The augmented systems expand users’ ability to identify targets in the field 
of view and to usefully interpret the egocentric information obtained from 
the retinal prosthesis by augmenting the basic Argus II device with the 
capability of decluttering the scene and by providing discriminative 
information to convey object identities and locations. 
2. Aided by the augmented system, the retinal prosthesis is capable of 
enabling the formation of an externalized spatial image that aids navigation. 
 
These hypotheses will be tested statistically with sighted subjects (see Section F) 
by measuring proportion correct assessments as well as through bias and 
precision measurements. Argus II subjects will be assessed individually to 
determine whether augmentation leads to lower error, and their error will be 
quantified through comparisons with the sighted subjects. 
 
F Statistical Plan  
F1  Sample Size Determination and Power 
 
No significance testing will be possible with Argus II participants due to the small 
number of people expected to be recruited for each Argus II participant group in 
this study (approximately 5). 
 
Importantly, however, we will use the sighted subjects, who simulate the 
performance of Argus II users through the head mounted display conveying 
information similar to what is perceived by Argus II users, as a baseline. As 
described in G3, we characterize performance by Argus II users on any given task 
relative to this baseline.   
 
In particular, for tests with sighted subjects, we use within-subject factorial designs 
and test with conventional ANOVA statistics; inter-subject correlations between 
abilities tests can also be computed. We use sample sizes based on prospective 
power analysis.  A sample size of 20-30 subjects was calculated with [9] to achieve 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
20
a power greater than 0.80 at the 0.05 level of significance, using effect sizes from 
previous studies reporting perceived spatial layout in within-subject designs [10]. 
As a more general point, the experiments use a within-subject design, in which 
every subject provides a replicate of the entire study. In psychophysical research 
(i.e., where a physical variable is parametrically explored), these designs are 
typically done with populations on the order of 10-20 participants. The proposed 
sample is also consistent with similar studies of navigation by spatial updating 
(e.g., [11]). 
F2 Analysis Plan 
 
For each measure, the Argus II participants will be characterized by the z-score 
distance of their performance from the sighted participants (as computed with their 
mean and variance). We will use two other approaches to assess the Argus II 
participants’ performance: (i) Use standards from the literature for basic 
psychophysical effects, including setting .2 as an effective d’ score for 2- choice 
discrimination, based on Cohen’s d effect-size statistics; setting 25% as the 
expectation for a size difference threshold, and setting a standard of 15° for 
absolute pointing error based on screening cutoffs in previous work. (ii) Set 
standards by considering the device capabilities; in particular, use distance 
expressed in device pixels between target and pointing locations in order to 
measure localization error. Finally, we intend to test the effects of learning by 
assessing performance over time. For this, we will fit trends to the measures over 
repeated trials and assess the linear component for a positive slope. 
F3  Statistical Methods  
 
See previous sections for additional information about the methods employed for 
this study. 
 
G Data Handling and Record Keeping  
G1 Confidentiality and Security 
 
Every effort will be made to keep Personally Identifiable Information (PII) 
associated with the study confidential: (1) subjects will be assigned a code number 
and only the code number will be used to identify the limited clinical information 
that will be analyzed; (2) the computers on which the data will be stored are 
password protected and all of them are located in offices with key locks; (3) written 
documents concerning the study will be kept in locked cabinets; (4) all personnel 
involved in human subject testing will have completed the appropriate HIPAA 
training and be fully aware of the need for confidentiality regarding PII. No health 
information will be acquired for this study.  

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
21
G2 Training  
 
All study team members are required to take IRB compliance training. At a 
minimum, this consists in Basic Human Subjects Research (CITI) training, Health 
Privacy Issues for Researchers training, and Conflict of Interest and Commitment 
training per JHM IRB policy, in addition to Good Clinical Practice training as 
required by NIH. PIs are also required to take Research Ethics Workshops About 
Responsibilities and Duties of Scientists training.  
G3 Records Retention 
 
In accordance with IRB policy, records will be retained for five years after the date 
of the last publication associated with the study.  
G4 Performance Monitoring  
 
Data will be analyzed following each testing session to evaluate system 
performance.   
 
H Study Monitoring, Auditing, and Inspecting  
H1 Study Monitoring Plan  
 
This is a non-significant risk study. The PIs at each site are responsible for the 
monitoring and execution of the testing. The JHM IRB, as the single IRB of record, 
will be responsible for reviewing the study protocols, consent forms, and case 
report forms. 
H2 Auditing and Inspecting  
 
This is a non-significant risk study. The JHM IRB regularly conducts audits to 
ensure that studies are in compliance.  
 
I Study Administration 
I1 
Organization and Participating Centers 
 
The following centers are involved in the study described in this protocol:  
• Johns Hopkins University Applied Physics Laboratory, Laurel, MD 
• Johns Hopkins Medicine, Wilmer Eye Institute, Baltimore, MD 
• Johns Hopkins University Homewood Campus, Baltimore, MD 
• Carnegie Mellon University, Pittsburgh, PA 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
22
I2 
Funding Source and Conflicts of Interest 
 
This research is supported by National Institutes of Health (NIH) National Eye 
Institute (NEI) grant number 1R01EY029741-01.  
I3 
Subject Stipends or Payments  
 
Subjects recruited from the Applied Physics Lab will be paid for their time in 
accordance with APL policies. Subjects recruited from the JHU Homewood 
campus will be uncompensated. Argus II users will be reimbursed for their costs 
associated with the study, such as travel to the testing facility and lodging, and will 
be provided with meals throughout their participation in the study. 
I4 
 Study Timetable 
 
Testing will commence upon IRB approval of the study and continue through the 
duration of the grant, expected to end in July 2024. Requests for changes in 
research and extensions to the protocol will be made as necessary during this time. 
 
J Publication Plan  
 
Deidentified data and results from this study may be submitted for submission to 
peer-reviewed conferences and journals, as well as abstracts and presentations 
at sector specific conferences (e.g. NIH Brain Initiative meeting). 
 
K  Attachments  
 
K1 Informed consent documents 
 
Informed consent documents are attached. 
 
L References 
  
[1] National 
Eye 
Institute, 
NIH, 
“Blindness.” 
[Online]. 
Available: 
https://nei.nih.gov/eyedata/blind. 
[2] A. K. Ahuja et al., “Blind subjects implanted with the Argus II retinal prosthesis are able 
to improve performance in a spatial-motor task,” Br. J. Ophthalmol., vol. 95, no. 4, pp. 
539–543, Apr. 2011, doi: 10.1136/bjo.2010.179622. 
[3] M. S. Humayun et al., “Preliminary 6 month results from the ArgusTM II epiretinal 
prosthesis feasibility study,” in 2009 Annual International Conference of the IEEE 

Environmental Localization Mapping and Guidance 
 
November 2023 
 
 
 
______________________________________________________________________________________ 
 
23
Engineering in Medicine and Biology Society, Minneapolis, MN, 2009, pp. 4566–4568, 
doi: 10.1109/IEMBS.2009.5332695. 
[4] A. C. Nau, C. Pintar, C. Fisher, J.-H. Jeong, and K. Jeong, “A standardized obstacle 
course for assessment of visual function in ultra low vision and artificial vision,” J. Vis. 
Exp., no. 84, Feb. 2014, doi: 10.3791/51205. 
[5] A. C. Ho et al., “Long-term results from an epiretinal prosthesis to restore sight to the 
blind,” Ophthalmology, vol. 122, no. 8, pp. 1547–1554, Aug. 2015, doi: 
10.1016/j.ophtha.2015.04.032. 
[6] J.-H. Jung, D. Aloni, Y. Yitzhaky, and E. Peli, “Active confocal imaging for visual 
prostheses,” 
Vision 
Res., 
vol. 
111, 
pp. 
182–196, 
Jun. 
2015, 
doi: 
10.1016/j.visres.2014.10.023. 
[7] F.-G. Zeng, “Auditory prostheses: Past, present, and future,” in Cochlear Implants: 
Auditory Prostheses and Electric Hearing, vol. 20, F.-G. Zeng, A. N. Popper, and R. 
R. Fay, Eds. New York, NY: Springer New York, 2004, pp. 1–13. 
[8] FDA, 
“Humanitarian 
Device 
Exemption.” 
[Online]. 
Available: 
https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfhde/hde.cfm?id=H110002. 
[9] “GLIMMPSE.” [Online]. Available: glimmpse.samplesizeshop.org/#/. 
[10] B. Wu, R. L. Klatzky, and G. Stetten, “Visualizing 3D objects from 2D cross sectional 
images displayed in-situ versus ex-situ,” J. Exp. Psychol. Appl., vol. 16, no. 1, pp. 45–
59, Mar. 2010. 
[11] Q. He, T. P. McNamara, and J. W. Kelly, “Reference frames in spatial updating when 
body-based cues are absent,” Mem. Cognit., vol. 46, no. 1, pp. 32–42, Jan. 2018, doi: 
10.3758/s13421-017-0743-y. 
 
 
M Safety Protocols for COVID-19 
 
Per JHU IRB guidance, we will not enroll any participants in this study until the 
COVID-19 emergency for Tier 3 research studies has been lifted by the IRB.