Transforming a Quadruped into a Guide Robot for the Visually Impaired: Formalizing Wayfinding, Interaction Modeling, and Safety Mechanism
Authors: J. Taery Kim
Wenhao Yu
Yash Kothari
Bruce Walker
Jie Tan
Greg Turk
Sehoon Ha
Venue: CoRL 2023 Poster (2023)
Keywords: Assistive Robot
Autonomous Navigation
Interaction Modeling

Abstract: This paper explores the principles for transforming a quadrupedal robot into a guide robot for individuals with visual impairments. A guide robot has great potential to resolve the limited availability of guide animals that are accessible to only two to three percent of the potential blind or visually impaired (BVI) users. To build a successful guide robot, our paper explores three key topics: (1) formalizing the navigation mechanism of a guide dog and a human, (2) developing a data-driven model of their interaction, and (3) improving user safety. First, we formalize the wayfinding task of the human-guide robot team using Markov Decision Processes based on the literature and interviews. Then we collect real human-robot interaction data from three visually impaired and six sighted people and develop an interaction model called the "Delayed Harness" to effectively simulate the navigation behaviors of the team. Additionally, we introduce an action shielding mechanism to enhance user safety by predicting and filtering out dangerous actions. We evaluate the developed interaction model and the safety mechanism in simulation, which greatly reduce the prediction errors and the number of collisions, respectively. We also demonstrate the integrated system on an AlienGo robot with a rigid harness, by guiding users over 100+ meter trajectories.

Transforming Quadruped into a Guide Robot for the Visually Impaired: Formalizing Wayfinding, Interaction Modeling, and Safety Mechanism
J. Taery Kim Georgia Institute of Technology taerykinggatech.co edu
Wenhao Yu Google DeepMind magi weg.immmoodddeemnndee com
Yash Kothari
Bruce N. Walker
Georgia Institute of Technology ykothari3@gatech edu
Georgia Institute of Technology
bruce. walker@psych gatech. edu
Jie Tan
Greg Turk
Sehoon Ha
Google DeepMind jiet detan@deepmind.comm com
Georgia Institute of Technology turkScc, .ww.ch...c.cc................... edu
Georgia Institute of Technology sehoonha@gatech. edu
Figure 1: We develop a robot guide dog by formalizing the wayfinding task, developing an interac- tion model from the collected data (Left), and employing action shielding, to guide a user (Right).
Abstract:
This paper explores the principles for transforming a quadrupedal robot into guide robot for individuals with visual impairments. A guide robot has great potential resolve the limited availability of guide animals that accessible only two to three percent of the potential blind or visually impaired (BVI) users. To build a successful guide robot, our paper explores three key topics: (1) for- malizing the navigation mechanism of a guide dog and a human, (2) developing a data-driven model of their interaction, and (3) improving user safety. First, we for- malize the wayfinding task of the human-guide robot team using Markov Decision Processes based on the literature and interviews. Then we collect real human- robot interaction data from three visually impaired and six sighted people and develop an interaction model called the "Delayed Harness" to effectively simulate the navigation behaviors of the team. Additionally, we introduce an action shield- ing mechanism to enhance user safety by predicting and filtering out dangerous actions. We evaluate the developed interaction model and the safety mechanism in simulation, which greatly reduce the prediction errors and the number of colli- sions, respectively. We also demonstrate the integrated system on quadrupedal robot with rigid harness, by guiding users over 100+ trajectories.
Keywords: Assistive Robot, Autonomous Navigation, Interaction Modeling
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

Introduction
Guide dogs play crucial role in the lives of blind or visually impaired individuals (BVIs) by pro- viding them with increased mobility and independence [1]. Unfortunately, limited availability is one of the known issues of guide animals due to the lengthy training process and their relatively short working lifespan. Consequently, only a small fraction, approximately two to three percent [2], of potential users are able to benefit from the assistance of guide dogs. Researchers have explored various types of assistive technologies for BVIs, including smart canes [3, 4, 5, 6, 7]. belts [8. 9], glasses [10], audio/haptic systems [11, 12], and mobile robots [13, 14, 15, 16, 17, 18). Recently, the development of affordable and capable quadrupedal robots has opened up opportunities for robot guide dogs [19, 20, 21, 22, 23], which possess notable qualities in terms of autonomy and mobility.
Developing robotic guiding systems for BVIs requires additional challenges beyond the common practices of the existing autonomous robot navigation systems [24, 25]. Unlike conventional for- mulations that simplify the problem as Point-goal Object-goal navigation [26], navigation of the human-robot dyad often involves more complex and subtle bidirectional interactions, including high-level command generation by the user and intelligent disobedience by the robot. Furthermore, ensuring BV user safety is critical during navigation, yet it is a challenging problem because the sensor configurations are robot-centric.
This paper discusses principles and practical solutions for transforming an autonomous robot into guide robot for individuals with visual impairments. The topics that we explore include formalizing the wayfinding task of a human and guide dog team, developing data-driven model of the human and the guide robot interaction, and improving the safety of the BVI user. First, we identify the mechanism of wayfinding in the literature [27, 28, 29] and formally define the guided navigation task using Markov Decision Processes (MDPs). Then we collect real human-robot interaction data of three BI and six sighted users and develop a concise model, Delayed Harness, to better predict the navigation behaviors of the team in simulation. Finally, we introduce an action shielding mech- anism [30] to improve the safety of the BVI user, which predicts the human position in the next step and filters out dangerous actions.
A mid-size quadrupedal robot, AlienGo [31], with a rigid hamness is chosen as the evaluation plat- form. Initially, we evaluate the accuracy of the developed Delayed Harness interaction model and the improved safety of the learned policy in the Habitat [25] simulator. Then we deploy the de- veloped system on real AlienGo robot to demonstrate the effective long-range navigation of the human-robot team. Our contributions include: (1) We develop formal definition of the wayfinding task of the human and guide dog team. (2) We collect human-robot interaction data, develop a con- cise interaction model, and open-source it to foster guide robot research in the robotics community. (3) We improve the safety of users by employing action shielding. (4) We demonstrate a complete system on hardware that can travel along trajectories for more than 100 meters.
2 Related Work
Assistive Guidance Systems for Visually Impaired People. Researchers have developed diverse assistive systems to enhance mobility for visually impaired people. Such assistive systems consist of essential elements, including sensors for perceiving the environment, computers for processing information, and various interfaces to instruct users. One common form is passive hand-held or wearable devices, such as canes [3, 5,5,6 6, 7], belts [8, 9, 11, 12), smart glasses [10], and smartphone applications [32, 33]. which generally give audio or haptic feedback to inform users. On the other hand, there exist guide robots [34, 13, 14, 19, 20, 22] that are designed to lead users actively toward destinations. Since the earliest guide robot MELDOG [34], such systems are often built on top of wheeled robots for mobility [13, 14, 15, 35, 16, 17, 36, 37, 38). Recently, legged robots are emerging in consideration of the ability travel as real guide dogs [19, 20, 21, 22, 23, 39, 40, 41].
Human Modeling in Guide Robots Developing guide robots necessitates modeling human-robot interaction because they communicate with users via physical interactions. For instance, Wang et al. [16] introduce rotating rigid rod model that assumes the user is holding the end of the rotating

---

rod at a fixed distance. On the other hand, some guide robots [19, 42, 22] communicate via flexible leash, which investigates a mathematical model for capturing slack and taut modes. Likewise, inter- actions are typically modeled based on the interface of robots that have been developed [43, 35]. The most common interface for a real guide dog is a fixed harness [44, 45]. this paper, we claim that simple offset between the human and the robot is insufficient to capture bidirectional interactions and propose novel mathematical model to improve the human-robot navigation experience.
Safety Safeey-aware -wware Reinforcement Learning for Navigation. Autonomous robot navigation has gained significant attention in robotics and has been approached through both planning- meth- ods [46, 47] and learning- bearning-based techniques [48, 49, 50]. Please refer to the survey paper [31, 52] for further detailed references. Numerous algorithms have been developed to enhance the safety of navigation, including action shielding [30, 53], model predictive methods [54], multi-agent strate- gies [55], and near future considerations [56, 57]. Our work is also inspired by these prior works while being more eustomized toward the human-robot team navigation task.
3 Formalizing the Wayfinding Task
This section identifies how a human interacts with a guide dog and formulates the wayfinding task for a guide robot using Markov Decision Processes (MDPs). Our problem definition aims to develop more natural and comfortable user interfaces of guide robots for BVI users, compared to typical Point-Goal or Objective- navigation problems [26].
Wayfinding with a Guide Dog. We first identify how guide dogs and human handlers communi- cate to work as team by reviewing relevant background materials [44, 27, 28, 29] and conducting interviews with actual guide dog users. Typically, the human user is the one who generates high- level directional cues, such as "go straight" or "turn left", based on his or her own knowledge, while the guide dog handles local path following and collision avoidance based on visual perception. Un- less the team routinely travels to the same destinations, the guide dog is not typically aware of the goal path. Then, the guide dog sees and leads the human handler based the given command. The guide dog needs to follow the virtual "travel line", a centerline of the hallway directed by the handler while adjusting a trajectory to ensure the safety of both the human and the dog itself. We re- fer to this type of human-dog navigation as Wayfinding. and it is our goal to replicate this interaction with a human-robot team.
In addition, a guide dog should understand user intention and compensate for any errors. We find two major types of human error: Orientation error and Timing error. For instance, orientation error can occur in environments such long hallway, where the user command is misaligned with the travel line and must be corrected. Timing error occurs when a human handler gives turn commands before or after the actual turning point. For instance, if a turning command is given early while traveling down a hallway, the robot is expected to find the next available turn or stop if no turn is available. Please refer to Figure 6 and the supplemental video for illustrative examples.
Problem Formulation. We define the wayfinding task for the robot as following high-level hu- man directional cues: going straight, turning left/right, or stopping while adjusting a detailed trajec- tory to avoid collisions. We use Markov Decision Processes (MDPs) to formalize this wayfinding task, which is defined as a tuple of the state space S, action space A. stochastic transition function p(st+1|se, a.), reward function r(st, a.), and the distribution of initial states ~ Po. The state and action spaces can vary depending on the choice of the robot. For instance, one common choice is to construct the state using onboard sensors, such as RGB-D or lidar, while defining the action space as a set of possible navigation commands to the robot. We use depth images, lidar readings, and a relative location from the start location as state and adopt discrete actions based on the recent autonomous navigation papers [49, 58, 50].
The reward function is critical to describe the desirable wayfinding behavior, which is defined as: Ft (de- d aA max ( 0 - 0,| b. 0) - - collide
(1) where de is the human's 's travel distance from the start point, 0 is the human's 's orientation, Ø is the target orientation (c.g., 0 for the straight command and 90 for the turn left command), is
3

---

scalar weight, b is the error margin above which the corresponding term is activated, cqullide is collision penalty, and À is a time penalty. The first term encourages the team to travel as far as possible while minimizing the human's unnecessary movements, such as stepping backward, the second term encourages matching the desired orientation (with some flexibility), and the third and fourth terms regularize unsafe and unnecessary behaviors, respectively.
Because our formulation rewards the team for traveling long distances, it encourages the navigation policy to compensate for any orientation or timing errors, particularly in narrow hallways. Also note that all the terms are defined in a human-centric manner in order to provide a smooth trajectory for the visually impaired user, not for the robot itself.
Modeling Human-Guide Robot Interactions
In order to develop successful guiding policy, it is important to simulate human-robot interactions accurately. Guide dogs wear a rigid harness, instead of a loose leash, allowing the human to feel both the guide dog's position and its orientation. We adopt such fixed harness for our human-robot teams. While there are established human-robot models, such as a rotating rod [16], a slack/taut leash [19], and geometric model [35], we require a new model specifically tailored to our guide dog robot with a harness. In this section, we propose a new interaction model, the Delayed Harness, and optimize the model parameters using real-world data.
Preliminary: Rigid Harness Model. The simplest model for describing rigid harness interactions is to use fixed offset between the human and the robot. In this model, the human maintains their relative position to the robot, facing the same forward direction. Formally, when the robot position and orientation (x/² y. 6,²) is given, the human state (z!". v." 0,") is computed by assuming a fixed distance d between the handler and the robot as follows:
x x d- cos 0t, vi v d sin 6,, 6 = 0.7
(2)
This model expects that the human and the robot rotate together, requiring extra space compared to leash, string, or rod connections.
Delayed Harness Model. A rigid harness model is not accurate because human cannot exactly follow the robot in reality. As result, a learned robot policy based on such model may lead to unnecessary movements or even collisions with a user.
To this end, we develop a more flexible model named De- layed Harness. This model is based on the observation that a robot's action causes the change of the relative location, but that human tends to gradually recover the default relative lo- cation (Figure 2). Let us define the robot and human states as x!" |= Hl ".IITT and x," = [x]² vf 0/' which gives the offset o = x -x"". Once the robot takes an action a.. it first changes its position: x = x.² + a,. Now, we have a temporary offset 6,41 = x x (gray dashed line in Figure 2). We gradually interpolate this temporary offsset to- ward the default offset, (green line), to obtain the corrected
R
x!|
oc+1
H
x
x++1
Ot
Figure 2: Delayed Harness Model.
offset in the next time: o++1 ab + (1 a)oō. Finally, we compute the next human position x xf + 0.+1- Note that we use simple arithmetic operators for illustration, but in reality, they should be handled as transformation operations.
Data Collection and Model Fitting. Our Delayed Harness model requires four parameters: the default offset = [Ar, Ay, ~ and the decaying parameter a. We determine these parameters by fitting them to the collected trajectories. We recruit three BVI users and six sighted people, and robot guide dog over five trajectories described in the work of Nanavati et al. [35]. The sighted individuals each wore a eye mask while walking with the robot. Please refer to the supplemental material for more details of the data collection process. The collected data is provided as supplemental material (interaction-data. zip).
ask them to walk with a

---

Improving Safety via Action Shielding
We further introduce an shiiod-- approach [30] to improve safety during human-robot navi- gation. Specifically, our focus on enhancing the safety of the BVI user based on data from sensors arranged in a robot-centric layout. Inspired by guide dog user who also uses a cane, our action shielding predicts the changes in the human and robot positions according to the possible actions and filters out unsafe actions. Particularly. we designed the shielding mechanism based on lidar, which has relatively better accuracy (#55 cm) than depth camera (5 % up to 15 m).
The implementation of such action shielding requires two subroutines: (1) predicting possible colli- sions in the next time step, and (2) suppressing potentially risky actions.
Computation of Shielding Zone. To iden- tify the shielding zone, we compute the occu- pied areas of both agents in current and next time steps, which are approximated as cir- cular shapes. While we have four collision primitives, the region between the occupied eas of the human and robot agents should be empty because the two agents are physically connected through harness. In addition, the zone between the current and the next time steps should also be collision-free to account for continuous movements. Therefore, we compute convex hull of all the collision shapes from both agents at two timesteps compute the shield- ing (red line in Figure 3). Once computed, sample rays from the origin to determine the distance thresholds for lidar. Refer to the Appendix 8.2 for more details.
200'
90"
210
225
Figure 3: Examples of Shielding Zones.
Suppressing Unsafe Actions. When shielding is activated, adjusting the action probability distribution by suppression factor B. A suppression factor of zero means assigning zero probability to the unsafe actions to force the policy to select safe actions only. is also possible to employ a small positive value to encourage exploration during training.
suppress unsafe actions by
The action shielding technique can be utilized during both training and testing. During training, we anticipate the agent will learn safer policy, especially when sensing is not perfect due to sensor noises or blind spots. Alternatively, to improve its safety at the evaluation stage. We examine how the learned policy performs under different action suppression factors in ideal and realistic sensing conditions in Section 6.3.
combine action shielding with any pre-trained policy
Experiments
We conducted experiments to address the following questions: (1) Is the proposed delayed harness model more accurate than other models? (2) How does the interaction model impact performance? (3) Does action shielding effectively decrease collisions in noisy environments? (4) When combined, can the proposed robot guide dog navigate to the destination with a human user?
6.1 Training Details
Our policies were trained in the Habitat simulator [25] with the Matterport3D dataset [59] using DD- PPO [49]. The training typically took 10 million steps until convergence, which roughly corresponds to two to three days. The observation space consists of all the sensor information, including depth images, 2D lidar reading, and the relative location from the starting position, followed by a one- hot encoded high-level directional cue provided by the user. For the low-level robot action space, we choose ten discrete actions (stop, forward, turn 10º left/right, sidestep left/right, diagonal steps toward the front left/right] inspired by the recent navigation literature [60]. We particularly included side and diagonal steps because they are effective for guiding the user to escape confined regions.
5

---

Table 1: Comparison of Different Interaction Models
Model
Fixed-unopt-ind
Fixed-opt-ind
RR-opt-ind
DH-opt-ind (ours)
DH-opt-all
RMSE
136.2
102.8
431.4
86.3
123.1
BVI
Robet
Human (subject)
positon (mm)
Human - Delayed (sutject)
Drea Poston
-600
-700
-1000
-75
-800
-1500
-850
-2500
-2000
-1500
-1000
-500
500
1000
Base Position
positiio (mm)
Figure 4: (Left) Different trajectories of interaction models optimized using the collected motion capture data on each individual subject. (Right) Each subject's base positions derived from the optimized results of the delayed harness model.
6.2 Comparison of Interaction Models
We first compared our delayed harness model against two baselines: a fixed harness and rotating rod [16]. By default, the parameters are optimized for each individual subject, which gives three models: Fixed-opt-ind, RR-opt-ind (rotating rod), and DH-opt-ind (delayed harness). In addition, included the unoptimized setting, Fixed-unopt-ind, which uses the starting parameter to estimate the default offset ō and assumes zero decay a 0. We also investigated the DH-opt-dr model, which optimizes single set parameters for all the subjects. For all the optimized models, we fitted the parameters over ten trajectories and validated them over five trajectories. The results indicated that the DH-opt-ind model exhibited the highest accuracy, followed by Fixed-opt-ind, Dil-opt-all, Fixed- unopt-ind, and RR-opt-ind (Table 1). Figure 4 left presents the example of the different trajectories between Fixed Harness and Delayed Harness models. For the same time step. Delayed Harness (connected via green line) is closer to the actual human location (orange line) compared to Fixed Harness (grey line) by capturing the delayed response of the human user. is also worth mentioning that the low accuracy of a rotating rod model was because its interface was different than a hamees: therefore, we did not further investigate more complex rod- bod-baeed models, such as the geometric model proposed by Nanavati al. [35].
The accuracy difference between DHH-op--ind and DH-opt-all highlights the importance of mod- els customized for users. We observed that subjects have different default offsets (Fig. 4 right) or responsiveness to direction changes. Therefore, we decided to train policies for each individual, assuming that it is possible to collect user-specific data at onboarding. In the future, it will be inter- esting to investigate preference optimization techniques [61] to improve robot guide dog behaviors.
Indeed, selecting the appropriate interaction model is critical to achieving the best human-robot team navigation. Table 2 presents the performance of the learned policies using var- ious interaction models at training and testing times. Both policies with the fixed or delayed harness models showed their best performance when evaluated with the correspond- ing model. However, the model is altered, the performance drops, approximately 25 % lower than the original perfor- mance. Therefore, we suggest to train policy with more accurate interaction model, such as Delayed Harness.
Table Rewards with Different Train- ing and Testing Interaction Models.
Test
Fixed
Delay
Fixed Delay
5.96
4.27
Train
4.00
5.04

---

Sensor Config
Supp. Factor during Train
Supp. Factor during Test
Collision-Free Ep. Ratio 1
Avg. Collisions
Ep.
Ideal
0.0
0.0
(diverges)
0.04
0.5
0.0
0.84
1.46
0.9
0.0
0,88
1.80
1.0
0.0
0.92
1.40
(no shielding)
1.0
1.0
0.32
8.20
Noisy
0.0
0.0
(diverges)
0.68
4.12
1.48
0.9
0.0
0.64
2.52
1.0
0.0
0.72
3.16
(no shielding)
1.0
1.0
0.52
3.96
Table 3: Performance of Action Shielding in Ideal and Noisy Environments.
6.3 Effectiveness of Action Shielding
The main objective of our action shielding mechanism is improve the safety of the human-robot team in unseen environments with noisy sensor readings. To this end, we compared a policy with action shielding against vamilla RL policy in both ideal and noisy simulated environments. Addi- tionally, we varied an action suppression factor 3 during training as well, which has a large effect on the performance. For noisy environments, we employed Gaussian noise for the lidar and Red- wood [62] noise with an intensity of 1.0 for the depth camera.
In Table 3, present the collision-free free episode ratio and the average number of collisions per episode metrics for evaluating the safety of the policies. Overall, it is evident that activating action shielding (B < 1.0) during evaluation effectively reduces the occurrence of collisions. However, the desirable training configurations differed in ideal and noisy environments. In an ideal setting, the performance not very sensitive to the choice of the suppression factor. It even very beneficial to turn on action shielding only at the testing stage, which shows great performance margin against no shielding (collision-fre episode ratios: 0.92 0.32). In noisy environment, training policy with mild action shielding (3 0.5) was more effective for adapting navigation strategy cope with action shielding. In any configuration, zero action supression during training results in divergence of learming, which is likely due to insufficient exploration.
Action shielding was effective even real environments with noisy sensor readings. We presented two distinct scenarios where the vanilla policy (without action shielding) failed generalize novel situations, such as encountering curtains large boxes that were unseen during training. Conversely, the policy with action shielding successfully navigated around these obstacles by predicting the user and robot trajectories, For more details,
please refer to the supplemental video.
6.4 Real-world Experiments
We deployed the proposed robot guide dog system on the hard- ware of AlienGo [31] by Unitree (Figure 5). This quadruped robot was equipped with Zed depth camera, a Slamtec RPLIDAR li- dar, and an Intel 1265 tracking camera. The robot leads the user via a harness while taking verbal commands directional cues. The policy was trained and deployed with action shielding with action supression of 0.5 and 0.O, respectively, based on Table 3.
iaadd
Guts
Dept
Figure Hardware Platform
General Navigation. We assigned two BVI and three sighted users wearing an eye mask the task of completing five indoor routes with four directional cues: forward, left, right, and stop. Route (146 m) is a curved path expected
traversed by single forward cue. Routes (130 m), (121 m). D (120 m), and E (80 m) required the user issue multiple own localization. Refer to Appendix 8.3 and the supplemental video for the details of the routes.
based on their
We also set additional obstacles, such wet floor signs and opened doors, top of the existing
7

---

chairs and trashcans. In our experiments, the user successfully accomplished all the tasks without collisions. In Route A, a single command of forward was sufficient to complete all the tasks, even in a very long curved corridor. In Routes B and C, the user sometimes issued turning commands early or late, but the robot was able to compensate for the timing errors. Routes C, D, and E, a robot intelligently adjusted the trajectory along the hallway without additional cues. Where there were obstacles ahead, the robot guide dog led the user to avoid them while staying in the travel line. Occasionally, the human got close to obstacles or walls, and the robot prevented human collisions by actively using action shielding, which encourages sidestepping as means of avoidance. After the navigation experiments, we conducted interviews with the BVI users (Appendix 8.3). The users were positive about the system's 's controllability and found their experience improved with familiarity during the second trial.
Orientation and Timing Errors. As ex- plained in Section 3, the guide robot must effec- tively handle cues that are provided with some timing or orientation errors caused by human users. We showcase the robustness of the de- veloped robot guide dog by displaying the tra- jectories involving human errors. Figure 6(a) displays orientation errors on forward cue, where the robot corrected the initial orientation eтror and followed the direction of the hallway to travel as long as possible. Figure 6(b) illus- trates the turning with timing errors, where the robot took diagonal stepping 120 turning to correct early late issued commands, respectively.
Cue
Figure 6: Trajectories with Two Type of Human Errors: Orientation Error (a) and Timing Error (b).
7 Conclusion and Limitation
This paper presents three topics and potential solutions for transforming standard quadrupedal robot into robot guide dog assist blind or visually impaired individuals. First, we study the navigation patterns of human-guide dog pairs and establish formal wayfinding task using Markov Decision Processes (MDPs). Then we propose a concise interaction model called the Delayed Har- ness to effectively represent the interaction between a human and robot guide dog, which leads to more accurate behavior prediction and better navigation performance. The parameters of this interaction model are optimized with respect to the real interaction data of three visually impaired and six sighted subjects, which is provided as supplemental material this manuscript. Finally, introduce an action shielding mechanism to improve the safety of the human user, inspired by guide dog user who also uses cane along with guide dog. We demonstrate that the integrated robot guide dog navigate with users over multiple 100+m long trajectories. We hope that the topics and techniques discussed in this study will inspire further research in the development of guide robots for blind and visually impaired individuals.
Limitation. The proposed solutions in the paper have room for improvement, and we plan to investigate the following research directions. Firstly, while we have explored only limited set of basic verbal cues, real guide dog users employ wider range of commands, such as "find an elevator" or "follow the person". combined with non-verbal cues. It will be beneficial to formalize a more expanded set of human-guide dog interactions. Additionally, although our Delayed Harness model effectively captures typical navigation behaviors, it struggles with -iitt--diittiittionn situations like users pausing to chat with someone. In the future, we will expand the provided dataset to encompass such diverse scenarios. The developed action shielding mechanism is effective but not perfect. For instance, it can fail even in simulation due blind spots of the lidar sensor. To resolve this issue, we will explore alternative sensors and safety mechanisms to improve the safety of the human and the robot. Our proposed system requires an hour-long data collection session for personalized navigation. To mitigate this, we will explore online adaptation algorithms to reduce training time. Finally, we plan to evaluate the system on a larger population of blind or visually impaired users to gather feedback for further improvements.
8

---

Acknowledgments
This research was supported by Google Research Collabs and Google Cloud. JK was supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE-2039655. Any opinions, findings, and conclusions recommendations expressed in this material those of the author(s) and do not necessarily reflect the views of the National Science Foundation, any sponsor.
References
[1] L. Whitmarsh. The benefits of guide dog ownership. Visual impairment research, 7(1):27-42, 2005.
[2] Guiding Eyes for the Blind. http:://wwww..dddiggeeeeee.eer
2022.
[3] J. Borenstein and I. Ulrich. The guidecane-a computerized travel aid for the active guidance of blind pedestrians. In Proceedings of International Conference Robotics and Aulomation, volume 2, pages 1283-1288. IEEE, 1997.
[4] S.-J. Kang, Y. Ho, and I. H. Moon. Development of an intelligent guide-stick for the blind. In Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164 164), volume 4, pages 3208-3213. IEEE, 2001.
[5] K. Yatani, N. Banovic, and K. Truong, Spacesense: representing geographical information to visually impaired people using spatial tactile feedback. In Proceedings of the SIGCHI Confer- ence on Human Factors in Computing Systems, pages 415-424, 2012.
[6] S. Kher Chaitrali, A. Dabhade Yogita, K. Kadam Snehal, D. Dhamdhere Swati, and V. Desh- pande Aarti. An intelligent walking stick for the blind. International Journal of Engineering Research and General Science, 3(1):1057-1063 2015.
[7] R. Velázquez, E. Pissaloux, P. Rodrigo, M. Carrasco, N. I. Giannoccaro, and A. Lay-Ekuakille. An outdoor navigation system for blind pedestrians using gps and tactile-foot feedback. Ap- plied Sciences, 8(4):578, 2018.
[8] S. Shoval, J. Borenstein, and Y. Koren. The navbelt-a computerized travel aid for the blind based on mobile robotics technology. IEEE Transactions on Biomedical Engineering, 45(11): 1376-1386, 1998.
[9] H.-C Wang, R. K. Katzschmann, S. Teng, B. Araki, L. Giarré, and D. Rus. Enabling indepen- dent navigation for visually impaired people through a wearable vision-based feedback system. 2017 IEEE international conference on robotics and automation (ICRA), pages 6533-6540. IEEE, 2017.
[10] W.-. Chang, J.PP. Su, L.-B. Chen, M.-C. Chen, C.-H. Hsu, C.-H. Yang, C.-Y. Sie, and C.-H. Chuang. An ai edge computing based wearable assistive device for visually impaired people zebra-crossing walking. In 2020 IEEE International Conference Consumer Electronics (ICCE), pages 1-2, 2020. doi:10.1 (D9/CCCEA6568.2220.904333
[11] Wilson, Walker, Lindsay, C. Cambias, and F. Dellaert. Swan: System for wearable audio navigation. In 2007 11th IEEE international symposium wearable computers, pages 91 91-98. IEEE, 2007.
[12] B. N. Walker and Wilson. Swan 2.0: Research and development on new system for wearable audio navigation. In Proceedings of the WirelessRERC State of the Technology Forum 2021 Atlanta, GA, USA (virtual conference) (23-24 March), 2021.

---

[13] Y. Wei, X. Kou, and M. C. Lee. Smart rope and vision based guide-dog robot system for the visually impaired self-walking in urban system. In 2013 IEEFAASEE International Confer- ence on Advanced Intelligent Mechatronics, pages 698-703, 2013. doi:10.1 109/AIM..013. 6584174.
[14] T.-K. Chuang, N.-C. Lin, J.-S. Chen, C.-H. Hung. Y.-W. Huang, C. Teng, H. Huang, L.-F. Yu, L. Giarré, and H.-C. Wang. Deep trail-following robotic guide dog in pedestrian environments for people who are blind and visually inpaired-leariinn from virtual and real worlds. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 5849-5855. IEEE, 2018.
[15] D. R. Bruno, M. H. De Assis, and F. S. Osorio. Development of a mobile robot: Robotic guide dog for aid of visual disabilities in urban environments. Proceedings 2019 Latin American Robotics Symposium, 2019 Brazilian Symposium on Robotics and 2019 Work- shop on Robotics Education, LARS/SBR/WRE 2019. pages 104-108, 2019. doi:10.1109; LARS-SBR 20W.W44444440000000000
[16] L. Wang. J. Zhao, and L. Zhang. Navdog: robotic navigation guide dog via model predictive control and human-robot modeling. In Proceedings of the 36th Annual ACM Symposium on Applied Computing, pages 815-818, 2021.
[17] V. Ranganeni, M. Sinclair, E. Ofek, A. Miller, J. Campbell, A. Kolobov, and E. Cutrell. Ex- ploring levels of control for a navigation assistant for blind travelers. Proceedings of the International Conference on Human-Robon Interaction, HRI '23, page 4-12, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 978 181443399647. 1 162.356663. URL htttps://doot ortp:/..rr/1...0 1145/356611611 3578630.
2023 ACM/IEEE
[18] S. Azenkot, C. Feng, and M. Cakmak. Enabling building service robots guide blind people participatory design approach. In 2016 / Ith ACM/IEEF International Conference on Human- Robot Interaction (HRI), pages 3-10. IEEE, 2016.
[19] A. Xiao, W. Tong, L.. Yang, J. Zeng, Z. Li, and K. Sreenath. Robotic guide dog: Leading a human with leash-guided hybrid physical interaction. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 1 11470-11477 IEEE, 2021.
[20] A. Sivacoumare, S. M. S, Satheesh, T. Athul, M. V. and T. Vinopraba. Ai quadruped robot assistant for the visually impaired. In IECON 2021 47th Annual Conference of the IEEE Industrial Electronics Sociery, pages 1-5, 2021. 111:101111//EEON481155 1522211.589950
[21] K. Mehrizi. Quadrupedal Robotic Guide Dog with Vocal Robot Interaction. pages 2-4, 2021. URL htap://arxiv. wwwpsiiivvr/////aa/ea otggabs//111..0.
[22] Y. Chen, Z. Xu, Z. Jian, G. Tang, Y. Yangli. A. Xiao, X. Wang, and B. Liang. Quadruped Guidance Robot for the Visually Impaired: A Comfort-Basse Approach. 2022. URL http:
//axxiv.or//abs 22033. 03927.
[23] K. A. Hamed, V. R. Kamidi, W. Ma, A. Leonessa, and D. Ames. Hierarchical and Safe Motion Control for Cooperative Locomotion of Robotic Guide Dogs and Humans: A Hybrid Systems Approach. IEEE Roboties and Automation Letters, 5(1):56-65 63, 2020. ISSN 23773766. doi: 109/LRA.2019.299977
[24] B. Shen, F. Xia, C. Li, R. Martin-arttn L. Fan, G. Wang, C. Pérez-D' Arpino, S. Buch, S. Srivastava, L. Tchapmi. et al. igibson 1.0: A simulation environment for interactive tasks in large realistic scenes. In 202/ JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7520-7527. IEEE, 2021.
[25] M. Savva, Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, Liu, V. Koltun, Malik, D. Parikh, and D. Batra. Habitat: A platform for embodied AI research. Proceed- ings of the IEEE International Conference Computer Vision, ISSN 15505499. 0tt110..109//CCC22199900099
111999338889960.33 2019.
10

---

[26] P. Anderson, A. Chang, D. S. Chaplot, A. Dosovitskiy, S. Gupta, V. Koltun, J. Kosecka, J. Ma-
lik, R. Mottaghi, M. Savva, and A. R. Zamir. On Evaluation of Embodied Navigation Agents.
7 2018. URL http://arxiv.orr/abbel aarggabs1180.007755 06757.
[27] Guide dogs for the blind puppy raising manual-training phase descriptions.
http:/ppiiieeoogs..cooaplpaaaaaa Traiinng-haae-Desciiipi.o.. pdf, 2022.
https://www.guiideee
[28] GDB class lecture: Guidework. https://s3-us-weett west- amazonaws.com/gdae Guidework txt bxt77tt.iee--0160022255555 2022.
[29] GDB class lecture: Why does your guide dog work? https://ww..u...eeogs
http:ppporrerreuuurre///immmmeer htt.reaaarrreeeeaaaaeeecarrrrrra why-does-yourz gunete-ggiiiieee guide--oo--or---eeeee
2022.
[30] M. Alshickh, R. Bloem, R. Ehlers, B. Könighofer, S. Nickum, and U. Topcu. Safe reinforce- ment learning via shielding. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pages 2669-2678, 2018.
[31] Aliengo by unitree roboties. http://www..nnnttrecccm//mdddeec 2020.
[32] A. Meliones, C. Filios, and J. Llorente. Reliable ultrasonic obstacle recognition for outdoor blind navigation. Technologies, 10(3):54, 2022.
[33] Theodorou, K. Tsiligkos, A. Meliones, and C. Filios. An extended usability and ux evalua- tion of mobile application for the navigation of individuals with blindness and visual impair- ments outdoors-an an evaluation framework based on training. Sensors, 22(12))4538, 2022.
[34] S. Tachi, K. Tanie, K. Komoriya, and M. Abe. Electrocutaneous communication in guide dog robot (meldog). IEEE transactions biomedical engineering, (7):41-465) 1985.
[35] A. Nanavati, X. Tan, Connolly, and A. Steinfeld. Follow the robot: Modeling coupled human-robot dyads during navigation. In 2019 IEEE/RSJ International Conference on Intel- ligent Robots and Systems (IROS), pages 3836-3843, 2019, doi:10. 10911009/0044997220999999 8967656.
[36] A. Kulkarni, A. Wang, L. Urbina, A. Steinfeld, and B. Dias. Robotic assistance in indoor navigation for people who are blind. In 2016 11th ACM/EEE International Conference on Himaan-Rbbe Interaction (HRI), pages 461-462. IEEE, 2016.
[37] M. Kuribayashi, T. Ishihara, D. Sato, Vongkulbhisal, K. Ram, S. Kayukawa, H. Takagi, S. Morishima, and C. Asakawa. Pathfinder: Designing map-less navigation system for blind people in unfamiliar buildings. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-16, 2023.
[38] S. Kayukawa, D. Sato, M. Murata, T. Ishihara, A. Kosugi, H. Takagi, S. Morishima, and C. Asakawa. How users, facility managers, and bystanders perceive and accept a navigation robot for visually impaired people in public buildings. 2022 31st IEEE International Con- ference on Robot and Human Interactive Communication (RO-MAN), pages 546-553. IEEE, 2022.
[39] H. Hwang, T. Xia. I. Keita, K. Suzuki, J. Biswas, S. I. Lee, and D. Kim. System configuration and navigation of guide dog robot: Toward animal guide dog-level guiding work. arXiv preprint arXiv:2210. 13366, 2022.
[40] T. Kim, W. Yu, J. Tan, G. Turk, and S. Ha. How train your guide dog: Wayfinding and safe navigation with human-robor modeling. In Companion of the 2023 ACM/IEEE International Conference Human Robot Interaction, pages 221-225, 2023.

---

[41] D. DeFazio, E. Hirota, and S. Zhang, Seeing-eye quadruped navigation with force responsive locomotion control. arXiv preprint arXiv:2309.043370, 2023.
[42] H. Tan, C. Chen, X. Luo, J. Zhang, C. Scibold, K. Yang. and R. Sticfelhagen. Flying guide dog: Walkable path discovery for the visually impaired utilizing drones and transformer-be bbsse semantic segmentation. In 2021 IEEE International Conference Robotics and Biomimetics (ROBIO). IEEE, 2021.
[43] J. Guerreiro, D. Sato, S. Asakawa, H. Dong, K. M. Kitani, and C. Asakawa, Cabot: Designing and evaluating an autonomous navigation robot for blind people. ASSETS 2019 - 21st Interna- tional ACM SIGACCESS Conference on Computers and Accessibility, (October):68 82, 2019. doi:10.1 153.1111433333511.3333 .3333771.
[44] B. Knol, C. Roozendaal, L. Van den Bogaard, and J. Bouw. The suitability of dogs as guide dogs for the blind: criteria and testing procedures. Veterinary Quarterly, 10(3):198-20) 1988.
[45] M. J. Tellefson. A perspective on teaching early hamness travel to young blind children using children's 's visual companion dogs. Journal of Visual Impairment & Blindness, 10665):306-312, 2012.
[46] G. E. Jan, K. Y. Chang, and I. Parberry. Optimal path planning for mobile robot navigation. IEEE/ASME Transactions on mechatronics, 13(4):451 2008.
[47] J. Bruce and M. Veloso. Real-time randomized path planning for robot navigation. In /EEE/RS/ international conference intelligent robots and systems, volume 3, pages 2383-2388. IEEE, 2002.
[48] C. Chen, Y. Liu, S. Kreiss, and A. Alahi. Crowd-robot interaction: Crowd-aware robot navi- gation with attention-based deep reinforcement learning. In 2019 international conference on robotics and automation (ICRA). pages 2011-6022... IEEE, 2019.
[49] E. Wijmans, A. Kadian, A. Morcos, S. Lee, I. Essa, D. Parikh, M. Savva, and D. Batra. Dd- ppo: Learning near- near-perfect pointgoal navigators from 2.5 billion frames. In International Conference on Learning Representations, 2020.
[50] M. Sorokin, J. Tan, C. K. Liu, and S. Ha. Learning to navigate sidewalks in outdoor environ- ments. IEEE Robotics and Automation Letters, 7(2):3906-3913, 2022.
[51] K. Zhu and T. Zhang. Deep reinforcement learning based mobile robot navigation: A review. Tsinghua Science and Technology, 26(5):674 691, 2021.
[52] X. Xiao, B. Liu, G. Warnell, and P. Stone. Motion planning and control for mobile robot navigation using machine learning: survey, Autonomous Robots, 46(5):569-55) 597, 2022.
[53] N. Jansen, B. Könighofer, Junges, A. Serban, and R. Bloem. Safe reinforcement learning using probabilistic shields. 2020.
[54] O. Bastani and S. Li. Safe reinforcement learning via statistical model predictive shielding. 2021. doi: 1i..1...6607/s..221..ooo
[55] I. ElSayed-Aly. S. Bharadwaj. C. Amato, R. Ehlers, U. Topeu, and L. Feng. Safe multi-agent reinforcement learning via shielding. Proceedings of the International Join! Conference on Autonomous Agents and Multiagent Systems, AAMAS, 1:483-491, 2021. ISSN 15582914.
[56] G. Thomas, Luo, and T. Ma. Safe Reinforcement Learning by Imagining the Near Future. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 13859-13869, Curran Associates, Inc., 2021.
12

---

[57] H.-L. Hsu, Q. Huang, and S. Ha. Improving safety in deep reinforcement learning using unsupervised action planning. In 2022 International Conference on Robotics and Automation (ICRA), pages 5567-5573. IEEE, 2022.
[58] N. Yokoyama, Q. Luo, D. Batra, and S. Ha. Benchmarking augmentation methods for learning robust navigation agents: the winning entry of the 2021 igibson challenge. arXiv preprint ar0i 10493, 2021.
[59] A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, Song, A. Zeng, and Y. Zhang. Matterport3d: Learning from rgb-d data in indoor environments. International Conference on 3D Vision (3DV), 2017.
[60] N. Yokoyama, S. Ha, and Batra. Success weighted by completion time: dynamics-aware evaluation criteria for embodied navigation. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1569. IEEE, 2021.
[61] K. A. Ingraham, C. D. Remy, and J. Rouse. The role user preference the customized control of robotic exoskeletons. Science robotics, 7(64))ea 2022.
[62] S. Choi, Q.-Y. Y. Zhou, and V. Koltun. Robust reconstruction of indoor scenes, In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5556-5565, 2015. 1oi:10.1109/CVPR 2011..99........77999955
13

---

Supplementary Material
8.1 Human-Robot Modeling Data Collection
Figure 7: Data collection of three blind or visually impaired (BVI) subjects.
We collected human and robot interaction data using a Vicon motion capture system with Pulsar active marker clusters (Figure 7). The study is reviewed and approved by the institutional review board (IRB). We recruited nine participants consisting of three blind or visually impaired (BVI) and six sighted subjects. For the data collection, the participants are asked to wear back strap place a motion capture marker on their lower back. Then, they follow the robot, AlienGo, by using their left hand to hold onto a harness handle that is attached to the robot. For white cane users, we asked them to hold their cane in their right hand for safety purposes, but they did not actively use the cane during data collection.
Table 4: Description of Five Robot Trajectories for Interaction Data Collection
Trajectory
Description
2.5 m forward. 90-degree in-place left turn
1.2 m forward, 90-degree left turn
0.75 forward, 45 45-degree gradual left turn, 135-degree in tlaplace right turn
0.5 m forward, 90-degree right turn, 0.5 m forward, 90-degree left turn, 0.5 forward
0.6 m forward, 90-degree in-place right turn, 0.6 m forward,
180-degree left u-turn, 0.6 m forward, 90-degree right turn, 0.6 m forward
Robot Trajectories. The five trajectories are based on the work of Nanavati et al. [35] with modifications to in the motion capture space.
The robot is scripted to follow pre-defined trajectories as described in Table 4.
Experimental Protocol. The data collection process is divided into three parts. Each part is de- signed to examine different aspects of the interaction behavior. Part 1: The subjects followed the robot without vision, but relying purely on the physical interaction without any visual cues. We repeated each trajectory three times in random order. To prevent the subject from predicting the trajectory, they were told that there were 15 trajectories. Part 2: The subjects followed the robot without vision, but with prior information on the robot's expected movement. We provided verbal descriptions of the expected cue just before the robot executed each cue. Part 3: The sighted sub- jects followed the robot with their full vision. This is extra data collected only from sighted subjects to respond to the robot's s movement with additional visual perception.
14

---

8.2 Pseudocode of Action Shielding
This section provides a more detailed algorithm of our action shielding mechanism. We compute the convex hull of the collision primitives first, compute the lidar thresholds from this hull, and then check whether the action is safe not. Please review this material with Section 5.
Algorithm Convex-huue -ull Action Shielding.
Input current position and orientation of robot and human x/', x list of possible actions A, vector of action probability action suppression scale 3, lidar reading Output modified action probability vector p procedure ACTIONSHIFLLDINGCII Tú A, s,
2: for i f- to len(A) do
3:
A [
4:
P P
5:
X x++ < EsttimattNNexxxff xf a)
b Estimate Via Interaction Model
6:
< ConvexHullxx,, x x x x(+1)
7:
I + ComputeLidarThnn Idaehessodds (S)
8:
if any of reading less than the threshold then
9:
p
end end for
11:
p noommaiize(p)
13:
return the modified action probability p
end procedure
8.3 Real World Navigation Experiments
User Study Overall, the users were positive about our suggested system: "I think the technology is going to be great. Once you tell it to go straight, it's 's in straight path, and that's 's very nice. I like the way it was pulling- it was very consistent, a nice solid pull forward." The users enjoyed developed system's controllability, allowing the users to navigate the hallway effectively: "I like being able to do the commands. So that was good, a little more control over the direction and the way the dog (robot) was walking." " The users also felt much more comfortable with the second trial than the first one because they became familiar with the system: "I thought the second run like 50 % better than the first one." However, one participant suggested reducing the noise from the gait: "(Q: What did you like least about this experience?) Well, the noise from the gait."
Teamed Navigation Routes We include here the map information for our real-world experiments (Figure and Figure 9).
15

---

Route A
Route B
Route
Figure 8: Illustration of three routes in our experiments.
0D0
EDC
Latal
NDvA6O
COG Bill
600
10461
OISA
KONA
TONOW
528 REW
MERAACI
NONADO
HOMABBAR
18h
IAEE
Route D
Route
0310
170M
Figure 9: Illustration two routes in our experiments with the visually impaired subjects.
16


Reviews:
[{"review_id": "kgKhhx_oRI", "reviewer": "Reviewer_SGRs", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents an RL-based method for controlling the motion of a quadriped robot that is intended for use as a guide for blind and low-vision (BLV) people.  The main contributions are a model of the relationship between the user and robot that accounts for delays in human following and an action shielding approach intended to prevent unsafe actions.", "summary_of_recommendation": "Overall, there are potentially interesting elements to the method, but given that the main innovation is that the system is intended to be useful as a guide for blind and low vision users, there is not sufficient evidence that the system is fit for this proposed purpose.  I encourage the authors to conduct a proper validation with at least 5-10 blind and low-vision participants and appropriate baselines and resubmit to a future venue.\n\nPost-rebuttal comments: I think the authors have addressed some of my key ethical concerns (although I am a bit dubious about the use case where a sighted person is in fog or smoke in an emergency situation; in this case, the ecological validity problems are with the scenario rather than the users), so I raise my score to a weak reject rather than a strong reject.  As I said before, I encourage the authors to conduct a more strongly valid and appropriately-powered evaluation and resubmit to a future venue.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "How would it change the work not to use the disability simulation? Which results can be justified without recourse to the assumption that a blindfolded sighted person is a good model for a blind person? Which results are called into question? For this reviewer, the authors would need to craft a compelling argument that the results stand if a blindfolded sighted person is a poor model of a blind person.  Please also address the handful of questions above.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n- The authors conduct relatively comprehensive testing of the system in simulation\n- Some blind/low-vision participants were included in the data collection (see below for remaining concerns).\n- The proposed approach is a potentially promising starting point, although more research is needed to confirm this.\n\n\nWeaknesses:\nOverall, the validation does not provide strong support for the efficacy of the proposed method.  More appropriate testing is needed to confirm that this approach is a step in the right direction.\n\nMost importantly, technologies for BLV users cannot be validated by blindfolding sighted users (see citations below), both for ethical reasons and because there is no reason to think that a blindfolded sighted person moves through the world in the same way as someone who is blind or low-vision.  This is easy to see in Figure 4 in the authors' own work, where there is a clear separation between the BLV participants' behavior and the sighted participants' behavior with the robot.  Disability simulations are to be avoided, but if they are used, it is imperative that the authors are extremely careful about their claims, which is not the case here -- for example, it is not true that the method \"improve[s] the safety of the BVI user\", since the system isn't actually tested with blind participants.  The authors must have some connection to the BLV community, since their initial data collection included BLV participants; validation should also be conducted with this community.\n\n- I would like to see more intuition for whether the simulation results are meaningful.  How can we interpret the error in the models? Is a difference of <20 in the RMSE meaningful to a user?\n\n- For most values of the suppression factor the average collisions per episode is greater than 1 even with perfect sensing, and 2-4 in many cases with noisy sensors.  This seems high for the context of guiding a blind person. Even 92% collision-free seems low for an action shielding method that is intended to prevent unsafe actions entirely.\n\n- Testing is conducted in an indoor hallway without obstacles; it is unclear why a legged robot is needed in this context.  The paper would be improved by the inclusion of some testing with obstacles for which a legged robot would be useful (e.g., stairs or uneven surfaces).  It is unclear whether the proposed method would work in such a context.\n\n- The validation in 6.4 is conducted with only a single participant, with no information about the recruitment of that participant.  Was this participant a member of the study team and/or a robotics expert? Was the validation study approved by a university ethics board? \n\n- The following work should probably be cited:\nS. Azenkot, C. Feng and M. Cakmak, \"Enabling building service robots to guide blind people a participatory design approach,\" 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, New Zealand, 2016, pp. 3-10, doi: 10.1109/HRI.2016.7451727.\n\nCitations on disability simulation:\n\nFrench, S. (1992). Simulation exercises in disability awareness training: A critique.\nDisability, Handicap & Society, 7, 257–266 10.1080/02674649266780261.\ndoi:10.1080/02674649266780261\n\nGarreth W. Tigwell. 2021. Nuanced Perspectives Toward Disability Simulations from Digital Designers, Blind, Low Vision, and Color Blind People. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 378, 1–15. https://doi.org/10.1145/3411764.3445620\n\nLeo, J., & Goodwin, D. (2016). Simulating Others’ Realities: Insiders Reflect on Disability Simulations, Adapted Physical Activity Quarterly, 33(2), 156-175. Retrieved May 22, 2023, from https://doi.org/10.1123/APAQ.2015-0031\n\nNario-Redmond, M. R., Gospodinov, D., & Cobb, A. (2017). Crip for a day: The unintended negative consequences of disability simulations. Rehabilitation Psychology, 62(3), 324–333. https://doi.org/10.1037/rep0000127\n\nCynthia L. Bennett and Daniela K. Rosner. 2019. The Promise of Empathy: Design, Disability, and Knowing the \"Other\". In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, Paper 298, 1–13. https://doi.org/10.1145/3290605.3300528\n\nJonathan Lazar, Jinjuan Heidi Feng, Harry Hochheiser, \"Chapter 16 - Working with research participants with disabilities\", Editor(s): Jonathan Lazar, Jinjuan Heidi Feng, Harry Hochheiser, Research Methods in Human Computer Interaction (Second Edition), Morgan Kaufmann, 2017, Pages 493-522, ISBN 9780128053904", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "R24H7x7nuo7", "reviewer": "Reviewer_1S13", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper explores the application of a quadrupedal robot as a replacement for guide dogs. The authors formalize the wayfinding task (wherein a human interacts with a guide by giving discrete navigation commands) as an MDP.  To model the human-guide interaction the authors propose a ‘Delayed Harness’ model that accounts for the human’s inability to rigidly follow the guide’s harness without a delay. Additionally, the authors introduce an action-shielding model designed to improve safety during navigation. The model predicts the changes in the human and robot positions and attempts to filter out unsafe actions. \n\nTo evaluate their proposed approaches, the authors first conduct a series of experiments situated in Matterport3D environments using the Habitat simulator.  Finally, the proposed system is deployed on an AlienGo robot for evaluation in the real world.", "summary_of_recommendation": "I would have liked to see a much more rigorous evaluation that includes more BVI participants (as BVI users are central to the motivation of the paper). In lieu of that, I would like to at least see some discussion of this limitation and some argument for why blind-folded sighted user data is sufficient.  Otherwise, I am left questioning whether these methods scale to the types of navigation interactions you might encounter \"in the wild\" with real BVI users.\n\nHowever, I do think the problem is well motivated, the proposed formulation and approach are sensible, and the empirical results presented in the paper are promising. For these reasons, I think the paper is interesting and worth sharing with the community. So I am weakly recommending \"Accept\".", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "- Is the use of blind-folded sighted users common in training/evaluating systems intended for BVI users? Is there some evidence that shows that blind-folded sighted users behavior is a sufficient emulation of BVI users behavior? My assumption is that the behavior of BVI users vs blind-folded sighted users would be different.  I don't think it's necessarily a problem that you include samples from blind-folded sighted users, but I do question the over-representation of sighted users in your model fitting (and that your subsequent on-robot evaluation is done with a sighted user).  Given that the paper is motivated by BVI users, I think the paper would be much stronger with a more rigorous evaluation that includes more BVI participants.  At the very least I would like to see this limitation addressed with some discussion about your decision to use sighted blind-folded users (and ideally some evidence to show that is sufficient). \n- I would like to see more details about the 10 training trajectories and 5 evaluation trajectories (e.g. how were the trajectories chosen and how do they vary in length and difficulty? Do you have reason to believe your methods will generalize/scale beyond the 5 evaluation trajectories?)\n- You might consider being more explicit about what ‘accuracy’ is measuring in the interaction model comparison. From the model described in Section 4 and the plot in Figure 4, it seems like this is measuring accuracy at predicting the human state based on the robot position and orientation. But you might consider making that more clear.\n- The reported results of Action Shielding seem very promising. However, I would like to see more details about the experiments (similar to the last section — e.g. how many environments are being used for train/test and how do they vary? How are trajectories synthesized in simulation? How is the human state synthesized in simulation?)", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n- In general, I found the paper to be well written, easy to follow, and enjoyable to read\n- The problem setting is well motivated and clearly described\n- The proposed problem formulation and approach seem well designed\n- The empirical results presented in the paper are promising\n\nScope for improvement:\n- The assumption that it is possible to collect user-specific data at onboarding seems like a big area for improvement. I would like to see at least more discussion about the limitation and possible directions for a more generalizable/scalable approach.\n- There was a very limited number of participants used for model fitting (3 BVI users and 6 sighted users).  I think the paper would be stronger with a much larger and more diverse set of users.  I also question the over-representation of sighted users — as I would expect the behavior of BVI users vs sighted users (even blindfolded) to be different.\n- I would like to see more details about the 10 training trajectories and 5 evaluation trajectories (e.g. how were the trajectories chosen and how do they vary in length and difficulty? Do you have reason to believe your methods will generalize/scale beyond the 5 evaluation trajectories?)\n- The reported results of Action Shielding seem very promising. However, I would like to see more details about the experiments (similar to the last section — e.g. how many environments are being used for train/test and how do they vary? How are trajectories synthesized in simulation? How is the human state synthesized in simulation?)\n- The real-world experiments seem very limited. If I’m understanding correctly, only a single sighted user participated and the system was never evaluated with any BVI users. This seems like a big area of improvement to me. As BVI users are used to motivate the paper, I think the paper would be much stronger with a more rigorous evaluation that includes more BVI participants. ", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "90vohfUtCr", "reviewer": "Reviewer_dd3b", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper works on getting robot dogs to be better guide dogs for the blind and visually impaired (BVI).\nThe authors introduce a new formalism for modelling guide-dog-following, they collect their own dataset an verify that this formulation indeed models the data. They also introduce a new action filtering system dubbed \"action shielding\", that reduces the number of collisions that the BVI-dog pair encounters and they evaluate that in sim. Lastly, they also deploy their policy and action filtering method to a real robot and verify qualitatively that everything works.", "summary_of_recommendation": "This paper is an easy recommendation for me. It's set out to do a seemingly simple thing, shows that current approaches don't work as well, and then goes above and beyond (custom dataset, sim experiments, real-robot experiments) to show the merits of its method.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "See **W.2** and **W.3** above.\nAlso,\n\n- **Q.1** How does the human indicate their desired trajectory? In the video, it looks like waving in the general direction.\n- **Q.2** I get how this method helps with turn indications that come too early (due to interpolation) but I don't fully understand why that also works when the turn indication is too late.\n- **Q.3** Any idea why PPO diverges in some cases (Tab.3)?\n- **Q.4** I don't fully understand how PPO is trained or even set up in Habitat - You model both the human and dog there, right? I assume the dog is taking a random path through the house, based on a randomly sampled goal? How do you deal with the user's direction indications? How do you terminate an episode? ", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n### Strengths\n\n- **S.1** The paper is very focused and in my opinion sets a very reasonable goal (model human-robot interactions better) and goes above and beyond accomplishing this goal (by not just recording their own data but also looping back and deploying the method on a real robot). \n- **S.2** The writing is clear and concise.\n- **S.3** As far as I can tell, the method is novel and straight-forward (as in \"easy to follow\"). \n\nI also like that you're re-stating the research questions at the beginning of the experiment section.\nI also massively appreciate the release of the interaction dataset with the submission of the paper. This makes everything more reproducible.\n\n### Weaknesses\n\nThere aren't any major downsides to this for me and the only things I'm criticizing are lack of polish.\n\n- **W.1** The figures could use some work. I'd include a better caption for Fig.2 that actually summarizes the model, also in Fig. 2, the corrected offset variables are in the center overlapping the lines and hard to read. In general, maybe there's a better way to represent this in terms of colors and maybe it's worthwhile to replace the anonymous geometric shaped with a stylized robot dog and human. Fig.1 is a neat header graphics that shows what you're doing but the right side doesn't actually show action shielding - you could draw over the photos (like arrows on the floor) that indicate how action shielding prevents a bad action in the situation. In Fig.3, the fonts are too small. Fig. 4 (left) is really hard to parse for me. In Fig. 4 (right), I don't really know what to do with that information.\n- **W.2** Is there any way to quantify the results of the real-world experiments or are they more of a demo? How are the cues given?\n- **W.3** I don't fully understand why the action shielding is only applied during training and at test time, it's binary on-or-off (cf. Tab.3).\n- **W.4** In your limitations, you mention gaps in the lidar but you're not controlling the robot at the same framerate that the lidar is outputting data, right? So why not accumulate the lidar data?\n", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers commend the paper for being well-written and organized. The formalization of quadrupedal robots using RL to assist as guide dogs for those with visual impairments presents some interesting ideas. Reviewer SGRs notes several areas for paper improvement, and I would encourage the authors to focus additional efforts on addressing remaining points raised in the reviews.
Decision: Accept (Poster)