Seeing-Eye Quadruped Navigation with Force Responsive Locomotion Control
Authors: David DeFazio
Eisuke Hirota
Shiqi Zhang
Venue: CoRL 2023 Poster (2023)
Keywords: seeing-eye robot
robotic guide dog
human-robot interaction
quadruped locomotion

Abstract: Seeing-eye robots are very useful tools for guiding visually impaired people, potentially producing a huge societal impact given the low availability and high cost of real guide dogs. Although a few seeing-eye robot systems have already been demonstrated, none considered external tugs from humans, which frequently occur in a real guide dog setting. In this paper, we simultaneously train a locomotion controller that is robust to external tugging forces via Reinforcement Learning~(RL), and an external force estimator via supervised learning. The controller ensures stable walking, and the force estimator enables the robot to respond to the external forces from the human. These forces are used to guide the robot to the global goal, which is unknown to the robot, while the robot guides the human around nearby obstacles via a local planner. Experimental results in simulation and on hardware show that our controller is robust to external forces, and our seeing-eye system can accurately detect force direction. We demonstrate our full seeing-eye robot system on a real quadruped robot with a blindfolded human.

Seeing-Eye Quadruped Navigation with Force Responsive Locomotion Control
David DeFazio Eisuke Hirota Shiqi Zhang
Binghamton University
celefazii,ehirotte zhange @binghamton. edu
Abstract:
Seeing-eye robots are very useful tools for guiding visually impaired people, po- tentially producing a huge societal impact given the low availability and high cost of real guide dogs. Although a few seeing-eye robot systems have al- ready been demonstrated, none considered external tugs from humans, which frequently occur in real guide dog setting. In this paper, we simultancously train locomotion controller that is robust to external tugging forces via Rein- forcement Learning (RL), and external force estimator via supervised learn- ing. The controller ensures stable walking, and the force estimator enables the robot to respond to the external forces from the human. These forces are used to guide the robot to the global goal, which is unknown to the robot, while the robot guides the human around nearby obstacles via local planner. Experi- mental results in simulation and on hardware show that our controller robust external forces, and our seeing-eye system can accurately detect force di- rection. We demonstrate robot with a blindfolded human. The video can be seen at our project page: https: http///uuaaaaee aouu-air-lab.... .ub.githe github. io/ ho/guide_dog!
full seeing-eye robot system
real quadruped
Tug Left
Before Tug
During Tug
Afler Tug
Figure 1: Tugs on the seeing-eye robot are used as navigation cues during blindfolded navigation.
Introduction
In the typical seeing-eye dog (also known guide dog) setting, human holds rigid handle attached to a harness dog. The dog will safely guide the human around nearby obstacles, while the human can tug on the dog to indicate which general direction to move in. Thus, the human has some idea of where they are, and is capable of making high-level navigation decisions, while the dog can better sense its immediate surroundings for obstacle avoidance and locomotion. Guide dogs have been shown to improve the lives of visually impaired people, via increasing independence, confidence, companionship. and mobility [1]. Unfortunately, guide dogs need roughly two years of training. and cost over $50, 000 USD per dog [2]. Despite efforts to reduce the production cost of guide dogs [3], their supply is still significantly lower than demand.
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

To better meet this demand, and potentially improve performance, seeing-ey. eye robot systems are being developed [4, 5, 6, 7, 8, 9]. There is growing interest in the development of systems of this type, which have undergone various human studies in order to measure the extent to which they are compatible with visually impaired humans, and their level of societal acceptance [10, 11, 12]. While these seeing-eye robot systems have successfully demonstrated navigation tasks, none of them have considered settings involving human tugs, which is very common for seeing-eye dogs. It's important for seeing-eye robots to be robust to human forces, as the human is constantly holding rigid handle directly attached to the robot, and large forces can cause the robot to stray from the optimal path, or even fall over. Furthermore, a typical method of communication between a human and a real guide dog is through tugs. This makes force estimation useful for a seeing-eye robot, as knowledge of the direction and magnitude of the applied force can be used to better facilitate navigation according to the human's 's intentions (see Figure 1).
To address the above mentioned issues in human-robot communication, we develop novel seeing- eye robot system which is robust to external forces, and estimates the magnitude and direction of these forces to determine which navigation actions to take for human-robot co-navigation. We achieve this by simultaneously training a locomotion controller via RL, and a force estimator via supervised learning. Our locomotion policy is trained over simulated tugs by sampling different base veloeities which are suddenly applied the robot [13]. These tugs serve as labels for training force estimator, and are estimated during deployment.
While the controller is running on a real robot, the force estimator estimates the direction and mag- nitude of force the human applies. These force estimates are computed exclusively from sensors onboard the robot (joint encoders and IMU). From force estimates, the robot detects when and in what direction the human tugs occur. This informs the robot which direction to go a global level, while local planner using information from a LIDAR sensor is used navigate the immediate environment. Different from existing seeing-eye robot systems that require major hardware up- grades, e.g., customized traction device [6], button interface [7], our seeing-eye system needs is compatible with any attachable leash or handle.
Our main contributions include the following:
1. The first seeing-eye robot system which takes directional cues via human tugs, while also safely navigating the immediate environment.
2. A force tolerant locomotion controller, jointly trained with a force estimator which can estimate the magnitude and direction of human forces.
3. Experimental results in simulation and on hardware to evaluate the robustness of our loco- motion controller and accuracy of our force estimator.
4. Demonstration of our seeing- eye robot system in an indoor environment with a blindfolded human.
2 Related Work
Various seeing-eye robot systems have been demonstrated for the task of blindfolded navigation. Among these works, one of them considers MPC-based motion planner for wheeled robot [4]. Another considers optimization based approach for path planning, which models a taut slack leash [5]. A third system designs an adjustable leash, and optimizes for human comfort during navigation [6]. These works all stray from real-world guide dog settings, in part because they assume the robot has full knowledge of the destination beforehand, and that the human does not need to communicate with the robot during navigation. In real-world settings, the human must decide on which high-level navigation actions take, and communicate these actions the robot.
'The term of "Seeing-eye eye robots" has been used by researchers that refer to quadruped robots to guide people [26]. Although our robot doesn't it use vision, it serves as seeing-ey eye platform through its Lidar sensors for navigation and obstacle avoidance.

---

Train
Bosse Vilecll Estimator
Deploy
Velnddty
Narigation
John Torques,
Controllr
Connutte
Valocity Eskimliet
Navigation
JointAngles
Gcol
Estimator
Slensor Pasdinggs
Locomrtiion Policy
Tarpet start Angles
Coall Sulection
Robel Baate
Prtee Treeeeeddlig Info V.
(c.
Essimaaed
Centraliar
Direcion
Force Estinator
John Torques
Satn Bath Xe- X₁
Figure 2: Overview of our approach. Our locomotion controller (circled in red) contains velocity estimator, force estimator, and locomotion policy, all which are trained in simulation. The base velocity estimator and force estimator mation from the simulator as labels. The locomotion policy is trained via RL, and outputs target joint angles to PD controller which converts them to joint torques which directly applied to the robot. During deployment, our locomotion controller estimates external force at each time step. Force direction derived from peaks in the estimated force signal. The direction of force determines the next local navigation goal for our navigation system to take, which returns velocity commands to our controller.
trained via supervised learning, using privileged infor-
More similar to our work, other approaches consider settings where the human communicates high- level directions to the robot during navigation [7, 8, 9]. However, the medium of communication in these works are either a custom designed handle with buttons [7], predefined directional actions prior to navigation [8], or verbal commands [9]. In our work, the human communicates directional cues via tugging, and is compatible with any rigid connection to the robot. Additionally, none of these works consider human forces being applied to the robot, making their systems unresponsive or susceptible to failure upon navigation under human forces.
Numerous quadruped locomotion controllers have been developed, many of which are robust to external forces [14, 15, 16, 17, 18, 19, 20, 21, 13, 22, 23, 24]. These works do not explicitly estimate the applied forces, the focus of these works develop controllers that are generally robust to varying environments. In work, explicitly estimate forces the robot generated by human tugs, in order communicate the human's 's navigation intentions with the robot.
Method
In this section, we present our seeing-eye robot system that is robust and responsive to external forces from human tugs. Figure. 2 presents an overview of how we train our locomotion controller, and deploy for seeing-eye navigation.
3.1 Locomotion Controller
We train our locomotion controller via RL, which models environments as a Markov Decision Pro- cess (MDP). An MDP is defined M (S, A, T, R. 7), where S is the set of states, A is the set of actions, 7' S x A S - [0, 1] is the transition function which outputs the probability of reaching state s' given state and action a, R S x A S + RR is the reward function which returns feedback from taking action from state and ending up in state s', and E [0, 1] is the discount factor which determines how valuable future reward should be considered in comparison to immediate reward.
We define robot state at time t x, = (c, v, q. å, ar- 1), where с (ca, Cas Cw) is the commanded base linear and angular velocity, è is the base acceleration, and are the joint angles and velocities respectively, and a is the action taken at time f. 1. Actions are target joint angles, which are

---

All terms of the reward function our locomotion policy is trained on. v refers to base velocity, с refers to commanded linear and angular base velocity, refers to base angular velocity, r refers to joint torques, i refers to joint velocities, tair refers to each foots air time, a refers to an action, and dt refers to the simulation time step.
Table I:
Term Description
Defiiiion
Scale
Linear Velocity 2,
exp(
Cr.y Vr. 70.25)
1.0dtt
Linear Velocity 2
-2.0dt
Angular Velocity æ.
| ||
0.05dt
Angular Velocity &
exp( (Cw - w/) /0.25)
0.5dt
Joint Torques
||||-
0.00022f
Joint Accelerations
(dlast d)/d
2.5e - 7dt
Feet Air Time
2 (tair,r 0.5)
1.0dt
Action Rate
|last all
0.01dt
converted to torques via PD controller. The reward function encourages tracking commands while minimizing energy consumption and large action changes [13] - fully defined in Table 1.
Our locomotion policy also takes the hase velocity and external force vector as input, to more easily learn to track commands с and respond to external forces. These variables are not easily estimated through robot sensors. Thus, we train state estimators via supervised learning over privileged infor- mation, which has been shown be more effective than classical methods, e.g., Kalman Filters [25]. It is common setting to learn with privileged information in simulation, and insert corresponding state estimator in real-world deployment. In line with those systems, we train velocity estimator and force estimator for the real robot to bridge the sim-to-real gap in locomotion policy learning. These estimators are trained jointly with the locomotion policy, where v = (Vz, By, Uz) is ground truth base velocity, and F = (Fa, Fy F2) is ground truth external force, both obtained as privileged information from the simulator.
Two variations of forces are applied to the robot during training. One variation is small and frequent backward pushes, which are designed to simulate human following the robot with taut leash, where a human incidentally applies frequent small backward forces on the robot as they are being guided. The other variation is larger, less frequent pushes occurring in any direction. These pushes are designed to simulate human directional tugs, in which the human intentionally tugs the robot to communicate the direction they want to move The force estimator is only trained on data from the second variation of tugs, as we do not want detect the small, incidental backward tugs which naturally occur during guided navigation.
The base velocity estimator is a multilayer perceptron (MLP), whose parameters are updated over the same data as the locomotion policy. To estimate forces, we find it helpful to access a history of states, to better capture the robot's behavior over the duration of the applied force. Thus, similar to training adaptation modules [17], our force estimator uses 1-D convolutional layers capture temporal relationships between states. Force estimator parameters are updated less frequently than the base velocity estimator and locomotion policy, because most time steps do not include external forces. Thus, most of the labels for our force estimator includes zero vectors, indicating that no force was applied at those time steps. This causes imbalanced training data, which we resolve by only training the force estimator when the training data contains nonzero forces. We then further re-balance this to ensure at most 20% of the force estimator's 's training samples include zero vectors as labels.
3.2 Seeing-eye Robot Navigation
To perform navigation tasks with our seeing-eye robot, we need to estimate when and in what general direction force being applied. This is done by running peak detection [26] on the previous 200 time steps of the estimated force signal Fy rate of 2Hz. In this work, we only analyze Fy. to determine whether left right tug has occurred. A peak in the signal indicates that the force estimator detected a significant external force applied to the robot. Thus, if peak is detected within

---

Turn left
End
Navigate through
Start
doorway
Move around person
Figure 3: Map of our navigation environment. Yellow circles correspond to decision points, where the human needs to decide which direction to move in via tugging. The blue lines indicate the path the human took in our demonstration.
the past 50 time steps, then we consider it as a recent tug applied by the human. Positive peaks correspond to left tugs, while negative peaks correspond to right tugs.
Local navigation goals are then selected based the robot's 's location, orientation, and tug direction. A domain expert manually labels map with decision points, where the human must decide which direction to go in next, based on the direction they came from, and their tug direction. A labelled map of the hallway domain we demonstrate our system on can be seen in Figure 3.
The local navigation goal then sent to our navigation system, which uses LIDAR sensor to localize itself on the map via AMCL [27], and compute local plans via DWA [28]. The local planner returns velocity commands our controller, which our controller tracks.
Implementation Details
We use Isaac Gym [29] physics simulator, and train controller with 2048 robots in parallel [13]. Our locomotion policy is trained via PPO [30], while our base velocity estimator and force estimator are trained via supervised learning, with Mean Squared Error loss. Our locomotion policy and base velocity estimator are both MLPs with three and two hidden layers respectively. while our force estimator is a convolutional neural network with three 1-D convolutional layers, which makes predictions over past 25 time steps. The PD controller converts target joint angles to torques with proportional gain
0.S. The policy is queried at 50Hz, and control signals are sent 200Hz. New velocity commands are sampled after each episode, where Cz. Cy. and cu sampled uniformly from [-1,1]. An episode terminates when any link other than foot touches the ground, the base height is below 0.25 meters, or the episode has lasted 20 seconds.
20 and derivative gain
RANDOM.UNIFORM To better facilitate sim-to-real transfer, we train over RANDOM UNIFORM TERRAI AIN which increases difficulty based on a curriculum [13]. We also include noise in observations, and domain random- ization over different surface frictions.
We add random external forces in our environment, to improve robustness of our locomotion policy and collect data to train our force estimator. The small and frequent backward pushes occur every 0.6 seconds, have duration 0. seconds, and sets the base velocity to 0.25 m/s backward. Meanwhile, the large and infrequent pushes used to train the force estimator occur every seconds, have duration sampled from [0.24, 0.48] seconds, and sets the base velocity vector sampled from F. [-0.75, 0.75), Fg [-0.75, 0.75), and Fz [0, 0.1]. Note that forces from F are implemented spontaneous updates in base velocity.
Experiments
We design experiments to evaluate the robustness of our controller, and accuracy of our force esti- mator. Although are able learn force estimator in simulation, could not directly evaluate
5

---

Table 2:
Our learned controllers fell significantly less frequently, and better tracked velocity com- mands under external forces when compared to an MPC controller. Including the output of the force estimator in the state marginally improved robustness. The large variance in drift is caused by the large range of force strengths and directions we sample from.
Controller
Proportion Fell
Drift from Trajectory
MPC
0.5990
1.1256 + 0.5908
I.carned No Fsst.
0.1904
0.6824 0.5447
Learned Est
0.1762
0.6790 + 0.5309
it in the real world due to the missing ground truth values. Instead, we chose to evaluate tug detec- tion (LEFT, RIGHT, and NONE) on the real robot, where the participants followed our instructions, and hence ground truth was available. We then demonstrate our full seeing-eye system via guiding a blindfolded human.
5.1 Force Tolerance Evaluation
To evaluate whether our learned force controller is actually robust to external forces, run exper- iments in simulation where we apply random forces to the base of the robot. In each trial, a single force in random direction is applied, whose duration is sampled from [0.25, 0.5] seconds, and strength is sampled from [25, 100] Newtons. Meanwhile, the robot is commanded to walk forward 0.5 meters/second, for a duration of five seconds.
We run this experiment on three different types of locomotion controllers, for 1000 trials each. The controllers include a commonly used MPC controller [31], variant of our learned controller which does not consider estimated force in its state (referred to as Learned No Est), and our controller described in Section 3.1 (referred to as Learned Est). All controllers are deployed in PyBullet [32].
We measure how frequently the robot fell across all trials (Proportion Fell), and how far the external force caused the robot to drift from its current trajectory average (Drift from Trajectory). We consider a robot to have fell if non-foot part of the robot touches the ground. Results are reported in Table 2, which indicate that our learned controllers fell significantly less frequently, and better maintained velocity tracking under external forces than the MPC controller. Including estimated forces in the state appears to marginally improve robustness. Note that for our two types of learning- based controllers, we train over five different random seeds each and average the results.
5.2
Force Estimation Evaluation
We evaluate the accuracy of our force estimator through experiments in simulation, and on hardware.
5.2.1 Simulation
Force Detection Accuracy and False Positive Rate
We deploy our learned controllers in PyBullet with a ve- locity command of 0.5 meters/second, and a single ex- ternal push per trial, for 1000 trials. Each push has force whose x-component is sampled from 50] New- tons, and y-component which is at fixed magnitude, and random direction (either left or right). There are three possible classes the force estimator can predict over:
1.0
Accuracy
0.5
Only
0.0
0.2
Faize Poztave Rate
1.0
Full Strte
Only
(LEFT, RIGHT, NONE). Each trial includes 150 time steps and takes seconds. each trial, the force estima- tor is queried every 25 time steps, six total queries per trial. Thus, in each trial, the force estimator makes total of six predictions. A trial is deemed correct if one of the six queries matches the force direction being applied in the simulator.
-0.2
30
45
55
60
applled
Figure 4: We report the accuracy and false positive rate of our force estima- tors, given forces of varied strength. The shaded region indicates the stan- dard deviation between the five policies trained over five different random seeds.
In order evaluate whether force direction can be esti- mated through only history of base velocities, train baseline force estimator which only makes force esti-

---

mates based on history of ground-trum trunh base velocity and base velocity commands. We refer to this baseline as only vel, while our force estimator trained over the full state and described in Section 3.1 is referred to as full state.
In Figure 4, report the accuracy and false positive rates of force estimators over varying force strengths. Accuracy is computed by dividing the number of trials in which the force estimator predicted the correct force direction at any of the six queries in the trial, by the number of total trials. Computing accuracy alone in this manner is not informative enough, because it is possible to achieve high accuracy by predicting LEFT at one point in the trial, and RIGHT at a different point within the same trial.
Thus, also consider the false positive rate, which we compute by dividing the number of extra forces (LEFT RIGHT predicted when ground truth is NONE) predicted during the trial, by the number of times the force estimator is queried (every 25 time steps). A high false positive rate corre- sponds to the force estimator oftentimes predicting forces when they do not occur. As force strength increases, our estimators achieve a higher accuracy while maintaining a relatively low false positive rate. Results indicate that knowledge of the full state significantly beneficial in estimating force direction, when compared to force estimator which is only trained over base velocity information.
5.2.2 Hardware
When human tugs our seeing-eye robot with sufficient force, the base of the robot will momentarily accelerate in the direction of the tug. Thus, one might wonder why do not consider accelerometer signals to detect tug direction, rather than train force estimator. In this sub-section, we validate the usefulness of our estimated force signals, which we compare to accelerometer readings.
In this experiment, we command a real Unitree A1 robot move forward 0.5m/s, while human par- ticipant tugs left after a few seconds of forward lo- comotion, followed by right tug after another few seconds of locomotion. This trial is performed by four human participants, three of which have no prior experience in operating this tugging interface. Each participant is a robotics researcher a university lab.
Accelerometer Readings During Tugs
Aicelbromener Stending
The three participants with no prior experience with this system were given demonstration of an example trial, before completing their own trials. In total, 42 trials were conducted, and data was collected from 40 trials (two trials were removed due to the robot falling over and data not being saved). Of these 40 trials, each participant performed ten of them.
100
200
300
400
500
600
Timestep
Estimated Forces During Tugs
Extimated Forca
-0.1
Forces are detected every 25 timesteps, and can be predicted one of three classes. We compute the ac- curacy and false positive rate for force detectors using the estimated force signal, compared to force detec- tion using the signal directly from the accelerometer on the robot. Accuracy computed as the percentage of trials which contain a LEFT force prediction be- fore the halfway point of the trial, and RIGHT force prediction after the halfway point of the trial. False positive rate is the average percentage of false positive forces being predicted across all trials. A force prediction is considered as false positive it is either LEFT or RIGHT, and does not correspond to the first expected LEFT force or later expected RIGHT force.
1.00
200
900
4000
500
600
Timestop
Figure 5: Measured acceleration (top) and estimated force (bottom) during a single trial. Tugs are denoted by red boxes.
Results are reported in Table 3, where find the force estimator more accurately predicted the correct forces, with fewer false positives compared to predictions from raw accelerometer signals. Both methods of detecting tug direction (accelerometer and force estimator) perform worse for be-

---

Table Force estimation via accelerometer readings vs force estimator signal.
Participant Type
Method
Accuracy
False Positive Rate
Accelerometer
0.20
0,2149
Expert (10 trials)
Force Estimator
L.00
0.0442
Accelcrometer
0.13
0.1906
Beginner (30 trials)
Force Estimator
0.70
0.0498
ginner participants, indicating that adding diverse tugging styles is important for complete evaluation of force estimators.
In Figure 5, we plot the raw signals from the accelerometer and force estimator from a single trial. We find the accelerometer signal is more noisy than the estimated force signal, which we hypothe- size because the force estimator has access to other sensor information along with accelerometer readings. Note that our force estimator is co-learned with the locomotion policy. The locomotion policy takes the estimated force input during training, and the states generated from the policy are used to train the force estimator. We believe this co-learning mechanism leverages the additional sensor information to help the force estimator outperform the tug detection from raw accelcrometer readings.
6 Hardware Demonstration
We demonstrate our full secing-eye robot system on a Unitree Al robot, in an indoor hallway envi- ronment (see link in Abstract). Our locomotion policy and force estimator are deployed on hardware without any additional fine-tuning.
In this demonstration, human is blindfolded, and holding taut leash attached to the robot. The human desires to reach some particular goal location, and chooses the rout to take by tugging the robot at decision points, while the robot autonomously avoids obstacles (including boxes, narrow doorways, and another human) along the way. Decision points occur at intersections in the hallway, where the human needs to decide in which direction they want to go. Similar to the setting in [7], the blindfolded human is not familiar with localizing himself without vision, so another sighted human verbally indicates when decision point is approaching.
This demonstration indicates that our locomotion policy and force detector are transferable to hard- ware, and can cooperate with local planner to enable blindfolded navigation. Thus, successful human-robot communication occurred, such that the human avoided all obstacles while the robot navigated to the desired goal location through detecting human tugs at decision points.
7 Discussion
Limitations and Future Work While our seeing-eye system can avoid obstacles and select routes course level in indoor hallway environments, real guide dog settings include outdoor environ- ments, and situations with many possible directions navigate in. future work, researchers can leverage methods to determine which paths are traversable [33], and train force detectors which can estimate the direction of force finer level. Another future direction is incorporate intelligent disobedience, which refers to rejecting human's 's navigation decision if unsafe [34]. Additionally, The evaluation can be further improved by replacing sighted people with those with visual impair- ments in the experiments. Finally, our robot's 's locomotion speed is relatively low. It might be the human tugs, the learned locomotion policy, or both causing the low speed. Further investigation into those factors can potentially lead very interesting future research and higher-speed speed seeing-eye robot systems.
Conclusion We train a locomotion controller which is tolerant to human tugs, and can estimate the direction of external forces. We evaluate the robustness of our controller, and accuracy of our force estimator through experiments in simulation and hardware. Finally, we demonstrate our controller real robot for the task of blindfolded navigation, where blindfolded human is successfully guided to destination, while giving directional cues through tugging the robot.

---

Acknowledgement
This work has taken place at the Autonomous Intelligent Robotics (AIR) Group, SUNY Bing- hamton. AIR research supported in part by grants from the National Science Foundation (NRI- 1925044), Ford Motor Company, OPPO, and SUNY Research Foundation.
References
[1] L. Whitmarsh. The benefits of guide dog ownership. Visual impairment research, 7(1):27-42, 2005.
[2] C. Morita. How much does guide dog cost? how oot.://ww......aooooooooooooooa.
http:://papyyintrraiinn...o http://www...oooooooooooooooooo.
[3] L. M. Tomkins, C. Thomson, and D. McGreevy. Behavioral and physiological predictors of guide dog success. Journal of Veterinary Behavior, 6(3):178-187, 2011.
[4] L. Wang, J. Zhao, and L. Zhang. Navdog: robotic navigation guide dog via model predictive control and human-robot modeling. In Proceedings of the 36th Annual ACM Symposium on Applied Computing, pages 815-818, 2021.
[5] A. Xiao, W. Tong, L. Yang. J. Zeng. Z. Li, and K. Sreenath. Robotic guide dog: Leading a human with leash- booh-guided hybrid physical interaction. 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 1 11470-11476. IEEE, 2021.
[6] Y. Chen, Z. Xu, Z. Jian, G. Tang. Y. Yangli, A. Xiao, X. Wang, and B. Liang. Quadruped guidance robot for the visually impaired: A comfort-based approach. arXiv preprint arXiv:2203.03927. 2022.
[7] H. Hwang, Xia, Keita, K. Suzuki, Biswas, Lee, and D. Kim. System configuration and navigation of guide dog robot: Toward animal guide dog-level guiding work. arXiv preprint arXiv:2210. 13368, 2022.
[8] J. T. Kim, W. Yu, J. Tan, G. Turk, and S. Ha. How to train your guide dog: Wayfinding and safe navigation with human-robot modeling. In Companion of the 2023 ACM/IEEE International Conference on Human-Robol Interaction, pages 221-225, 2023.
[9] T. Kim, W. Yu, Y. Kothari, J. Tan, G. Turk, and S. Ha. Transforming a quadruped into a guide robot for the visually impaired: Formalizing wayfinding, interaction modeling, and safety mechanism. arXiv preprint arXiv:2306. 22022.66.4055... 2023.
[10] Q. Chen, L. Wang, Y. Zhang, Z. Li, T. Yan, F. Wang, G. Zhou, and J. Gong. Can quadruped navigation robots be used as guide dogs? arXiv preprint arXiv:2210.08727, 2022.
[11] Y. Zhang, Li, Guo, L. Wang, Q. Chen, W. Jiang, M. Fan, Zhou, and Gong. " am the follower, also the boss": Exploring different levels of autonomy and machine forms of guiding robots for the visually impaired. In Proceedings of the 2023 CHIl Conference on Human Factors in Computing Systems, pages 1-22, 2023.
[12] P. Chonkar, G. Hemkumar, H. Wang, D. Dua, S. Gupta, Y.C. Chan, J. Hart, E. Hauser, R. Mirsky, J. Biswas, et al. Look to my lead: How does a leash affect perceptions of a quadruped robot?
[13] N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning walk in minutes using massively parallel deep reinforcement learning, In Conference on Robot Learning, pages 91-100. PMLR, 2022.

---

[14] N. Kohl and P. Stone. Policy gradient reinforcement learning for fast quadrupedal locomotion. In IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004, volume 3, pages 2619-2624 IEEE, 2004.
[15] Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and Kim. Dynamic locomotion in the mit cheetah 3 through convex model-predictio control. In 2018 IEEE/RS/ international confer- ence intelligent robots and systems (IROS), pages 1-9, IEFE, 2018.
[16] J. Tan, T. Zhang, F. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke. Sim- to-real: Learning agile locomotion for quadruped robots. arXiv preprint 18X4:1800..000 2018.
[17] A. Kumar, Z. Fu, D. Pathak, and J. Malik. Rma: Rapid motor adaptation for legged robots. 2021.
arXiv preprint
[18] S. Chen, B. Zhang, M. W. Mueller, A. Rai, and K. Sreenath. Learning torque control for quadrupedal locomotion, arXiv preprint arXiv:2203.05. 200.22003.05944, 2022.
[19] L. Campanaro, S. Gangapurwala, W. Merkt, and I. Havoutis. Learning and deploying robust locomotion policies with minimal dynamics randomization. arXiv preprint arXiv:2209, 12878, 2022.
[20] G. B. Margolis and P. Agrawal. Walk these ways: Tuning robot control for generalization with multiplicity of behavior. arXiv preprint arXiv:2212.0322. 2022.
[21] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning robust per- ceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7(62))ebb22222, 2022.
[22] A. Agarwal, A. Kumar, J. Malik, and D. Pathak. Legged locomotion challenging terrains using egocentric vision. Conference on Robot Learning, pages 403-415. PMLR, 2023.
[23] 7. Zhuang, 7. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao. Robot parkour learning. arXiv preprint 1.XXi:2339000660 2023.
[24] X. Cheng, Shi, A. Agarwal, and D. Pathak. Extreme parkour with legged robots. arXiv preprint arXiv:2309.14:: 14341, 2023.
[25] G. Ji. J. Mun, H. Kim, and J. Hwangbo. Concurrent training of control policy and state estimator for dynamic and robust legged locomotion. IEEE Robotics and Automation Letters, 70))4660-4463,2003
[26] M. Dede. peakdetect. http:://ggicuub..comaaav///aaaee 2022.
[27] B. P. Gerkey. Amel, 2013. URL http:///:/iik wrg.a/ol.....mml........
[28] D. Fox, W. Burgard, and S. Thrun. The dynamic window approach to collision avoidance.
IEEE Robotics Automation Magazine, 4(1):23-33, 1997.
[29] V. Makoviychuk, Wawrzyniak, Guo, M. Lu, Storey, M. Macklin, Hoeller, N. Rudin,
A. Allshire, A. Handa, et al. Isaae gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.1I 104.000,0 2021.
[30] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint 1707.06677. 2017.
[31] X. B. Peng. E. Coumans, T. Zhang, T.-. E. Lee, J. Tan, and S. Levine. Learning agile robotic locomotion skills by imitating animals. In Robotics: Science and Systems, 07 2020. 200.1.115607//SS2220000000
10

---

[32] E. Coumans and Y. Bai. Pybullet, python module for physics simulation for games, robotics and machine learning. 2016.
[33] Frey, D. Hoeller, S. Khattak, and M. Hutter. Locomotion policy guided traversability learning using volumetric representations of complex environments. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1722-5729 IEEE, 2022.
[34] R. Mirsky and Stone. The seeing-eye robot grand challenge: Rethinking automated care. In Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), 2021.


Reviews:
[{"review_id": "TMjLimW9Nb", "reviewer": "Reviewer_BKt3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This work presents a seeing-eye quadrupedal robot capable of taking directional cues from human tugs via force estimation, simultaneously offering safe traversal in an indoor environment. The authors are motivated by the lack of human tug input to the robot for directional purposes.  The authors present a scheme allowing the robot to learn to estimate the magnitude and direction of the forces due to human tugs. The paper presents simulation results and real-world tests to show their proposed approach. ", "summary_of_recommendation": "Motivated by the limitations on the use of human tugs to give direction cues to seeing-eye robots, the authors utilized learning techniques on force estimation and simultaneous locomotion generation from intermittent tugs. The presented approach to guidance robots is novel and future robots might benefit from the contribution.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "It would be interesting to see the robot navigate at higher speeds.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is well organized and the writing is clear. The presented methods are sound. The paper is interesting and original because the authors use force estimation from human tugs to give direction commands to a seeing-eye robot where such human-robot interaction had drawn little attention in the literature. Using simulation and real-world experiments the authors illustrate that the robot can learn to facilitate and assist navigation according to the human’s intentions. However, the robot locomotion speeds remain slow and might affect the applicability of the proposed approach in real settings. \n\n", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "LonM73p2I6p", "reviewer": "Reviewer_wGQP", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This is the first paper to consider human tugs for guiding a guide robot dog.", "summary_of_recommendation": "Overall a nice application of machine learning in robotics, for an inspiring paper and purpose.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "There are two questions for the authors:\n1. why locomotion was learned instead of using Unitree's (or other robots) MPC locomotion and learn the force estimation only?\n2. how did you evaluate the correct force estimation on the real hardware without ground truth?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This is really a nice and inspiring paper -- mainly for the application and the promotion of diversity in the area of legged robots, than for the methodology itself. Nevertheless, a supervised force estimation and implementation on the real robot makes the paper really nice. The locomotion is basic with RL (and maybe not the focus of the paper). The focus is the estimation of the force to affect the local plan decision when navigating globally.", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "DwJlsMxmXj", "reviewer": "Reviewer_Pxez", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents a system that allows a quadruped robot to guide visually impaired humans. Directional cues can be taken from human tug via the use of the force estimator. Demonstration of the system is done in both simulation and the real robot.", "summary_of_recommendation": "This paper builds a nice system to use quadrupedal robots as guide dogs. However, because of my concern about the importance of a learned force estimator, I recommend weak rejection for now. If additional experiments can show its necessity, I am willing to raise my score.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "Given my concern above, it will be nice to have a baseline where the force estimator is only trained with base information instead of the full state. \n\nIt will also be interesting to filter the raw accelerator readings to see if the signal becomes clearer.\n\nAnother comparison will be to compare the learned base velocity estimator with a standard velocity estimator based on Kalman Filter. A Kalman Filter-based estimator can also be used to generate acceleration signals to detect tugs for comparison.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The system is straightforward and well executed, with sufficient evaluations in both simulation and the real robot. \n\nA main novelty of the work is the use of a force estimator, which can both help improve the robustness of the policy as well as help the user to provide directional cues.\n\nHowever, the arguments against directly using an accelerometer signal are a little bit circular. In the paper, the authors state: \". For example, the policy might learn to make certain joint movements when a tug occurs. Our force estimator can learn to associate these joint movements with the corresponding tug direction, while the accelerometer has no such access to this information. \"\n\nHere are some questions about this argument:\n\n1. The policy makes certain joint movements when a tug occurs. But what triggers those movements? Because of the tug, not the other way around, i.e., the policy first detects the tug. Then how does the policy detect it? It is kind of like a chicken-egg argument, hope this makes sense.\n\n2. My guess will be the policy detects the tug by observing changes in velocity signal that deviates from what will happen if there are no tugs. Then it is entirely possible to detect tugs from body velocity only. \n\nSince the force estimator is the main contribution here, I think it is important to investigate this issue further.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "7PzQGr74K8", "reviewer": "Reviewer_5maN", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The authors proposed a locomotion algorithm for seeing-eye robots where external tug forces are used to determine high-level local navigation directions, which further specify low-level velocity targets. The novel proposal of an online external force estimation module provides a reasonable replacement for accelerometer reading which can be noisy and unreliable in practice. The hardware experiments showed convincing results and effectiveness in guiding visually impaired people by using the proposed system.", "summary_of_recommendation": "I acknowledge the authors' insights into the potential of the work in helping visually impaired people. And I acknowledge the fact that the proposed system works well in limited scenarios under specific assumptions. This work can be viewed as the first step in achieving the eventual goal it argues for. However, as a research paper, I would expect it to have either technical novelty or strong application performance. As the paper is developed by combining existing works, I did not see strong evidence of novel technical contribution. In this case, to be considered to be accepted, the paper should show strong and convincing application system performance with high generality and robustness. Nevertheless, the proposed system is shown effective only in very limited domains under strong simplifications and assumptions. I would suggest the authors bring more convincing results of extended capabilities at the rebuttal phase.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "- The paper did not touch on how the navigation system selects the goal and determines the commanded velocity after receiving detected tug forces. More information on the navigation module is needed. \n- The paper did not argue clearly the reason for using the base velocity estimator. More information is needed.\n- The results in Table 2 are not convincing given the large variance in the drift column. Please either explain the reason or consider other evaluation metrics. \n- More technical novelty or better deployment capability is expected. Please illustrate the challenges you solved, which would have not been solved by using other methods.\n- It would be more convincing to show ablation on external force sampling strategy during training, especially for the need of considering small back-pushing forces.\n- Please address the weaknesses above accordingly.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Strengths**\n- The work provides a reasonable interface for commanding legged locomotion. The hardware experiment on the specified setup is convincing.\n- It argues for the value of the external force estimation by explicitly comparing it with the results from accelerometer readings.\n- The distinguishment of two kinds of external forces applied to the robot makes a good point.\n\n**Weaknesses**\n- The work has limited novelty on the technical side. The RL locomotion framework follows an existing open-sourced work and the addition of the training of the estimators is also not new. The work seems to combine two existing works (Learning to Walk in Minutes + RMA) to me with a well-wrapped estimated force command interface. \n- While the work seems to be motivated for significant help in guiding visually impaired people, the final results do not seem convincing. As the authors mention in the limitation, the setting assumes prescribed decision points in a known map, considers only binary tug directions, works under specific running velocity settings, and has limited traversability. Summing up these points, the work seems to have too many simplifications to stand for its argument to help visually impaired people. More results on an extended capability are expected, especially given the limited algorithmic contribution.\n- The setup does not seem self-contained. At decision points (which are already known following the assumption in the work), why not directly set such binary commands rather than going through a tug-force interface? And if there is a sighted person dictating the commands, why would we even need the robot?", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "1-2fwgWtnN", "reviewer": "Reviewer_SGaq", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper documented co-training of a legged robot locomotion policy and external force estimator and using the force estimator as a form of user input in navigation context.\n\nIn the paper the method of training is documented in detail (reward design, model arch, task setup). The results include force tolerance in sim and force estimation evaluation in both simulation and real.  ", "summary_of_recommendation": "Recommend accept the paper if the rebuttal questions are addressed.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1) The force estimation work can be better supported by a quantitatively evaluated on real robot. This will address the sim to real question.\n\n2) Please clarify how the metrics are computed. Otherwise some of the numbers are difficult to interpret.\n\n3) No vision is involved in this work although the paper is titled \"Seeing-eye\", this is a bit misleading. Consider changing the title or clarify at the beginning of the paper.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper has extend the quadruped locomotion controller with force estimator and noveally use the estimator not only in the locomotion policy but also as a form of user input.\n\nThis work is well documented with details.\n* The model arch and reward is clearly illustrated and documented. Training methods is also well documented.\n* The idea of using force estimation not only for a more robust but also slightly improves tolerance of force.\n\nA few places that can be improved.\n* The force estimation part is not quantitatively evaluated on real robot.\n* Some metrics of the experiments are not clearly described. For example, in table 2, the \"proportion fell\" for \"learning no est\" is 0.1904 in 1000 trials. It is not very clear how the metrics is measured.\n* The force tolerant part of the paper is not compared with other work in \"related work\".\n* While the paper is titled \"Seeing-eye\" and is for visually impaired people, the paper's content requires human to \"guide\" the robot. This can be misleading.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The AC is excited to recommend accepting this work on the given positive reviews. The paper presents a novel force-responsive locomotion control system, which is well-validated. Please resolve the reviewers' comments as much as possible at the final stage. It will also be great to have a user study with the actual user population. Congratulations!
Decision: Accept (Poster)