OpenRaiser/Intern-Atlas huggingface dataset:
Gastrocnemius and Power Amplifier Soleus Spring-Tendons Achieve Fast Human-like Walking in a Bipedal Robot
Venue: Conference IROS 2022
Year: 2022

Legged locomotion in humans is governed by natural dynamics of the human body and neural control. One mechanism that is assumed to contribute to the high efficiency of human walking is the impulsive ankle push-off, which potentially powers the swing leg catapult. However, the mechanics of the human lower leg with its complex muscle-tendon units spanning over single and multiple joints is not yet understood. Legged robots allow testing the interaction between complex leg mechanics, control, and environment in real-world walking gait. We developed a 0.49m tall, 2.2kg anthropomorphic bipedal robot with Soleus and Gastrocnemius muscle-tendon units represented by linear springs, acting as mono- and biarticular elastic structures around the robot's ankle and knee joints. We tested the influence of three Soleus and Gastrocnemius spring-tendon configurations on the ankle power curves, the coordination of the ankle and knee joint movements, the total cost of transport, and walking speed. We controlled the robot with a feed-forward central pattern generator, leading to walking speeds between 0.35m/s and 0.57m/s at 1.0Hz locomotion frequency, at 0.35m leg length. We found differences between all three configurations; the Soleus spring-tendon modulates the robot's speed and energy efficiency likely by ankle power amplification, while the Gastrocnemius spring-tendon changes the movement coordination between ankle and knee joints during push-off.
Graph-Structured Policy Learning for Multi-Goal Manipulation Tasks
Venue: Conference IROS 2022
Year: 2022

Multi-goal policy learning for robotic manipulation is challenging. Prior successes have used state-based representations of the objects or provided demonstration data to facilitate learning. In this paper, by hand-coding a high-level discrete representation of the domain, we show that policies to reach dozens of goals can be learned with a single network using Q-learning from pixels. The agent focuses learning on simpler, local policies which are sequenced together by planning in the abstract space. We compare our method against standard multi-goal RL baselines, as well as other methods that leverage the discrete representation, on a challenging block construction domain. We find that our method can build more than a hundred different block structures, and demonstrate forward transfer to structures with novel objects. Lastly, we deploy the policy learned in simulation on a real robot.
A Riemannian Take on Human Motion Analysis and Retargeting
Venue: Conference IROS 2022
Year: 2022

Dynamic motions of humans and robots are widely driven by posture-dependent nonlinear interactions between their degrees of freedom. However, these dynamical effects remain mostly overlooked when studying the mechanisms of human movement generation. Inspired by recent works, we hypothesize that human motions are planned as sequences of geodesic synergies, and thus correspond to coordinated joint movements achieved with piecewise minimum energy. The underlying computational model is built on Riemannian geometry to account for the inertial characteristics of the body. Through the analysis of various human arm motions, we find that our model segments motions into geodesic synergies, and successfully predicts observed arm postures, hand trajectories, as well as their respective velocity profiles. Moreover, we show that our analysis can further be exploited to transfer arm motions to robots by reproducing individual human synergies as geodesic paths in the robot configuration space.
Detecting Adversarial Perturbations in Multi-Task Perception
Venue: Conference IROS 2022
Year: 2022

While deep neural networks (DNNs) achieve impressive performance on environment perception tasks, their sensitivity to adversarial perturbations limits their use in practical applications. In this paper, we (i) propose a novel adversarial perturbation detection scheme based on multi-task perception of complex vision tasks (i.e., depth estimation and semantic segmentation). Specifically, adversarial perturbations are detected by inconsistencies between extracted edges of the input image, the depth output, and the segmentation output. To further improve this technique, we (ii) develop a novel edge consistency loss between all three modalities, thereby improving their initial consistency which in turn supports our detection scheme. We verify our detection scheme's effectiveness by employing various known attacks and image noises. In addition, we (iii) develop a multi-task adversarial attack, aiming at fooling both tasks as well as our detection scheme. Experimental evaluation on the Cityscapes and KITTI datasets shows that under an assumption of a 5% false positive rate up to 100% of images are correctly detected as adversarially perturbed, depending on the strength of the perturbation. Code is available at https://github.com/ifnspaml/AdvAttackDet. A short video at https://youtu.be/KKa6gOyWmH4 provides qualitative results.
RECALL: Rehearsal-free Continual Learning for Object Classification
Venue: Conference IROS 2022
Year: 2022

Convolutional neural networks show remarkable results in classification but struggle with learning new things on the fly. We present a novel rehearsal-free approach, where a deep neural network is continually learning new unseen object categories without saving any data of prior sequences. Our approach is called RECALL, as the network recalls categories by calculating logits for old categories before training new ones. These are then used during training to avoid changing the old categories. For each new sequence, a new head is added to accommodate the new categories. To mitigate forgetting, we present a regularization strategy where we replace the classification with a regression. Moreover, for the known categories, we propose a Mahalanobis loss that includes the variances to account for the changing densities between known and unknown categories. Finally, we present a novel dataset for continual learning, especially suited for object recognition on a mobile robot (HOWS-CL-25), including 150,795 synthetic images of 25 household object categories. Our approach RECALL outperforms the current state of the art on CORe50 and iCIFAR-100 and reaches the best performance on HOWS-CL-25.
InCloud: Incremental Learning for Point Cloud Place Recognition
Venue: Conference IROS 2022
Year: 2022

Place recognition is a fundamental component of robotics, and has seen tremendous improvements through the use of deep learning models in recent years. Networks can experience significant drops in performance when deployed in unseen or highly dynamic environments, and require additional training on the collected data. However naively fine-tuning on new training distributions can cause severe degradation of performance on previously visited domains, a phenomenon known as catastrophic forgetting. In this paper we address the problem of incremental learning for point cloud place recognition and introduce InCloud, a structure-aware distillation-based approach which preserves the higher-order structure of the network's embedding space. We introduce several challenging new benchmarks on four popular and large-scale LiDAR datasets (Oxford, MulRan, In-house and KITTI) showing broad improvements in point cloud place recognition performance over a variety of network architectures. To the best of our knowledge, this work is the first to effectively apply incremental learning for point cloud place recognition. Data pre-processing, training and evaluation code for this paper can be found at https://github.com/csiro-robotics/InCloud.
L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learning
Venue: Conference IROS 2022
Year: 2022

This paper proposes a new regularization technique for reinforcement learning (RL) towards making policy and value functions smooth and stable. RL is known for the instability of the learning process and the sensitivity of the acquired policy to noise. Several methods have been proposed to resolve these problems, and in summary, the smoothness of policy and value functions learned mainly in RL contributes to these problems. However, if these functions are extremely smooth, their expressiveness would be lost, resulting in not obtaining the global optimal solution. This paper therefore considers RL under local Lipschitz continuity constraint, so-called L2C2. By designing the spatio-temporal locally compact space for L2C2 from the state transition at each time step, the moderate smoothness can be achieved without loss of expressiveness. Numerical noisy simulations verified that the proposed L2C2 outperforms the task performance while smoothing out the robot action generated from the learned policy.
The Role of Tactile Sensing in Learning and Deploying Grasp Refinement Algorithms
Venue: Conference IROS 2022
Year: 2022

A long-standing question in robot hand design is how accurate tactile sensing must be. This paper uses simulated tactile signals and the reinforcement learning (RL) framework to study the sensing needs in grasping systems. Our first experiment investigates the need for rich tactile sensing in the rewards of RL-based grasp refinement algorithms for multi-fingered robotic hands. We systematically integrate different levels of tactile data into the rewards using analytic grasp stability metrics. We find that combining information on contact positions, normals, and forces in the reward yields the highest average success rates of 95.4% for cuboids, 93.1% for cylinders, and 62.3% for spheres across wrist position errors between 0 and 7 centimeters and rotational errors between 0 and 14 degrees. This contact-based reward outperforms a non-tactile binary-reward baseline by 42.9%. Our follow-up experiment shows that when training with tactile-enabled rewards, the use of tactile information in the control policy's state vector is drastically reducible at only a slight performance decrease of at most 6.6% for no tactile sensing in the state. Since policies do not require access to the reward signal at test time, our work implies that models trained on tactile-enabled hands are deployable to robotic hands with a smaller sensor suite, potentially reducing cost dramatically.
On CAD Informed Adaptive Robotic Assembly
Venue: Conference IROS 2022
Year: 2022

We introduce a robotic assembly system that streamlines the design-to-make workflow for going from a CAD model of a product assembly to a fully programmed and adaptive assembly process. Our system captures (in the CAD tool) the intent of the assembly process for a specific robotic workcell and generates a recipe of task-level instructions. By integrating visual sensing with deep-learned perception models, the robots infer the necessary actions to assemble the design from the generated recipe. The perception models are trained directly from simulation, allowing the system to identify various parts based on CAD information. We demonstrate the system with a workcell of two robots to assemble interlocking 3D part designs. We first build and tune the assembly process in simulation, verifying the generated recipe. Finally, the real robotic workcell assembles the design using the same behavior.
Scalable Fiducial Tag Localization on a 3D Prior Map via Graph-Theoretic Global Tag-Map Registration
Venue: Conference IROS 2022
Year: 2022

This paper presents an accurate and scalable method for fiducial tag localization on a 3D prior environmental map. The proposed method comprises three steps: 1) visual odometry-based landmark SLAM for estimating the relative poses between fiducial tags, 2) geometrical matching-based global tag-map registration via maximum clique finding, and 3) tag pose refinement based on direct camera-map alignment with normalized information distance. Through simulation-based evaluations, the proposed method achieved a 98 \% global tag-map registration success rate and an average tag pose estimation accuracy of a few centimeters. Experimental results in a real environment demonstrated that it enables to localize over 110 fiducial tags placed in an environment in 25 minutes for data recording and post-processing.
EVOPS Benchmark: Evaluation of Plane Segmentation from RGBD and LiDAR Data
Venue: Conference IROS 2022
Year: 2022

This paper provides the EVOPS dataset for plane segmentation from 3D data, both from RGBD images and LiDAR point clouds. We have designed two annotation methodologies (RGBD and LiDAR) running on well-known and widely-used datasets for SLAM evaluation and we have provided a complete set of benchmarking tools including point, planes and segmentation metrics. The data includes a total number of 10k RGBD and 7K LiDAR frames over different selected scenes which consist of high quality segmented planes. The experiments report quality of SOTA methods for RGBD plane segmentation on our annotated data. We also have provided learnable baseline for plane segmentation in LiDAR point clouds. All labeled data and benchmark tools used have been made publicly available at https://evops.netlify.app/.
Virtual Reality Simulator for Fetoscopic Spina Bifida Repair Surgery
Venue: Conference IROS 2022
Year: 2022

Spina Bifida (SB) is a birth defect developed during the early stage of pregnancy in which there is incomplete closing of the spine around the spinal cord. The growing interest in fetoscopic Spina-Bifida repair, which is performed in fetuses who are still in the pregnant uterus, prompts the need for appropriate training. The learning curve for such procedures is steep and requires excellent procedural skills. Computer-based virtual reality (VR) simulation systems offer a safe, cost-effective, and configurable training environment free from ethical and patient safety issues. However, to the best of our knowledge, there are currently no commercial or experimental VR training simulation systems available for fetoscopic SB-repair procedures. In this paper, we propose a novel VR simulator for core manual skills training for SB-repair. An initial simulation realism validation study was carried out by obtaining subjective feedback (face and content validity) from 14 clinicians. The overall simulation realism was on average marked 4.07 on a 5-point Likert scale (1 - very unrealistic, 5 - very realistic). Its usefulness as a training tool for SB-repair as well as in learning fundamental laparoscopic skills was marked 4.63 and 4.80, respectively. These results indicate that VR simulation of fetoscopic procedures may contribute to surgical training without putting fetuses and their mothers at risk. It could also facilitate wider adaptation of fetoscopic procedures in place of much more invasive open fetal surgeries.
Conflict-Based Search for Multi-Robot Motion Planning with Kinodynamic Constraints
Venue: Conference IROS 2022
Year: 2022

Multi-robot motion planning (MRMP) is the fundamental problem of finding non-colliding trajectories for multiple robots acting in an environment, under kinodynamic constraints. Due to its complexity, existing algorithms either utilize simplifying assumptions or are incomplete. This work introduces kinodynamic conflict-based search (K-CBS), a decentralized (decoupled) MRMP algorithm that is general, scalable, and probabilistically complete. The algorithm takes inspiration from successful solutions to the discrete analogue of MRMP over finite graphs, known as multi-agent path finding (MAPF). Specifically, we adapt ideas from conflict-based search (CBS) - a popular decentralized MAPF algorithm - to the MRMP setting. The novelty in this adaptation is that we work directly in the continuous domain, without the need for discretization. In particular, the kinodynamic constraints are treated natively. K-CBS plans for each robot individually using a low-level planner and and grows a conflict tree to resolve collisions between robots by defining constraints for individual robots. The low-level planner can be any sampling-based, tree-search algorithm for kinodynamic robots, thus lifting existing planners for single robots to the multi-robot settings. We show that K-CBS inherits the (probabilistic) completeness of the low-level planner. We illustrate the generality and performance of K-CBS in several case studies and benchmarks.
Gravity-constrained point cloud registration
Venue: Conference IROS 2022
Year: 2022

Visual and lidar Simultaneous Localization and Mapping (SLAM) algorithms benefit from the Inertial Measurement Unit (IMU) modality. The high-rate inertial data complement the other lower-rate modalities. Moreover, in the absence of constant acceleration, the gravity vector makes two attitude angles out of three observable in the global coordinate frame. In visual odometry, this is already being used to reduce the 6-Degrees Of Freedom (DOF) pose estimation problem to 4-DOF. In lidar SLAM, the gravity measurements are often used as a penalty in the back-end global map optimization to prevent map deformations. In this work, we propose an Iterative Closest Point (ICP)-based front-end which exploits the observable DOF and provides pose estimates aligned with the gravity vector. We believe that this front-end has the potential to support the loop closure identification, thus speeding up convergences of global map optimizations. The presented approach has been extensively tested in large-scale outdoor environments as well as in the Subterranean Challenge organized by Defense Advanced Research Projects Agency (DARPA). We show that it can reduce the localization drift by 30% when compared to the standard 6-DOF ICP. Moreover, the code is readily available to the community as a part of the libpointmatcher library.
Adapting Rapid Motor Adaptation for Bipedal Robots
Venue: Conference IROS 2022
Year: 2022

Recent advances in legged locomotion have enabled quadrupeds to walk on challenging terrains. However, bipedal robots are inherently more unstable and hence it's harder to design walking controllers for them. In this work, we leverage recent advances in rapid adaptation for locomotion control, and extend them to work on bipedal robots. Similar to existing works, we start with a base policy which produces actions while taking as input an estimated extrinsics vector from an adaptation module. This extrinsics vector contains information about the environment and enables the walking controller to rapidly adapt online. However, the extrinsics estimator could be imperfect, which might lead to poor performance of the base policy which expects a perfect estimator. In this paper, we propose A-RMA (Adapting RMA), which additionally adapts the base policy for the imperfect extrinsics estimator by finetuning it using model-free RL. We demonstrate that A-RMA outperforms a number of RL-based baseline controllers and model-based controllers in simulation, and show zero-shot deployment of a single A-RMA policy to enable a bipedal robot, Cassie, to walk in a variety of different scenarios in the real world beyond what it has seen during training. Videos and results at https://ashish-kmr.github.io/a-rma/
Contact-Implicit Trajectory Optimization with Hydroelastic Contact and iLQR
Venue: Conference IROS 2022
Year: 2022

Contact-implicit trajectory optimization offers an appealing method of automatically generating complex and contact-rich behaviors for robot manipulation and locomotion. The scalability of such techniques has been limited, however, by the challenge of ensuring both numerical reliability and physical realism. In this paper, we present preliminary results suggesting that the Iterative Linear Quadratic Regulator (iLQR) algorithm together with the recently proposed pressure-field-based hydroelastic contact model enables reliable and physically realistic trajectory optimization through contact. We use this approach to synthesize contact-rich behaviors like quadruped locomotion and whole-arm manipulation. Furthermore, open-loop playback on a Kinova Gen3 robot arm demonstrates the physical accuracy of the whole-arm manipulation trajectories. Code is available at https://bit.ly/ilqr_hc and videos can be found at https://youtu.be/IqxJKbM8_ms.
When Geometry is not Enough: Using Reflector Markers in Lidar SLAM
Venue: Conference IROS 2022
Year: 2022

Lidar-based SLAM systems perform well in a wide range of circumstances by relying on the geometry of the environment. However, even mature and reliable approaches struggle when the environment contains structureless areas such as long hallways. To allow the use of lidar-based SLAM in such environments, we propose to add reflector markers in specific locations that would otherwise be difficult. We present an algorithm to reliably detect these markers and two approaches to fuse the detected markers with geometry-based scan matching. The performance of the proposed methods is demonstrated on real-world datasets from several industrial environments.
Towards edible drones for rescue missions: design and flight of nutritional wings
Venue: Conference IROS 2022
Year: 2022

Drones have shown to be useful aerial vehicles for unmanned transport missions such as food and medical supply delivery. This can be leveraged to deliver life-saving nutrition and medicine for people in emergency situations. However, commercial drones can generally only carry 10 % - 30 % of their own mass as payload, which limits the amount of food delivery in a single flight. One novel solution to noticeably increase the food-carrying ratio of a drone, is recreating some structures of a drone, such as the wings, with edible materials. We thus propose a drone, which is no longer only a food transporting aircraft, but itself is partially edible, increasing its food-carrying mass ratio to 50 %, owing to its edible wings. Furthermore, should the edible drone be left behind in the environment after performing its task in an emergency situation, it will be more biodegradable than its non-edible counterpart, leaving less waste in the environment. Here we describe the choice of materials and scalable design of edible wings, and validate the method in a flight-capable prototype that can provide 300 kcal and carry a payload of 80 g of water.
NMPC-LBF: Nonlinear MPC with Learned Barrier Function for Decentralized Safe Navigation of Multiple Robots in Unknown Environments
Venue: Conference IROS 2022
Year: 2022

In this paper, we present a decentralized control approach based on a Nonlinear Model Predictive Control (NMPC) method that employs barrier certificates for safe navigation of multiple nonholonomic wheeled mobile robots in unknown environments with static and/or dynamic obstacles. This method incorporates a Learned Barrier Function (LBF) into the NMPC design in order to guarantee safe robot navigation, i.e., prevent robot collisions with other robots and the obstacles. We refer to our proposed control approach as NMPC-LBF. Since each robot does not have a priori knowledge about the obstacles and other robots, we use a Deep Neural Network (DeepNN) running in real-time on each robot to learn the Barrier Function (BF) only from the robot's LiDAR and odometry measurements. The DeepNN is trained to learn the BF that separates safe and unsafe regions. We implemented our proposed method on simulated and actual Turtlebot3 Burger robot(s) in different scenarios. The implementation results show the effectiveness of the NMPC-LBF method at ensuring safe navigation of the robots.