OpenRaiser/Intern-Atlas huggingface dataset:
Harmonious Sampling for Mobile Manipulation Planning
Venue: Conference IROS 2019
Year: 2019

Mobile manipulation planning commonly adopts a decoupled approach that performs planning separately on the base and the manipulator. While this approach is fast, it can generate sub-optimal paths. Another direction is a coupled approach jointly adjusting the base and manipulator in a high-dimensional configuration space. This coupled approach addresses sub-optimality and incompleteness of the decoupled approach, but has not been widely used due to its excessive computational overhead. Given this trade-off space, we present a simple, yet effective mobile manipulation sampling method, harmonious sampling, to perform the coupled approach mainly in difficult regions, where we need to simultaneously maneuver the base and the manipulator. Our method identifies such difficult regions through a low-dimensional base space by utilizing a reachability map given the target end-effector pose and narrow passage detected by generalized Voronoi diagram. For the rest of simple regions, we perform sampling mainly on the base configurations with a predefined joint configuration, accelerating the planning process. We compare our method with the decoupled and coupled approaches in six different problems with varying difficulty. Our method shows meaningful improvements experimentally in terms of time to find an initial solution (up to 5.6 times faster) and final solution cost (up to 17% lower) over the decoupled approach, especially in difficult scenes with narrow space. We also demonstrate these benefits with a real, mobile Hubo robot.
Riverine Coverage with an Autonomous Surface Vehicle over Known Environments
Venue: Conference IROS 2019
Year: 2019

Environmental monitoring and surveying operations on rivers currently are performed primarily with manually-operated boats. In this domain, autonomous coverage of areas is of vital importance, for improving both the quality and the efficiency of coverage. This paper leverages human expertise in river exploration and data collection strategies to automate and optimize these processes using autonomous surface vehicles(ASVs). In particular, three deterministic algorithms for both partial and complete coverage of a river segment are proposed,providing varying path length, coverage density, and turning patterns. These strategies resulted in increases in accuracy and efficiency compared to human performance.The proposed methods were extensively tested in simulation using maps of real rivers of different shapes and sizes. In addition, to verify their performance in real world operations, the algorithms were deployed successfully on several parts of the Congaree River in South Carolina, USA, resulting in total of more than 35km of coverage trajectories in the field.
Look Further to Recognize Better: Learning Shared Topics and Category-Specific Dictionaries for Open-Ended 3D Object Recognition
Venue: Conference IROS 2019
Year: 2019

Service robots are expected to operate effectively in human-centric environments for long periods of time. In such realistic scenarios, fine-grained object categorization is as important as basic-level object categorization. We tackle this problem by proposing an open-ended object recognition approach which concurrently learns both the object categories and the local features for encoding objects. In this work, each object is represented using a set of general latent visual topics and category-specific dictionaries. The general topics encode the common patterns of all categories, while the category-specific dictionary describes the content of each category in details. The proposed approach discovers both sets of general and specific representations in an unsupervised fashion and updates them incrementally using new object views. Experimental results show that our approach yields significant improvements over the previous state-of-the-art approaches concerning scalability and object classification performance. Moreover, our approach demonstrates the capability of learning from very few training examples in a real-world setting. Regarding computation time, the best result was obtained with a Bag-of-Words method followed by a variant of the Latent Dirichlet Allocation approach.
A Robust Biped Locomotion Based on Linear-Quadratic-Gaussian Controller and Divergent Component of Motion
Venue: Conference IROS 2019
Year: 2019

Generating robust locomotion for a humanoid robot in the presence of disturbances is difficult because of its high number of degrees of freedom and its unstable nature. In this paper, we used the concept of Divergent Component of Motion~(DCM) and propose an optimal closed-loop controller based on Linear-Quadratic-Gaussian to generate a robust and stable walking for humanoid robots. The biped robot dynamics has been approximated using the Linear Inverted Pendulum Model~(LIPM). Moreover, we propose a controller to adjust the landing location of the swing leg to increase the withstanding level of the robot against a severe external push. The performance and also the robustness of the proposed controller is analyzed and verified by performing a set of simulations using~\mbox{MATLAB}. The simulation results showed that the proposed controller is capable of providing a robust walking even in the presence of disturbances and in challenging situations.
Towards Generalizing Sensorimotor Control Across Weather Conditions
Venue: Conference IROS 2019
Year: 2019

The ability of deep learning models to generalize well across different scenarios depends primarily on the quality and quantity of annotated data. Labeling large amounts of data for all possible scenarios that a model may encounter would not be feasible; if even possible. We propose a framework to deal with limited labeled training data and demonstrate it on the application of vision-based vehicle control. We show how limited steering angle data available for only one condition can be transferred to multiple different weather scenarios. This is done by leveraging unlabeled images in a teacher-student learning paradigm complemented with an image-to-image translation network. The translation network transfers the images to a new domain, whereas the teacher provides soft supervised targets to train the student on this domain. Furthermore, we demonstrate how utilization of auxiliary networks can reduce the size of a model at inference time, without affecting the accuracy. The experiments show that our approach generalizes well across multiple different weather conditions using only ground truth labels from one domain.
Learning Safe Unlabeled Multi-Robot Planning with Motion Constraints
Venue: Conference IROS 2019
Year: 2019

In this paper, we present a learning approach to goal assignment and trajectory planning for unlabeled robots operating in 2D, obstacle-filled workspaces. More specifically, we tackle the unlabeled multi-robot motion planning problem with motion constraints as a multi-agent reinforcement learning problem with some sparse global reward. In contrast with previous works, which formulate an entirely new hand-crafted optimization cost or trajectory generation algorithm for a different robot dynamic model, our framework is a general approach that is applicable to arbitrary robot models. Further, by using the velocity obstacle, we devise a smooth projection that guarantees collision free trajectories for all robots with respect to their neighbors and obstacles. The efficacy of our algorithm is demonstrated through varied simulations.
Improved Planetary Rover Inertial Navigation and Wheel Odometry Performance through Periodic Use of Zero-Type Constraints
Venue: Conference IROS 2019
Year: 2019

We present an approach to enhance wheeled planetary rover dead-reckoning localization performance by leveraging the use of zero-type constraint equations in the navigation filter. Without external aiding, inertial navigation solutions inherently exhibit cubic error growth. Furthermore, for planetary rovers that are traversing diverse types of terrain, wheel odometry is often unreliable for use in localization, due to wheel slippage. For current Mars rovers, computer vision-based approaches are generally used whenever there is a high possibility of positioning error; however, these strategies require additional computational power, energy resources, and significantly slow down the rover traverse speed. To this end, we propose a navigation approach that compensates for the high likelihood of odometry errors by providing a reliable navigation solution that leverages non-holonomic vehicle constraints as well as state-aware pseudo-measurements (e.g., zero velocity and zero angular rate) updates during periodic stops. By using this, computationally expensive visual-based corrections could be performed less often. Experimental tests that compare against GPS-based localization are used to demonstrate the accuracy of the proposed approach. The source code, post-processing scripts, and example datasets associated with the paper are published in a public repository.
A Penetration Metric for Deforming Tetrahedra using Object Norm
Venue: Conference IROS 2019
Year: 2019

In this paper, we propose a novel penetration metric, called deformable penetration depth PDd, to define a measure of inter-penetration between two linearly deforming tetrahedra using the object norm. First of all, we show that a distance metric for a tetrahedron deforming between two configurations can be found in closed form based on object norm. Then, we show that the PDd between an intersecting pair of static and deforming tetrahedra can be found by solving a quadratic programming (QP) problem in terms of the distance metric with non-penetration constraints. We also show that the PDd between two, intersected, deforming tetrahedra can be found by solving a similar QP problem under some assumption on penetrating directions, and it can be also accelerated by an order of magnitude using pre-calculated penetration direction. We have implemented our algorithm on a standard PC platform using an off-the-shelf QP optimizer, and experimentally show that both the static/deformable and deformable/deformable tetrahedra cases can be solvable in from a few to tens of milliseconds. Finally, we demonstrate that our penetration metric is three-times smaller (or tighter) than the classical, rigid penetration depth metric in our experiments.
DISCOMAN: Dataset of Indoor SCenes for Odometry, Mapping And Navigation
Venue: Conference IROS 2019
Year: 2019

We present a novel dataset for training and benchmarking semantic SLAM methods. The dataset consists of 200 long sequences, each one containing 3000-5000 data frames. We generate the sequences using realistic home layouts. For that we sample trajectories that simulate motions of a simple home robot, and then render the frames along the trajectories. Each data frame contains a) RGB images generated using physically-based rendering, b) simulated depth measurements, c) simulated IMU readings and d) ground truth occupancy grid of a house. Our dataset serves a wider range of purposes compared to existing datasets and is the first large-scale benchmark focused on the mapping component of SLAM. The dataset is split into train/validation/test parts sampled from different sets of virtual houses. We present benchmarking results forboth classical geometry-based and recent learning-based SLAM algorithms, a baseline mapping method, semantic segmentation and panoptic segmentation.
Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking
Venue: Conference IROS 2019
Year: 2019

In this paper, we introduce a new public dataset for 6D object pose estimation and instance segmentation for industrial bin-picking. The dataset comprises both synthetic and real-world scenes. For both, point clouds, depth images, and annotations comprising the 6D pose (position and orientation), a visibility score, and a segmentation mask for each object are provided. Along with the raw data, a method for precisely annotating real-world scenes is proposed. To the best of our knowledge, this is the first public dataset for 6D object pose estimation and instance segmentation for bin-picking containing sufficiently annotated data for learning-based approaches. Furthermore, it is one of the largest public datasets for object pose estimation in general. The dataset is publicly available at http://www.bin-picking.ai/en/dataset.html.
Aerial Robot Control in Close Proximity to Ceiling: A Force Estimation-based Nonlinear MPC
Venue: Conference IROS 2019
Year: 2019

Being motivated by ceiling inspection applications via unmanned aerial vehicles (UAVs) which require close proximity flight to surfaces, a systematic control approach enabling safe and accurate close proximity flight is proposed in this work. There are two main challenges for close proximity flights: (i) the trust characteristics varies drastically for the different distance from the ceiling which results in a complex nonlinear dynamics; (ii) the system needs to consider physical and environmental constraints to safely fly in close proximity. To address these challenges, a novel framework consisting of a constrained optimization-based force estimation and an optimization-based nonlinear controller is proposed. Experimental results illustrate that the performance of the proposed control approach can stabilize UAV down to 1 cm distance to the ceiling. Furthermore, we report that the UAV consumes up to 12.5% less power when it is operated 1 cm distance to ceiling, which is promising potential for more battery-efficient inspection flights.
Non-myopic Planetary Exploration Combining In Situ and Remote Measurements
Venue: Conference IROS 2019
Year: 2019

Remote sensing can provide crucial information for planetary rovers. However, they must validate these orbital observations with in situ measurements. Typically, this involves validating hyperspectral data using a spectrometer on-board the field robot. In order to achieve this, the robot must visit sampling locations that jointly improve a model of the environment while satisfying sampling constraints. However, current planners follow sub-optimal greedy strategies that are not scalable to larger regions. We demonstrate how the problem can be effectively defined in an MDP framework and propose a planning algorithm based on Monte Carlo Tree Search, which is devoid of the common drawbacks of existing planners and also provides superior performance. We evaluate our approach using hyperspectral imagery of a well-studied geologic site in Cuprite, Nevada.
Learning to Estimate Pose and Shape of Hand-Held Objects from RGB Images
Venue: Conference IROS 2019
Year: 2019

We develop a system for modeling hand-object interactions in 3D from RGB images that show a hand which is holding a novel object from a known category. We design a Convolutional Neural Network (CNN) for Hand-held Object Pose and Shape estimation called HOPS-Net and utilize prior work to estimate the hand pose and configuration. We leverage the insight that information about the hand facilitates object pose and shape estimation by incorporating the hand into both training and inference of the object pose and shape as well as the refinement of the estimated pose. The network is trained on a large synthetic dataset of objects in interaction with a human hand. To bridge the gap between real and synthetic images, we employ an image-to-image translation model (Augmented CycleGAN) that generates realistically textured objects given a synthetic rendering. This provides a scalable way of generating annotated data for training HOPS-Net. Our quantitative experiments show that even noisy hand parameters significantly help object pose and shape estimation. The qualitative experiments show results of pose and shape estimation of objects held by a hand "in the wild".
Online Motion Planning Over Multiple Homotopy Classes with Gaussian Process Inference
Venue: Conference IROS 2019
Year: 2019

Efficient planning in dynamic and uncertain environments is a fundamental challenge in robotics. In the context of trajectory optimization, the feasibility of paths can change as the environment evolves. Therefore, it can be beneficial to reason about multiple possible paths simultaneously. We build on prior work that considers graph-based trajectories to find solutions in multiple homotopy classes concurrently. Specifically, we extend this previous work to an online setting where the unreachable (in time) part of the graph is pruned and the remaining graph is reoptimized at every time step. As the robot moves within the graph on the path that is most promising, the pruning and reoptimization allows us to retain candidate paths that may become more viable in the future as the environment changes, essentially enabling the robot to dynamically switch between numerous homotopy classes. We compare our approach against prior work without the homotopy switching capability and show improved performance across several metrics in simulation with a 2D robot in multiple dynamic environments under noisy measurements and execution.
Real-Time 6D Object Pose Estimation on CPU
Venue: Conference IROS 2019
Year: 2019

We propose a fast and accurate 6D object pose estimation from a RGB-D image. Our proposed method is template matching based and consists of three main technical components, PCOF-MOD (multimodal PCOF), balanced pose tree (BPT) and optimum memory rearrangement for a coarse-to-fine search. Our model templates on densely sampled viewpoints and PCOF-MOD which explicitly handles a certain range of 3D object pose improve the robustness against background clutters. BPT which is an efficient tree-based data structures for a large number of templates and template matching on rearranged feature maps where nearby features are linearly aligned accelerate the pose estimation. The experimental evaluation on tabletop and bin-picking dataset showed that our method achieved higher accuracy and faster speed in comparison with state-of-the-art techniques including recent CNN based approaches. Moreover, our model templates can be trained only from 3D CAD in a few minutes and the pose estimation run in near real-time (23 fps) on CPU. These features are suitable for any real applications.
Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation
Venue: Conference IROS 2019
Year: 2019

Accurately estimating the orientation of pedestrians is an important and challenging task for autonomous driving because this information is essential for tracking and predicting pedestrian behavior. This paper presents a flexible Virtual Multi-View Synthesis module that can be adopted into 3D object detection methods to improve orientation estimation. The module uses a multi-step process to acquire the fine-grained semantic information required for accurate orientation estimation. First, the scene's point cloud is densified using a structure preserving depth completion algorithm and each point is colorized using its corresponding RGB pixel. Next, virtual cameras are placed around each object in the densified point cloud to generate novel viewpoints, which preserve the object's appearance. We show that this module greatly improves the orientation estimation on the challenging pedestrian class on the KITTI benchmark. When used with the open-source 3D detector AVOD-FPN, we outperform all other published methods on the pedestrian Orientation, 3D, and Bird's Eye View benchmarks.
Pose Estimation for Omni-directional Cameras using Sinusoid Fitting
Venue: Conference IROS 2019
Year: 2019

We propose a novel pose estimation method for geometric vision of omni-directional cameras. On the basis of the regularity of the pixel movement after camera pose changes, we formulate and prove the sinusoidal relationship between pixels movement and camera motion. We use the improved Fourier-Mellin invariant (iFMI) algorithm to find the motion of pixels, which was shown to be more accurate and robust than the feature-based methods. While iFMI works only on pin-hole model images and estimates 4 parameters (x, y, yaw, scaling), our method works on panoramic images and estimates the full 6 DoF 3D transform, up to an unknown scale factor. For that we fit the motion of the pixels in the panoramic images, as determined by iFMI, to two sinusoidal functions. The offsets, amplitudes and phase-shifts of the two functions then represent the 3D rotation and translation of the camera between the two images. We perform experiments for 3D rotation, which show that our algorithm outperforms the feature-based methods in accuracy and robustness. We leave the more complex 3D translation experiments for future work.
An optimization framework for simulation and kinematic control of Constrained Collaborative Mobile Agents (CCMA) system
Venue: Conference IROS 2019
Year: 2019

We present a concept of constrained collaborative mobile agents (CCMA) system, which consists of multiple wheeled mobile agents constrained by a passive kinematic chain. This mobile robotic system is modular in nature, the passive kinematic chain can be easily replaced with different designs and morphologies for different functions and task adaptability. Depending solely on the actuation of the mobile agents, this mobile robotic system can manipulate or position an end-effector. However, the complexity of the system due to presence of several mobile agents, passivity of the kinematic chain and the nature of the constrained collaborative manipulation requires development of an optimization framework. We therefore present an optimization framework for forward simulation and kinematic control of this system. With this optimization framework, the number of deployed mobile agents, actuation schemes, the design and morphology of the passive kinematic chain can be easily changed, which reinforces the modularity and collaborative aspects of the mobile robotic system. We present results, in simulation, for spatial 4-DOF to 6-DOF CCMA system examples. Finally, we present experimental quantitative results for two different fabricated 4-DOF prototypes, which demonstrate different actuation schemes, control and collaborative manipulation of an end-effector.
LEGO: Leveraging Experience in Roadmap Generation for Sampling-Based Planning
Venue: Conference IROS 2019
Year: 2019

We consider the problem of leveraging prior experience to generate roadmaps in sampling-based motion planning. A desirable roadmap is one that is sparse, allowing for fast search, with nodes spread out at key locations such that a low-cost feasible path exists. An increasingly popular approach is to learn a distribution of nodes that would produce such a roadmap. State-of-the-art is to train a conditional variational auto-encoder (CVAE) on the prior dataset with the shortest paths as target input. While this is quite effective on many problems, we show it can fail in the face of complex obstacle configurations or mismatch between training and testing. We present an algorithm LEGO that addresses these issues by training the CVAE with target samples that satisfy two important criteria. Firstly, these samples belong only to bottleneck regions along near-optimal paths that are otherwise difficult-to-sample with a uniform sampler. Secondly, these samples are spread out across diverse regions to maximize the likelihood of a feasible path existing. We formally define these properties and prove performance guarantees for LEGO. We extensively evaluate LEGO on a range of planning problems, including robot arm planning, and report significant gains over heuristics as well as learned baselines.