Reviewerly/exHarmony huggingface dataset:
Intelligent Helmet Detection System based on the Improved YOLOv5

In the train maintenance workshop, the standardized wearing of helmets by maintenance personnel can effectively avoid a larger part of safety accidents, so it is extremely significant to monitor the wearing of helmets by overhaul personnel in real time. In this paper, we propose an improved YOLOv5s helmet detection algorithm based on the deep learning method, and deploy it to the edge-end devices to realize a set of intelligent monitoring system for the helmet wearing of the maintenance personnel. In this paper, two improvement measures are proposed for the YOLOv5s algorithm. Firstly, for the problem of helmet misidentification and missed identification, the convolutional block attention module (CBAM) is incorporated in three different positions of the Backbone part of YOLOv5s to enhance the extraction capability of Backbone for important features and improve the recognition accuracy of the model for helmets; secondly, since workers’ head areas are often obscured by each other in real scenes, replacing GIoU Loss with CIoU Loss in YOLOv5s can effectively improve the convergence speed of the network training and the regression localization accuracy of the network. The comparison experimental results fully prove the effectiveness of the proposed method.
Global Relation Auxiliary Graph Attention Network for Knowledge Graph Completion

Knowledge Graph Completion (KGC) has been an active research topic in recent years, which is the task of predicting missing links based on known triples of knowledge graphs. Some recent work has shown that graph neural networks (GNNs) using graph structure can perform well on KGC. These models learn information from entities and relations within the subject’s neighborhood and update the representation through a message passing mechanism. However, existing GNN models rarely include the modeling of relational information, and they tend to represent and learn nodes through complex networks, ignoring the underlying semantic information between relations. In this work, we propose a global relationship-assisted graph attention network. It not only models entities but also builds directed graph structures and updates the representation of relations between different relations. Specifically, the strongly correlated neighboring relations are identified for aggregation by an attention function based on the information and spatial domains. We also use a learnable nonlinear function to activate the attention values, allowing the model to adaptively aggregate information. Experiments show that GRA-GAT has a very advanced performance on link prediction tasks.
Scalable Gradient Ascent for Controllers in Constrained POMDPs

This paper presents a novel gradient ascent al-gorithm and nonlinear programming algorithm for finite state controller policies in constrained partially observable Markov decision processes (CPOMDPs). A key component of the gradient ascent algorithm is a constraint projection to ensure constraints are satisfied. Both an optimal and an approximate projection are formally defined. A theoretical analysis of the algorithm and its projections is presented, formally proving aspects of projection correctness and algorithm convergence. Experiments evaluate the baseline and novel algorithms, as well as both constraint projections, on seven CPOMDP benchmark domains. The proposed novel algorithm is demonstrated on an actual robot performing a navigation task in a real household environment.
Depth-SIMS: Semi-Parametric Image and Depth Synthesis

In this paper we present a compositing image synthesis method that generates RGB canvases with well aligned segmentation maps and sparse depth maps, coupled with an in-painting network that transforms the RGB canvases into high quality RGB images and the sparse depth maps into pixel-wise dense depth maps. We benchmark our method in terms of structural alignment and image quality, showing an increase in mIoU over SOTA by 3.7 percentage points and a highly competitive FID. Furthermore, we analyse the quality of the generated data as training data for semantic segmentation and depth completion, and show that our approach is more suited for this purpose than other methods.
Improved Task Planning through Failure Anticipation in Human-Robot Collaboration

Human-Robot Collaboration (HRC) has become a major trend in robotics in recent years with the idea of combining the strengths from both humans and robots.In order to share the work to be done, many task planning approaches have been implemented.However, they don't fully satisfy the required adaptability in human-robot collaborative tasks, with most approaches not considering neither the state of the human partner nor the possibility of adapting the collaborative plan during execution or even anticipating failures.In this paper, we present a planning system for human-robot collaborative plans that takes into account the agents' states and deals with unforeseen human behaviour, by replanning in anticipation when the human state changes to prevent action failure.The human state is defined in terms of capacity, knowledge and motivation.The system has been implemented in a standardised environment using the Planning Domain Definition Language (PDDL) and the modular ROSPlan framework, and we have validated the approach in multiple simulation settings.Our results show that using the human model fosters an appropriate task allocation while allowing failure anticipation, replanning in time to prevent it.
Online Assistance Control of a Pneumatic Gait Assistive Suit Using Physical Reservoir Computing Exploiting Air Dynamics

Wearable gait assistive devices have been im-proved by soft robotics in terms of safety and physical burden of the wearer. Currently, electrical sensors and computers around the wearer are bottlenecks in enhancing the wearer's activities. In this paper, we present a wearable gait assistive system using soft pneumatic system for all the actuation, sensing, and computation. Pneumatic artifical muscles (PAMs) on the thighs were used to generate assistance force, and thin PAMs on the whole legs were used for sensing the wearer's motion. All thin PAMs were connected to each other by tubes, and the pressure response in the tubes were exploited to compute the wearer's motion in manner of physical reservoir computing. The left thigh angular velocity estimated from the reservoir response was used for control the PAMs for actuation. Our experiment with the use of gait assistance showed that the system worked correctly. This paper shows that the pneumatic analog computation system for sensing soft body can help the functionality of soft wearable assistive devices.
OpenSceneVLAD: Appearance Invariant, Open Set Scene Classification

Scene classification is a well-established area of computer vision research that aims to classify a scene image into pre-defined categories such as playground, beach and airport. Recent work has focused on increasing the variety of pre-defined categories for classification, but so far failed to consider two major challenges: changes in scene appearance due to lighting and open set classification (the ability to classify unknown scene data as not belonging to the trained classes). Our first contribution, SceneVLAD, fuses scene classification and visual place recognition CNNs for appearance invariant scene classification that outperforms state-of-the-art scene classification by a mean F1 score of up to 0.1. Our second contribution, OpenSceneVLAD, extends the first to an open set classification scenario using intra-class splitting to achieve a mean increase in F1 scores of up to 0.06 compared to using state-of-the-art openmax layer. We achieve these results on three scene class datasets extracted from large scale outdoor visual localisation datasets, one of which we collected ourselves.
SEHLNet: Separate Estimation of High- and Low-Frequency components for Depth Completion

Depth completion refers to inferring the dense depth map from a sparse depth map with or without corre-sponding color image. Numerous neural networks have been proposed to accomplish this task. However, insufficient uti-lization of heteromorphic data and the fact that predicted dense depth prefers a sparse depth enormously damage the performance of approaches. To reduce data preference and fully utilize two modalities, this paper proposes a novel network that predicts high- and low-frequency components of dense depth separately. Specifically, the framework consists of a Low-Frequency(LF) branch and a High-Frequency(HF) branch. In the LF branch, we recover the low-frequency depth component from sparse depth through an Adaptive Graph-Generate Graph Attention Network, which can be seen as a low-pass filter. In the HF branch, we model the high-frequency component, e.g. boundaries, as residuals to mitigate the impact of data preferences. Moreover, in this branch, we propose an Attention-based Self-Fusion mechanism to efficiently fuse multi-scale features extracted from the sparse depth and color image. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on the KITTI benchmark and ranks 1st in root mean squared error among other published approaches.
JST: Joint Self-training for Unsupervised Domain Adaptation on 2D&amp;3D Object Detection

2D&3D object detection always suffers from a dramatic performance drop when transferring the model trained in the source domain to the target domain due to various domain shifts. In this paper, we propose a Joint Self-Training (JST) framework to improve 2D image and 3D point cloud detectors with aligned outputs simultaneously during the transferring. The proposed framework contains three novelties to overcome object biases and unstable self-training processes: 1) an anchor scaling scheme is developed to efficiently eliminate the object size biases without any modification on point clouds; 2) a 2D&3D bounding box alignment method is proposed to generate high-quality pseudo labels for the self-training process; 3) a model smoothing based training strategy is developed to reduce the training oscillation properly. Experiment results show that the proposed approach improves the performance of 2D and 3D detectors in the target domain simultaneously; especially the superior accuracy of 3D detection can be achieved on benchmark datasets over the state-of-the-art methods.
Failure is an option: Task and Motion Planning with Failing Executions

Future robotic deployments will require robots to be able to repeatedly solve a variety of tasks in application domains. Task and motion planning addresses complex robotic problems that combine discrete reasoning over states and actions and geometric interactions during action executions. Moving beyond deterministic settings, stochastic actions can be handled by modeling the problem as a Markov Decision Process. The underlying probabilities however are typically hard to model since failures might be caused by hardware imperfections, sensing noise, or physical interactions. We pro-pose a framework to address a task and motion planning setting where actions can fail during execution. To achieve a task goal actions need to be computed and executed despite failures. The robot has to infer which actions are robust and for each new problem effectively choose a solution that reduces expected execution failures. The key idea is to continually recover and refine the underlying beliefs associated with actions across multiple different problems in the domain. Our proposed method can find solutions that reduce the expected number of discrete, executed actions. Results in physics-based simulation indicate that our method outperforms baseline replanning strategies to deal with failing executions.
Immersive Virtual Walking System Using an Avatar Robot

The ongoing COVID-19 pandemic has enforced governments across the world to impose social restrictions on the movement of people and confined them to their homes to avoid the spread of the disease. This not only forbids them from leaving their homes but also greatly reduces their physical activities. This situation has brought attention to virtual technologies such as virtual tours or telepresence robots. While these technologies allow people to remotely participate in activities, it does not address the problem of reduction in physical activities due to the pandemic. In this paper, we propose a telepresence robotic system driven by the user's gait to provide an immersive virtual walking experience in remote locations. To this end, we developed a control interface consisting of an automated treadmill that adjusts its speed to the user's pace automatically. This interface is used to control an avatar robot that sends a 360-degree live image back to the user for visual feedback. We conducted an evaluation experiment to compare the experience using the proposed system in two different conditions to that of regular walking. The results indicated that the proposed system gives an immersive and realistic virtual walking experience while demanding physical effort from the user.
Unsupervised Learning of Terrain Representations for Haptic Monte Carlo Localization

Haptic sensing has recently been used effectively for legged robot localization in extreme scenarios where cam-eras and LiDAR might fail, such as dusty mines and foggy sewers. However, existing haptic sensing mainly relies on supervised classification, with training and evaluation executed over explicit terrain classes. Defining classes is a significant limitation to real-world applications, where prior labelling and handcrafted classes are often impractical. This paper proposes a novel haptic localization system based on a fully unsupervised terrain representation learned solely from the force/torque sensors located in the quadruped robot's feet. Instead of using the detected terrain class for localization, we propose an improved autoencoder architecture to generate a sparse map of encodings on the first run and to localize against this sparse map during subsequent runs. We compare our approach to a haptic localization system based on supervised terrain classification, showing that the unsupervised method has comparable or better performance than the supervised one for the same trajectories while clearly outperforming the proprioceptive odometry estimator available on the robot. Therefore, the proposed approach is well-suited for a routine maintenance application, increasing the platform's robustness.
Rethinking LiDAR Object Detection in adverse weather conditions

LiDAR sensors are becoming crucial for achieving higher levels of autonomy. With the current sensor technology, LiDAR sensors are still susceptible to erroneous measurements in adverse weather conditions due to weather artifacts observed in the point cloud data. In this work, we analyze the performance of deep learning LiDAR object detectors in adverse weather conditions. We study the under-researched albeit crucial aspect of deep learning - the size and the content of the training data, as we believe that efforts in data curation bring more benefit than often preferred complex algorithmic enhancements. We argue that with sufficient data, learning-based object detectors are inherently capable of distinguishing between points from a true object and weather-induced artifacts in an end-to-end manner, thus, making explicit, handcrafted and computationally expensive point cloud prefiltering steps obsolete. We corroborate our hypothesis by conducting experiments on a variety of LiDAR object detection architectures over two subsets of training data - one comprising of all weather conditions and the other one comprising of only clear conditions. Contrary to popular belief that object detectors need to be trained on data from adverse weather conditions to be performant in adverse weather, we show that training on datasets with a higher number of annotated objects, predominantly acquired in clear conditions, is sufficient to achieve almost similar or sometimes better KPIs across all weather conditions. This makes the data collection more accessible and convenient compared to adverse weather conditions that are often hardly accessible (e.g., heavy snow and fog). Finally, our proposed methodology streamlines the LiDAR perception pipeline in the direction of keeping well-known end-to-end trainable architectures, removing additional pre-processing blocks that are often handcrafted and bring an additional computational cost.
PA-AWCNN: Two-stream Parallel Attention Adaptive Weight Network for RGB-D Action Recognition

Due to overly relying on appearance information or adopting direct static feature fusion, most of the existing action recognition methods based on multi-modality have poor robustness and insufficient consideration of modality differences. To address these problems, we propose a two-stream adaptive weight integration network with a three-dimensional parallel attention module, PA-AWCNN. Firstly, a three-dimensional Parallel Attention (PA) module is proposed to effectively extract features of spatial, temporal and channel dimensions and reduce the cross-dimensional interference, to achieve better robustness. Secondly, a Common Feature-driven (CFD) feature integration module is proposed to dynamically integrate appearance and depth features with adaptive weights, utilizing modality differences to redeem the lack of each feature, thereby balance the influence of both. The proposed PA-AW CNN uses the representative integrated feature generated by attention enhancement and feature integration for action recognition; it can not only get higher recognition accuracy but also improve the performance of distinguishing similar actions. Experiments illustrate that the proposed method achieves com-parable performances to state-of-the-art methods and obtains the accuracy of 92.76% and 95.65% on NTU RGB+D Dataset and SBU Kinect Interaction Dataset, respectively. The code is publicly available at: https://github.com/Luu-Yao/PA-AWCNN.
Graph Neural Network Based Relation Learning for Abnormal Perception Information Detection in Self-Driving Scenarios

Robustness and safety concerns of perception systems are of great importance for autonomous vehicle navigation applications. Recent researches demonstrate that the surrounding dynamic object detection results of current perception systems can be easily interfered or attacked to mislead the navigation performance of the victim vehicle. In this paper, we develop a GNN based relation learning network to detect the abnormal information in the vehicle perception results, by investigating the relations among the surrounding dynamic objects and also the overall scenario information. Our underlying logic is that the motion of each surrounding object is also affected by its neighbors as well as the whole traffic scenario information, so there should exist a certain amount of consistency among those agents. Learning their spatiotemporal relations provides critical information for detecting the abnormal perception information. Experimental results on the standard CARLA simulator demonstrate our effectiveness in various scenarios and scalability to unseen cases.
Exploiting Abstract Symmetries in Reinforcement Learning for Complex Environments

Reinforcement Learning is rapidly establishing itself as the foremost choice for optimization of sequential autonomous decision-making problems. Encumbered by its sample inefficiency, the extension of the field to large state space and dynamic environments remains an open problem. We present a novel concept that exploits abstract spatial symmetry in complex environments for extending the skills of naïvely trained agents in local abstractions of the environment. The concept of EASE (Exploitation of Abstract Symmetry of Environments), when incorporated, improves the sample efficiency of traditional reinforcement learning algorithms. The presented work exemplifies the concept of EASE by presenting three distinct settings; EASE with heuristics-based planning, EASE with learning from demonstrations and EASE with state-space abstraction and proposes a novel algorithm for each setting.
Deep Reinforcement Learning for Next-Best-View Planning in Agricultural Applications

Automated agricultural applications, i.e., fruit picking require spatial information about crops and, especially, their fruits. In this paper, we present a novel deep reinforcement learning (DRL) approach to determine the next best view for automatic exploration of 3D environments with a robotic arm equipped with an RGB-D camera. We process the obtained images into an octree with labeled regions of interest (ROIs), i.e., fruits. We use this octree to generate 3D observation maps that serve as encoded input to the DRL network. We hereby do not only rely on known information about the environment, but explicitly also represent information about the unknown space to force exploration. Our network takes as input the encoded 3D observation map and the temporal sequence of camera view pose changes, and outputs the most promising camera movement direction. Our experimental results show an improved ROI targeted exploration performance resulting from our learned network in comparison to a state-of-the-art method.
Learning-Guided Exploration for Efficient Sampling-Based Motion Planning in High Dimensions

Optimal motion planning is a long-studied problem with a wide range of applications in robotics, from grasping to navigation. While sampling-based motion planning methods have made solving such problems significantly more feasible, these methods still often struggle in high-dimensional spaces wherein exploration is computationally costly. In this paper, we propose a new motion planning algorithm that reduces the computational burden of the exploration process. The proposed algorithm utilizes a guidance policy acquired offline through model-free reinforcement learning. The guidance policy is used to bias the exploration process in motion planning and to guide it toward promising regions of the state space. Moreover, we show that the gradients of the corresponding learned value function can be used to locally fine-tune the sampled states. We empirically demonstrate that the proposed approach can significantly reduce planning time and improve success rate and path quality.
Graph Filtering Over Expanding Graphs

Our capacity to learn representations from data is related to our ability to design filters that can leverage their coupling with the underlying domain. Graph filters are one such tool for network data and have been used in a myriad of applications. But graph filters work only with a fixed number of nodes despite the expanding nature of practical networks. Learning filters in this setting is challenging not only because of the increased dimensions but also because the connectivity is known only up to an attachment model. We propose a filter learning scheme for data over expanding graphs by relying only on such a model. By characterizing the filter stochastically, we develop an empirical risk minimization framework inspired by multi-kernel learning to balance the information inflow and outflow at the incoming nodes. We particularize the approach for denoising and semi-supervised learning (SSL) over expanding graphs and show near-optimal performance compared with baselines relying on the exact topology. For SSL, the proposed scheme uses the incoming node information to improve the task on the existing ones. These findings lay the foundation for learning representations over expanding graphs by relying only on the stochastic connectivity model.