Reviewerly/exHarmony huggingface dataset:
The Human Gaze Helps Robots Run Bravely and Efficiently in Crowds

In human-aware navigation, the robot tacitly games with humans, balancing safety and efficiency according to human intentions. Poor balance or bad intent recognition causes the robot to stop conservatively or advance rashly, resulting in a deadlock or even a collision respectively. To address the issue, this paper proposes an improved limit cycle for collaboratively parameterizing human intentions and planning robot motions. The human-robot interaction is modeled as a dynamic chicken game with incomplete information, where the human gaze is introduced to depict the unique characteristics of each person, allowing the robot to approach with different safety margins. Our method is tested in challenging indoor scenarios and outperforms traditional methods in both safety and efficiency. We enable robots to utilize human wisdom to solve problems that cannot be solved on their own. The robot bravely goes through oncoming crowds by getting closer to people with higher attention on it and has the foresight to stably cross in front or behind people.
Practical Visual Deep Imitation Learning via Task-Level Domain Consistency

Recent work in visual end-to-end learning for robotics has shown the promise of imitation learning across a variety of tasks. Such approaches are however expensive both because they require large amounts of real world data and rely on time-consuming real-world evaluations to identify the best model for deployment. These challenges can be mitigated by using simulation evaluations to identify high performing policies. However, this introduces the well-known “reality gap” problem, where simulator inaccuracies decorrelate performance in simulation from that of reality. In this paper, we build on top of prior work in GAN-based domain adaptation and introduce the notion of a Task Consistency Loss (TCL), a self-supervised loss that encourages sim and real alignment both at the feature and action-prediction levels. We demonstrate the effectiveness of our approach by teaching a 9-DoF mobile manipulator to perform the challenging task of latched door opening purely from visual inputs such as RGB and depth images. We achieve 69% success across twenty seen and unseen meeting rooms using only ~ 16.2 hours of teleoperated demonstrations in sim and real. To the best of our knowledge, this is the first work to tackle latched door opening from a purely end-to-end learning approach, where the task of navigation and manipulation are jointly modeled by a single neural network.
Challenges in Creating Text Summarization Models in Malayalam: A Study

Automatic text summarization (ATS) is a task where information is being condensed for easy understanding. Malayalam text summarization is in formative state due to the bottleneck of methodologies, datasets and evaluation techniques. Deep learning is a better solution to achieve the balance between output accuracy and system complexity. Through this literature, we are probing into the fundamental aspects of building Malayalam text summarization models so as to highlight prevailing mechanisms, challenges, and future scope in the field. A comprehensive understanding of Malayalam grammatical structure and language-specific feature study has been carried out to achieve the goal. Afterward, available preprocessing tools and their deficiencies were discussed followed by prevailing text summarization mechanisms in Malayalam. Meanwhile, different datasets and evaluation strategies were inquired in order to demarcate their incapacities. This survey would give an overall insight for designing an ATS system in Malayalam, and we hope that the fellow researchers in this area would make use of this road map for literature works.
A Benchmark for Multi-Robot Planning in Realistic, Complex and Cluttered Environments

Several successful approaches exist for solving the complex problem of multi-robot planning and coordination. Due to the lack of adequate benchmarking tools, comparing these approaches and judging their suitability for use in realistic scenarios is currently difficult. Therefore, we propose an open-source benchmark suite that aims to close this gap. Unlike existing benchmarks, our approach uses full-stack multi-robot navigation systems in realistic 3D simulated environments from the intralogistic and household domains. Using the open-source frameworks ROS 2, Gazebo and RMF allows the user to add other robot platforms easily. The framework provides easy-to-use abstractions, typical metrics and interfaces to several established planning libraries for multi-robot systems. With all these features, our framework successfully aids practitioners and researchers in comparing multi-robot planning and coordination systems to the state of the art. Our experiments show how the proposed benchmark simplifies gaining insights on relevant close to real-life robotics use cases.
Policy-Guided Lazy Search with Feedback for Task and Motion Planning

PDDLStream solvers have recently emerged as viable solutions for Task and Motion Planning (TAMP) problems, extending PDDL to problems with continuous action spaces. Prior work has shown how PDDLStream problems can be reduced to a sequence of PDDL planning problems, which can then be solved using off-the-shelf planners. However, this approach can suffer from long runtimes. In this paper we propose LAZY, a solver for PDDLStream problems that maintains a single integrated search over action skeletons, which gets progressively more geometrically informed, as samples of possible motions are lazily drawn during motion planning. We explore how learned models of goal-directed policies and current motion sampling data can be incorporated in LAZY to adaptively guide the task planner. We show that this leads to significant speed-ups in the search for a feasible solution evaluated over unseen test environments of varying numbers of objects, goals, and initial conditions. We evaluate our TAMP approach by comparing to existing solvers for PDDLStream problems on a range of simulated 7DoF rearrangement/manipulation problems. Code can be found at https://rvl.cs.toronto.edu/learning-based-tamp.
Learning-based Relational Object Matching Across Views

Intelligent robots require object-level scene understanding to reason about possible tasks and interactions with the environment. Moreover, many perception tasks such as scene reconstruction, image retrieval, or place recognition can benefit from reasoning on the level of objects. While keypoint-based matching can yield strong results for finding correspondences for images with small to medium view point changes, for large view point changes, matching semantically on the object-level becomes advantageous. In this paper, we propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network. We demonstrate our approach in a large variety of views on realistically rendered synthetic images. Our approach compares favorably to previous state-of-the-art object-level matching approaches and achieves improved performance over a pure keypoint-based approach for large view-point changes.
Explainable Action Prediction through Self-Supervision on Scene Graphs

This work explores scene graphs as a distilled representation of high-level information for autonomous driving, applied to future driver-action prediction. Given the scarcity and strong imbalance of data samples, we propose a self-supervision pipeline to infer representative and well-separated embeddings. Key aspects are interpretability and explainability; as such, we embed in our architecture attention mechanisms that can create spatial and temporal heatmaps on the scene graphs. We evaluate our system on the ROAD dataset against a fully-supervised approach, showing the superiority of our training regime.
A General Method for Power Equipment State Classification Using Time Series Data Based on Attention Mechanism

With refinement of power equipment, the identification and classification of its operation states is quite important for the stable operation of the system. As the power equipment states are rather complicated, it is difficult to realize general classification due to the limitation of data types and model characteristics when using deep learning. In the light of this, based on the time series data of various power equipment, an equipment state classification method is proposed by combining Recurrent Neural Network (RNN), Long short-term memory (LSTM) and attention mechanism. Firstly, RNN is used as the bottom frame network, and the LSTM core network integrating attention mechanism is added to construct the complex Feature Extraction Module (FEM). Secondly, main feature extraction is implemented in the LSTM network integrated with attention mechanism. Then, the extracted features are input into the complex FEM model and trained to match the evaluation state and obtain the evaluation results. In this paper, confusion matrix, F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1micro</inf> and accuracy are used as evaluation indexes. Three cases were selected to verify the method and five groups of comparison models were set for each case. Results show that the proposed method has good accuracy in power equipment state classification and also has strong generalization ability in other data classification.
Comparison Analysis of Various Optimization Algorithms for Classification of Radar Returns from the Ionosphere

Machine learning is developing swiftly as an everexpanding field. The development of the same is occurring rapidly and has made many theoretical breakthroughs in recent times. Due to its importance as a part of machine learning, intelligent optimization algorithms are expected to become increasingly. The exponential growth of data volume and the increase in model complexity present increasing challenges for machine learning optimization strategies. Numerous initiatives have been launched to improve machine learning optimization approaches or address optimization-related problems. Future optimization and machine-learning research can be guided by a detailed evaluation and analysis of optimization strategies from a machine-learning perspective. Machine learning uses a variety of optimization strategies, which makes it easier to compare and analyze how well they function in various situations. In this study, we analyze and contrast seven well-known bio-inspired data engineering techniques and their effectiveness. We apply these techniques to the Radar Returns from the Ionosphere data-set and assess the results with a range of assessment metrics.
Computational Tradeoff in Minimum Obstacle Displacement Planning for Robot Navigation

In this paper, we look into the minimum obstacle displacement (MOD) planning problem from a mobile robot motion planning perspective. This problem finds an optimal path to goal by displacing movable obstacles when no path exists due to collision with obstacles. However this problem is computationally expensive and grows exponentially in the size of number of movable obstacles. This work looks into approximate solutions that are computationally less intensive and differ from the optimal solution by a factor of the optimal cost.
AeriaLPiPS: A Local Planner for Aerial Vehicles with Geometric Collision Checking

Real-time navigation in non-trivial environments by micro aerial vehicles (MAVs) predominantly relies on modelling the MAV with idealized geometry, such as a sphere. Simplified, conservative representations increase the likelihood of a planner failing to identify valid paths. That likelihood increases the more a robot's geometry differs from the idealized version. Few current approaches consider these situations; we are unaware of any that do so using perception space representations. This work introduces the egocan, a perception space obstacle representation using line-of-sight free space estimates, and 3D Gap, a perception space approach to gap finding for identifying goal-directed, collision-free directions of travel through 3D space. Both are integrated, with real-time considerations in mind, to define a local planner module of a hierarchical navigation system. The result is Aerial Local Planning in Perception Space (AeriaLPiPS). AeriaLPiPS is shown to be capable of safely navigating a MAV with non-idealized geometry through various environments, including those impassable by traditional real-time approaches. The open source implementation of this work is available at github.com/ivaROS/AeriaLPiPS.
A Gaze-Speech System in Mixed Reality for Human-Robot Interaction

Human-robot interaction (HRI) demands efficient time performance along the tasks. However, some interaction approaches may extend the time to complete such tasks. Thus, the time performance in HRI must be enhanced. This work presents an effective way to enhance the time performance in HRI tasks with a mixed reality (MR) method based on a gaze-speech system. In this paper, we design an MR world for pick-and-place tasks. The hardware system includes an MR headset, the Baxter robot, a table, and six cubes. In addition, the holographic MR scenario offers two modes of interaction: gesture mode (GM) and gaze-speech mode (GSM). The input actions during the GM and GSM methods are based on the pinch gesture and gaze with speech commands, respectively. The proposed GSM approach can improve the time performance in pick-and-place scenarios. The GSM system is 21.33 % faster than the traditional system, GM. Also, we evaluated the target- to-target time performance against a reference based on Fitts' law. Our findings show a promising method for time reduction in HRI tasks through MR environments.
GP-Frontier for Local Mapless Navigation

We propose a new frontier concept called the Gaussian Process Frontier (GP-Frontier) that can be used to locally navigate a robot towards a goal without building a map. The GP-Frontier is built on the uncertainty assessment of an efficient variant of sparse Gaussian Process. Based only on local ranging sensing measurement, the GP-Frontier can be used for navigation in both known and unknown environments. The proposed method is validated through intensive evaluations, and the results show that the GP-Frontier can navigate the robot in a safe and persistent way, i.e., the robot moves in the most open space (thus reducing the risk of collision) without relying on a map or a path planner. A supplementary video that demonstrates the robot navigation behavior is available at https://youtu.be/ndpqTNYqGfw.
Self-Supervised Monocular Depth Underwater

Depth estimation is critical for any robotic system. In the past years, the estimation of depth from monocular images has shown great improvement. However, in the underwater environment results are still lagging behind due to appearance changes caused by the medium. So far little effort has been invested in overcoming this. Moreover, underwater, there are more limitations to using high-resolution depth sensors, which is a serious obstacle to generating ground truth. So far unsupervised methods that tried to solve this have achieved limited success as they relied on domain transfer from a dataset in the air. We suggest network training using subsequent frames, self-supervised by a reprojection loss, as was demonstrated successfully above water. We propose several additions to the self-supervised framework to cope with the underwater environment and achieve state-of-the-art results on a challenging forward-looking underwater dataset.
Kinodynamic Rapidly-exploring Random Forest for Rearrangement-Based Nonprehensile Manipulation

Rearrangement-based nonprehensile manipulation still remains as a challenging problem due to the high-dimensional problem space and the complex physical uncertainties it entails. We formulate this class of problems as a coupled problem of local rearrangement and global action optimization by incorporating free-space transit motions between constrained rearranging actions. We propose a forest-based kinodynamic planning framework to concurrently search in multiple problem regions, so as to enable global exploration of the most task-relevant subspaces, while facilitating effective switches between local rearranging actions. By interleaving dynamic horizon planning and action execution, our framework can adaptively handle real-world uncertainties. With extensive experiments, we show that our framework significantly improves the planning efficiency and manipulation effectiveness while being robust against various uncertainties.
Object Reconfiguration with Simulation-Derived Feasible Actions

3D object reconfiguration encompasses common robot manipulation tasks in which a set of objects must be moved through a series of physically feasible state changes into a desired final configuration. Object reconfiguration is challenging to solve in general, as it requires efficient reasoning about environment physics that determine action validity. This information is typically manually encoded in an explicit transition system. Constructing these explicit encodings is tedious and error-prone, and is often a bottleneck for planner use. In this work, we explore embedding a physics simulator within a motion planner to implicitly discover and specify the valid actions from any state, removing the need for manual specification of action semantics. Our experiments demonstrate that the resulting simulation-based planner can effectively produce physically valid rearrangement trajectories for a range of 3D object reconfiguration problems without requiring more than an environment description and start and goal arrangements.
Explain What You See: Open-Ended Segmentation and Recognition of Occluded 3D Objects

Local-HDP (Local Hierarchical Dirichlet Process) is a hierarchical Bayesian method recently used for open-ended 3D object category recognition. It has been proven to be efficient in real-time robotic applications. However, the method is not robust to a high degree of occlusion. We address this limitation in two steps. First, we propose a novel semantic 3D object-parts segmentation method that has the flexibility of Local-HDP. This method is shown to be suitable for open-ended scenarios where the number of 3D objects or object parts are not fixed and can grow over time. We show that the proposed method has a higher percentage of mean intersection over union, using a smaller number of learning instances. Second, we integrate this technique with a recently introduced argumentation-based online incremental learning method, enabling the model to handle a high degree of occlusion. We show that the resulting model produces explicit explanations for the 3D object category recognition task.
FECD:-Facial Expression Classification &amp; Detection

This paper primarily focuses on Facial Expression detection and classification and beating other SOTA [State-of-the-Art Models] in the space. When discussing face images, the term “facial expressions” is a common terminology. These expressions can sometimes be a blend of two or more human feelings that lead to wrong predictions. When taking about the systems it is commonly classified into four systems, Recognition, Extraction, Detection, and segmentation.
Learning Feasibility of Factored Nonlinear Programs in Robotic Manipulation Planning

A factored Nonlinear Program (Factored-NLP) explicitly models the dependencies between a set of continuous variables and nonlinear constraints, providing an expressive formulation for relevant robotics problems such as manipulation planning or simultaneous localization and mapping. When the problem is over-constrained or infeasible, a fundamental issue is to detect a minimal subset of variables and constraints that are infeasible. Previous approaches require solving several nonlinear programs, incrementally adding and removing constraints, and are thus computationally expensive. In this paper, we propose a graph neural architecture that predicts which variables and constraints are jointly infeasible. The model is trained with a dataset of labeled subgraphs of Factored-NLPs, and importantly, can make useful predictions on larger factored nonlinear programs than the ones seen during training. We evaluate our approach in robotic manipulation planning, where our model is able to generalize to longer manipulation sequences involving more objects and robots, and different geometric environments. The experiments show that the learned model accelerates general algorithms for conflict extraction (by a factor of 50) and heuristic algorithms that exploit expert knowledge (by a factor of 4).
Sentiment Prediction of IMDb Movie Reviews Using CNN-LSTM Approach

This article describes sentiment classification of movie reviews given by the user using deep neural networks. Long short term memory (LSTM) and convolutional neural network (CNN) are two popular deep neural networks used for sentiment analysis. Sentiment analysis is carried out on internet movie dataset (IMDb) which consist of 50K movie reviews. CNN and LSTM architectures are used individually and later combination of CNN-LSTM architecture is used. Accuracy and loss metrics measures are plotted for each architecture where LSTM architecture outperforms compare to CNN and CNN-LSTM architecture. Accuracy of GRU, CNN, LSTM and CNN-LSTM architecures are 53% 85%, 87% and 85% respectively. Adam optimizer and binary cross entropy is used for loss function. CNN-LSTM model is very good for long term dependency and accuracy is also good. Combination of CNN-LSTM reduces a training time for larger dataset and CNN has a convolutional layer to extract information by a larger piece of text.
ANSEL Photobot: A Robot Event Photographer with Semantic Intelligence

Our work examines the way in which large language models can be used for robotic planning and sampling in the context of automated photographic documentation. Specifically, we illustrate how to produce a photo-taking robot with an exceptional level of semantic awareness by leveraging recent advances in general purpose language (LM) and vision-language (VLM) models. Given a high-level description of an event we use an LM to generate a natural-language list of photo descriptions that one would expect a photographer to capture at the event. We then use a VLM to identify the best matches to these descriptions in the robot's video stream. The photo portfolios generated by our method are consistently rated as more appropriate to the event by human evaluators than those generated by existing methods.
Learning Perception-Aware Agile Flight in Cluttered Environments

Recently, neural control policies have outperformed existing model-based planning-and-control methods for autonomously navigating quadrotors through cluttered environments in minimum time.However, they are not perception aware, a crucial requirement in vision-based navigation due to the camera's limited field of view and the underactuated nature of a quadrotor.We propose a learning-based system that achieves perceptionaware, agile flight in cluttered environments.Our method combines imitation learning with reinforcement learning (RL) by leveraging a privileged learning-by-cheating framework.Using RL, we first train a perception-aware teacher policy with full-state information to fly in minimum time through cluttered environments.Then, we use imitation learning to distill its knowledge into a vision-based student policy that only perceives the environment via a camera.Our approach tightly couples perception and control, showing a significant advantage in computation speed (10×faster) and success rate.We demonstrate the closed-loop control performance using hardware-in-the-loop simulation.