nils-herrmann/clean_scirepeval huggingface dataset:
Robotic playing for hierarchical complex skill learning

In complex manipulation scenarios (e.g. tasks requiring complex interaction of two hands or in-hand manipulation), generalization is a hard problem. Current methods still either require a substantial amount of (supervised) training data and / or strong assumptions on both the environment and the task. In this paradigm, controllers solving these tasks tend to be complex. We propose a paradigm of maintaining simpler controllers solving the task in a small number of specific situations. In order to generalize to novel situations, the robot transforms the environment from novel situations into a situation where the solution of the task is already known. Our solution to this problem is to play with objects and use previously trained skills (basis skills). These skills can either be used for estimating or for changing the current state of the environment and are organized in skill hierarchies. The approach is evaluated in complex pick-and-place scenarios that involve complex manipulation. We further show that these skills can be learned by autonomous playing.
A Probabilistic Framework for 3D Visual Object Representation

We present an object representation framework that encodes probabilistic spatial relations between 3D features and organizes these features in a hierarchy. Features at the bottom of the hierarchy are bound to local 3D descriptors. Higher level features recursively encode probabilistic spatial configurations of more elementary features. The hierarchy is implemented in a Markov network. Detection is carried out by a belief propagation algorithm, which infers the pose of high-level features from local evidence and reinforces local evidence from globally consistent knowledge, effectively producing a likelihood for the pose of the object in the detection scene. We also present a simple learning algorithm that autonomously builds hierarchies from local object descriptors. We explain how to use our framework to estimate the pose of a known object in an unknown scene. Experiments demonstrate the robustness of hierarchies to input noise, viewpoint changes, and occlusions.
Visual task outcome verification using deep learning

Manipulation tasks requiring high precision are difficult for reasons such as imprecise calibration and perceptual inaccuracies. We present a method for visual task outcome verification that provides an assessment of the task status as well as information for the robot to improve this status. The final status of the task is assessed as success, failure or in progress. We propose a deep learning strategy to learn the task with a small number of training episodes and without requiring the robot. A probabilistic, appearance-based pose estimation method is used to learn the demonstrated task. For real-data efficiency, synthetic training images are created around the trajectory of the demonstrated task. We show that our method can estimate the task status with high accuracy in several instances of different tasks, and demonstrate the accuracy of a high-precision task on a real robot.
Monocular obstacle avoidance for blind people using probabilistic focus of expansion estimation

Visually impaired people have a much higher chance of head injuries in daily life because of obstacles that cannot be reliably detected using conventional aids. We present part of a solution to this problem, using only one head mounted camera and optical flow techniques. As part of the system, a novel method to estimate the focus of expansion is presented, which also provides a metric for the quality of the estimate. The final result is a real time capable software system, which can detect obstacles at eye level.
Toward learning visual discrimination strategies

Humans learn strategies for visual discrimination through interaction with their environment. Discrimination skills are refined as demanded by the task at hand, and are not a priori determined by any particular feature set. Tasks are typically incompletely specified and evolve continually. This work presents a general framework for learning visual discrimination that addresses some of these characteristics. It is based on an infinite combinatorial feature space consisting of primitive features such as oriented edgels and texture signatures, and compositions thereof. Features are progressively sampled from this space in a simple-to-complex manner. A simple recognition procedure queries learned features one by one and rules out candidate object classes that do not sufficiently exhibit the queried feature. Training images are presented sequentially to the learning system, which incrementally discovers features for recognition. Experimental results on two databases of geometric objects illustrate the applicability of the framework.
What a successful grasp tells about the success chances of grasps in its vicinity

Infants gradually improve their grasping competences, both in terms of motor abilities as well as in terms of the internal shape grasp representations. Grasp densities  provide a statistical model of such an internal learning process. In the concept of grasp densities, kernel density estimation is used based on a six-dimensional kernel representing grasps with given position and orientation. For this so far an isotropic kernel has been used which exact shape have only been weakly justified. Instead in this paper, we use an anisotropic kernel that is statistically based on measured conditional probabilities representing grasp success in the neighborhood of a successful grasp. The anisotropy has been determined utilizing a simulation environment that allowed for evaluation of large scale experiments. The anisotropic kernel has been fitted to the conditional probabilities obtained from the experiments. We then show that convergence is an important problem associated with the grasp density approach and we propose a measure for the convergence of the densities. In this context, we show that the use of the statistically grounded anisotropic kernels leads to a significantly faster convergence of grasp densities.
Data Fusion by Belief Propagation for Multi-Camera Tracking

Multi-camera tracking poses a data fusion problem that integrates image measurements from different cameras. A novel approach to tracking using multiple cameras is proposed that combines particle filters and belief propagation in a unified framework. In each view, a target is tracked by a dedicated particle-filter-based local tracker. The trackers in different views collaborate via belief propagation so that a local tracker operating in one view is able to take advantage of additional information from other views. The message passing mechanism in belief propagation guarantees that wrong information is not propagated across views, thus avoiding a common problem in multi-camera tracking. Target states in each view and in 3D are inferred based on the multi-view image measurements by a set of particle filters, and a sequential belief propagation algorithm implements collaboration between the view-specific particle filters. We demonstrate the effectiveness of our approach on sequences of video surveillance and soccer games
Unsupervised learning of predictive parts for cross-object grasp transfer

We present a principled solution to the problem of transferring grasps across objects. Our approach identifies, through autonomous exploration, the size and shape of object parts that consistently predict the applicability of a grasp across multiple objects. The robot can then use these parts to plan grasps onto novel objects. By contrast to most recent methods, we aim to solve the part-learning problem without the help of a human teacher. The robot collects training data autonomously by exploring different grasps on its own. The core principle of our approach is an intensive encoding of low-level sensorimotor uncertainty with probabilistic models, which allows the robot to generalize the noisy autonomously-generated grasps. Object shape, which is our main cue for predicting grasps, is encoded with surface densities, that model the spatial distribution of points that belong to an object's surface. Grasp parameters are modeled with grasp densities, that correspond to the spatial distribution of object-relative gripper poses that lead to a grasp. The size and shape of grasp-predicting parts are identified by sampling the cross-object correlation of local shape and grasp parameters. We approximate sampling and integrals via Monte Carlo methods to make our computer implementation tractable. We demonstrate the applicability of our method in simulation. A proof of concept on a real robot is also provided.
Using 3D contours and their relations for cognitive vision and robotics

In this work, we make use of 3D visual contours carrying geometric as well as appearance information. Between these contours, we define 3D relations that encode structural information relevant to object-level operations such as similarity assessment and grasping. We show that this relational space can also be used as input features for learning which we exemplify for the grasping of unknown objects. Our representation is motivated by the human visual system in two respects. First, we make use of a visual descriptor that is motivated by hyper-columns in V1. Secondly, the contours can be seen as one stage in a visual hierarchy bridging between local symbolic descriptors to higher level stages of processing such as object coding and grasping.
Emergent structuring of interdependent affordance learning tasks

In this paper, we study the learning mechanisms that facilitate autonomous discovery of an effective affordance prediction structure with multiple actions of different levels of complexity. A robot can benefit from a hierarchical structure where pre-learned basic affordances are used as inputs to bootstrap learning of complex affordances. In a developmental setting, links from basic affordances to the related complex affordances should be self-discovered by the robot, along with a suitable learning order. In order to discover the developmental order, we use Intrinsic Motivation approach that can guide the robot to explore the actions it should execute in order to maximize the learning progress. During this learning, the robot also discovers the structure by discovering and using the most distinctive object features for predicting affordances. We implemented our method in an online learning setup, and tested it in a real dataset that includes 83 objects and the discrete effects (such as pushed, rolled, inserted) created by three poke and one stack action. The results show that the hierarchical structure and the development order emerged from the learning dynamics that is guided by Intrinsic Motivation mechanisms and distinctive feature selection approach.
Duality between the Watershed by Image Foresting Transform and the Fuzzy Connectedness Segmentation Approaches

This paper makes a rereading of two successful image segmentation approaches, the fuzzy connectedness (FC) and the watershed (WS) approaches, by analyzing both by means of the image foresting transform (IFT). This graph-based transform provides a sound framework for analyzing and implementing these methods. This paradigm allows to show the duality existing between the WS by IFT and the FC segmentation approaches. Both can be modeled by an optimal forest computation in a dual form (maximization of the similarities or minimization of the dissimilarities), the main difference being the input parameters: the weights associated to each arc of the graph representing the image. In the WS approach, such weights are based on the (possibly filtered) image gradient values whereas they are based on much more complex affinity values in the FC theory. An efficient algorithm for both FC and IFT-WS computation is proposed. Segmentation robustness issue is also discussed
IFT-Watershed from gray-scale marker

The watershed transform and the morphological reconstruction are two of the most important operators for image segmentation in the framework of mathematical morphology. In many situations, the segmentation requires the classical watershed transform of a reconstructed image. In this paper, we introduce the IFT-watershed from gray scale marker-a method to compute at same time, the reconstruction and the classical watershed transform of the reconstructed image, without explicit computation of any regional minima. The method is based on the Image Foresting Transform (IFT)-a unified and efficient approach to reduce image processing problems to a minimum-cost path forest problem in a graph. As additional contributions, we demonstrate that (i) the cost map of the IFT-watershed from markers is identical to the output of the superior gray scale reconstruction; (ii) other reconstruction algorithms are not watersheds; and (iii) the proposed method achieves competitive advantages as compared to the current classical watershed approach.
Morphological image segmentation applied to video quality assessment

This paper presents a methodology to morphological video segmentation used as part of a method to estimate the subjective video quality assessment. The segmentation goal is to partition the sequence frames in three different regions: homogeneous, border and texture. Previous work has shown the importance of computing video impairment measurements to each of these regions separately, in contrast to use global measurements. The segmentation approach presented in this work uses a collection of morphological tools such as connected smoothing filters, morphological gradients, and watershed. Special attention is devoted to describe the recent concept of connected filters, used as the kernel of the morphological segmentation algorithms. The performance of two morphological segmentation paradigms, one based on the flat zone, and the other based on the watershed-plus-markers approach are evaluated and compared to other segmentation methodology used in video quality assessment.
A Max-Tree Simplification Proposal and Applications for the Interactive Max-Tree Visualization Tool

The max-tree is a data structure that represents hierarchically all connected components of an image. It is a powerful structure to perform image processing and analysis tasks. The interactive max-tree visualization tool was the first proposal of an interactive graphical representation for the max-tree, a tool that can be used to navigate, visualize and manipulate the max-tree nodes. In this paper, we propose a more meaningful simplification procedure for the max-tree, we extend the interactive max-tree visualization tool to work with 3D images and we propose the visualization of attributes, a new feature using the interactive max-tree that provides insight for the development of automatic methods.
An Overview of Max-Tree Principles, Algorithms and Applications

The max-tree is a mathematical morphology data structure that represents an image through the hierarchical relationship of connected components resulting from different thresholds. It was proposed in 1998 by Salembier et al., since then,many efficient algorithms to build and process it were proposed.There are also efficient algorithms to extract size, shape and contrast attributes of the max-tree nodes. These algorithms al-lowed efficient implementation of connected filters like attribute-openings and development of automatic and semi-automatic applications that compete with the state-of-the-art. This paper reviews the max-tree principles, algorithms, applications and its current trends.
A Comparison between Optimum-Path Forest and k-Nearest Neighbors Classifiers

This paper presents a comparison between the k-Nearest Neighbors, with an especial focus on the 1-Nearest Neighbor, and the Optimum-Path Forest supervised classifiers. The first was developed in the 1960s, while the second was recently proposed in the 2000s. Although, they were developed around 40 years apart, we can find many similarities between them, especially between 1-Nearest Neighbor and Optimum-Path Forest. This work shows that the Optimum-Path Forest classifier is equivalent to the 1-Nearest Neighbor classifier when all training samples are used as prototypes. The decision boundaries generated by the classifiers are analysed and also some simulations results for both algorithms are presented to compare their performances in real and synthetic data.
Segmentation of thalamic nuclei based on tensorial morphological gradient of diffusion tensor fields

Although thalamic nuclei are not directly visible on conventional anatomical magnetic resonance images (MRI), it is possible to observe differences between the nuclei using diffusion tensor imaging (DTI), because of their distinct fiber orientation. This work presents a method to segment the various nuclei of human thalamus using diffusion MRI. Our approach is to use the watershed transform and other concepts from mathematical morphology to segment the nuclei. However, to segment structures using the tensor data produced with DTI (as opposed to scalar images) the concept of a tensorial morphological gradient (TMG) needs to be introduced. Based on the TMG, segmentation of the nuclei of the thalamus was successful using the watershed transform. Our segmentation is consistent with a histological atlas. Since the proposed method, as opposed to the majority of the DTI-based segmentation methods, does not require manual seed and/or surface placement, its results are highly repeatable.
Teaching image processing with Khoros

The Khoros scientific software environment has become a popular tool for teaching image processing. The broad scope of Khoros and the communities emphasis on collaboration have contributed to this success. The paper describes how Khoros can impact the learning process and gives an example of how it can be utilized by an instructor to enhance an image processing course. The concept of thinking of Khoros as an "authoring system" is introduced and supported by briefly explaining the components of Khoros that make up the authoring system.<<ETX>>
Extinction Profiles for the Classification of Remote Sensing Data

With respect to recent advances in remote sensing technologies, the spatial resolution of airborne and spaceborne sensors is getting finer, which enables us to precisely analyze even small objects on the Earth. This fact has made the research area of developing efficient approaches to extract spatial and contextual information highly active. Among the existing approaches, morphological profile and attribute profile (AP) have gained great attention due to their ability to classify remote sensing data. This paper proposes a novel approach that makes it possible to precisely extract spatial and contextual information from remote sensing images. The proposed approach is based on extinction filters, which are used here for the first time in the remote sensing community. Then, the approach is carried out on two well-known high-resolution panchromatic data sets captured over Rome, Italy, and Reykjavik, Iceland. In order to prove the capabilities of the proposed approach, the obtained results are compared with the results from one of the strongest approaches in the literature, i.e., APs, using different points of view such as classification accuracies, simplification rate, and complexity analysis. Results indicate that the proposed approach can significantly outperform its alternative in terms of classification accuracies. In addition, based on our implementation, profiles can be generated in a very short processing time. It should be noted that the proposed approach is fully automatic.
A New Hierarchical Decomposition Applied to Object Segmentation in Image Sequences: The Uniform Decomposition

The uniform decomposition is a new hierarchical segmentation method which main property is the preservation of a given characteristic for each subset of the partition provided by the decomposition. Its computation is incremental and, in each iteration, a candidate region is split only if it is found a valid split where each new subset respects an input criterion, for instance size or shape. The uniform decomposition is comparable to the classical one given by the threshold of the extinction values from each regional minima. More, it is comparable while maintaining the criterion for each partition subset. The paper also proposes the application of the uniform decomposition as the core of the watershed from propagated markers, a method applied to assisted segmentation of objects in image sequences. An quantitative evaluation benchmark was applied to assess the combination of the uniform decomposition to the watershed from propagated markers. The evaluation highlights the achievements of good segmentation and efficiency.
Pearson's Correlation Coefficient for Discarding Redundant Information in Real Time Autonomous Navigation System

Lately, many applications for control of autonomous vehicles are being developed and one important aspect is the excess of information, frequently redundant, that imposes a great computational cost in data processing. Based on the fact that real-time navigation systems could have their performance compromised by the need of processing all this redundant information (all images acquired by a vision system, for example), this work proposes an automatic image discarding method using the Pearson's correlation coefficient (PCC). The proposed algorithm uses the PCC as the criteria to decide if the current image is similar to the reference image and could be ignored or if it contains new information and should be considered in the next step of the process (identification of the navigation area by an image segmentation method). If the PCC indicates that there is a high correlation, the image is discarded without being segmented. Otherwise, the image is segmented and is set as the new reference frame for the subsequent frames. This technique was tested in video sequences and showed that more than 90% of the images can be discarded without loss of information, leading to a significant reduction of computational time necessary to identify the navigation area.
Automatic design of morphological operators for motion segmentation

A problem of interest in digital video editing is the elimination of moving objects from one video and the introduction of pieces of other videos in their places. A fundamental problem to build computational tools for this purpose is the segmentation of moving objects. The paper approaches this problem with a novel technique based on Beucher-Meyer's paradigm (F. Meyer and S. Beucher, 1990), with markers detected by morphological operators designed by computational learning techniques. The objects in the first frames of the video are marked manually and used to train the marker detector. Then, the operator designed is used to mark the objects in the other frames and the paradigm is applied to all frames marked by the detector. Some synthetic and real world examples illustrate the application of the technique proposed. Complex situations such as occlusion are examined.
Benchmark for Quantitative Evaluation of Assisted Object Segmentation Methods to Image Sequences

Evaluation of segmentation methods applied to image sequences consists in the analysis of such methods according to quantitative and/or qualitative criteria, usually driven to some application. Literature proposes several metrics for quantitative evaluation of object segmentation methods to image sequences, but it is still considered an open problem, since no one of the proposed metrics is considered the standard one. More, as the best of our knowledge, there is no method in literature that does computational quantitative evaluation of assisted methods to object segmentation in image sequence. This paper introduces a benchmark to do such quantitative evaluation. This evaluation is done according to several criteria such as the robustness of segmentation and the easiness to segment the objects through the sequence. Experimental results also evaluates the robustness of the watershed from propagated markers technique.