nils-herrmann/clean_scirepeval huggingface dataset:
Intuitive human interaction with an arm robot for severely handicapped people - A One Click Approach

Assistance to disabled people is still a domain in which a lot of progress needs to be done. The more severe the handicap is, more complex are the devices, implying increased efforts to simplify the interactions between man and these devices. In this document we propose a solution to reduce the interaction between a user and a robotic arm. The system is equipped with two cameras. One is fixed on the top of the wheelchair (eye-to-hand) and the other one is mounted on the end effector of the robotic arm (eye-in-hand). The two cameras cooperate to reduce the grasping task to "one click". The method is generic, it does not require marks on the object, geometrical model or the database. It thus provides a tool applicable to any kind of graspable object. The paper first gives an overview of the existing grasping tools for disabled people and proposes a novel approach toward an intuitive human machine interaction.
[POSTER] Illumination Estimation Using Cast Shadows for Realistic Augmented Reality Applications

Augmented Reality (AR) scenarios aim to provide realistic blending between real world and virtual objects. A key factor for realistic AR is thus a correct illumination simulation. This consists in estimating the characteristics of real light sources and use them to model virtual lighting. In this paper, we briefly introduce a novel method for recovering both 3D position and intensity of multiple light sources using detected cast shadows. Our algorithm has been successfully tested on a set of real scenes where virtual objects have visually coherent shadows.
Robust Real-Time Visual Tracking: Comparison, Theoretical Analysis and Performance Evaluation

In this paper, two real-time pose tracking algorithms for rigid objects are compared. Both methods are 3D-model based and are capable of calculating the pose between the camera and an object with a monocular vision system. Here, special consideration has been put into defining and evaluating different performance criteria such as computational efficiency, accuracy and robustness. Both methods are described and a unifying framework is derived. The main advantage of both algorithms lie in their real-time capabilities (on standard hardware) whilst being robust to miss-tracking, occlusion and changes in illumination.
A new information theoretic approach for appearance-based navigation of non-holonomic vehicle

In this paper we propose a new way to achieve a navigation task for a non-holonomic vehicle. We consider an image-based navigation process. We show that it is possible to navigate along a visual path without relying on the extraction, matching and tracking of geometric visual features such as keypoint. The new proposed approach relies directly on the information (entropy) contained in the image signal. We show that it is possible to build a control law directly from the maximisation of the shared information between the current image and the next key image in the visual path. The shared information between those two images are obtained using mutual information that is known to be robust to illumination variations and occlusions. Moreover the generally complex task of features extraction and matching is avoided. Both simulations and experiments on a real vehicle are presented and show the possibilities and advantages offered by the proposed method.
Photometric moments: New promising candidates for visual servoing

In this paper, we propose a new type of visual features for visual servoing: photometric moments. These global features do not require any segmentation, matching or tracking steps. The analytical form of the interaction matrix is developed in closed form for these features. Results from experiments carried out with photometric moments have been presented. The results validate our modelling and the control scheme. They perform well for large camera displacements and are endowed with a large convergence domain. From the properties exhibited, photometric moments hold promise as better candidates for IBVS over currently existing geometric and pure luminance features.
Vision-based space autonomous rendezvous: A case study

For a space rendezvous mission, autonomy imposes stringent performance requirements regarding navigation. For the final phase, a vision-based navigation can be a solution. A 3D model-based tracking algorithm has been studied and tested on a mock-up of a telecommunication satellite, using a 6-DOF robotic arm, with satisfactory results, in terms of precision of the pose estimation and computational costs. Quantitative tests in open loop have been carried out to show the robustness of the algorithm to relative inter frame motions chaser/target, orientation variations and illumination conditions. The tracking algorithm has also been successfully implemented in a closed loop chain for visual servoing.
Modeling complex luminance variations for target tracking

Lambertpsilas model is widely used in low level computer vision algorithms such as matching, tracking or optical flow computation for example. However, it is well known that these algorithms often fail when they face complex luminance variations. Therefore, we revise in this paper the underlying hypothesis of its temporal constancy and propose a new optical flow constraint. To do that, we use the Blinn-Phong reflection model to take into account that the scene may move with respect to the lighting and/or to the observer, and that specular highlights may occur. To validate in practice these analytical results, we consider the case where a camera is mounted on a robot end-effector with a lighting mounted on this camera and show experimental results of target tracking by visual servoing. Such an approach requires to analytically compute the luminance variations due to the observer motion which can be easily derived from our revised optical flow constraint. In addition, while the visual servoing classical approaches rely on geometric features, we present here a new method that directly relies on the luminance of all pixels in the image which does not require any tracking or matching process.
Active rough shape estimation of unknown objects

This paper presents a method to determine the rough shape of an object. This is a step in the development of a ldquoone click grasping toolrdquo, a grasping tool of everyday-life objects for an assistant robot dedicated to elderly or disabled. The goal is to determine the quadric that approximates at best the shape of an unknown object using multi-view measurements. Non-linear optimization techniques are considered to achieve this goal. Since multiple views are necessary, an active vision process is considered in order to minimize the uncertainty on the estimated parameters and determine the next best view. Finally, results that show the validity of the approach are presented.
Three-dimensional visual tracking and pose estimation in Scanning Electron Microscopes

Visual tracking and estimation of the 3D posture of a micro/nano-object is a key issue in the development of automated manipulation tasks using the visual feedback. The 3D posture of the micro-object is estimated based on a template matching algorithm. Nevertheless, a key challenge for visual tracking in a scanning electron microscope (SEM) is the difficulty to observe the motion along the depth direction. In this paper, we propose a template-based hybrid visual tracking scheme that uses luminance information to estimate the object displacement on x-y plane and uses defocus information to estimate object depth. This approach is experimentally validated on 4-DoF motion of a sample in a SEM.
Active Vision for Complete Scene Reconstruction and Exploration

Deals with the 3D structure estimation and exploration of static scenes using active vision. Our method is based on the structure from controlled motion approach that constrains camera motions to obtain an optimal estimation of the 3D structure of a geometrical primitive. Since this approach involves gazing on the considered primitive, we have developed perceptual strategies able to perform a succession of robust estimations. This leads to a gaze planning strategy that mainly uses a representation of known and unknown areas as a basis for selecting viewpoints. This approach ensures a reconstruction as complete as possible of the scene.
Visual planes-based simultaneous localization and model refinement for augmented reality

This paper presents a method for camera pose tracking that uses a partial knowledge about the scene. The method is based on monocular vision simultaneous localization and mapping (SLAM). With respect to classical SLAM implementations, this approach uses previously known information about the environment (rough map of the walls) and profits from the various available databases and blueprints to constraint the problem. This method considers that the tracked image patches belong to known planes (with some uncertainty in their localization) and that SLAM map can be represented by associations of cameras and planes. In this paper, we propose an adapted SLAM implementation and detail the considered models. We show that this method gives good results for a real sequence with complex motion for augmented reality (AR) application.
The sequencing of data flow tasks in SIGNAL: application to active vision in robotics

Many applications of real-time systems feature a combination of "continuous" (possibly sampled) and discrete (sequencing and task control) behaviors, both reactive to the evolutions of the execution environment. In particular, robotics applications involve the programming of the control functions for each task, and the sequencing of such tasks at a higher level. In the framework of the real-time synchronized data-flow language SIGNAL, we propose extensions based on time intervals, enabling the specification of tasks, and of their suspension or abortion for their sequencing in reaction to discrete events. This paper illustrates the application of these constructs to active vision in robotics, where successive phases have to be sequenced, each associated with a particular control law, in order to recognise a complex environment.<<ETX>>
Visual servoing set free from image processing

This paper proposes a new way to achieve robotic tasks by visual servoing. Instead of using geometric features (points, straight lines, pose, homography, etc.) as it is usually done, we use directly the luminance of all pixels in the image. Since most of the classical control laws fail in this case, we turn the visual servoing problem into an optimization problem leading to a new control law. Experimental results validate the proposed approach and show its robustness regarding to approximated depths, non Lambertian objects and partial occlusions.
A day at the museum: An augmented fine-art exhibit

This paper examines how an augmented reality guide can enrich museum visits. The results we obtained form an experiment conducted for a museum exhibit. An ergonomic experimentation has been conducted where real visitors use our augmented reality prototype. We collected feedback from these users, helping us to identify the usefulness of AR for museum visits or appreciation of art work. We conclude this paper by proposing some implications about the future usage of augmented reality in a museum context.
MIHash: Online Hashing with Mutual Information

Learning-based hashing methods are widely used for nearest neighbor retrieval, and recently, online hashing methods have demonstrated good performance-complexity trade-offs by learning hash functions from streaming data. In this paper, we first address a key challenge for online hashing: the binary codes for indexed data must be recomputed to keep pace with updates to the hash functions. We propose an efficient quality measure for hash functions, based on an information-theoretic quantity, mutual information, and use it successfully as a criterion to eliminate unnecessary hash table updates. Next, we also show how to optimize the mutual information objective using stochastic gradient descent. We thus develop a novel hashing method, MIHash, that can be used in both online and batch settings. Experiments on image retrieval benchmarks (including a 2.5M image dataset) confirm the effectiveness of our formulation, both in reducing hash table recomputations and in learning high-quality hash functions.
Scale and Rotation Invariant Matching Using Linearly Augmented Trees

We propose a novel linearly augmented tree method for efficient scale and rotation invariant object matching. The proposed method enforces pairwise matching consistency defined on trees, and high-order constraints on all the sites of a template. The pairwise constraints admit arbitrary metrics while the high-order constraints use L1 norms and therefore can be linearized. Such a linearly augmented tree formulation introduces hyperedges and loops into the basic tree structure. But, different from a general loopy graph, its special structure allows us to relax and decompose the optimization into a sequence of tree matching problems that are efficiently solvable by dynamic programming. The proposed method also works on continuous scale and rotation parameters; we can match with a scale up to any large value with the same efficiency. Our experiments on ground truth data and a variety of real images and videos show that the proposed method is efficient, accurate and reliable.
Path Modeling and Retrieval in Distributed Video Surveillance Databases

We propose a framework for querying a distributed database of video surveillance data in order to retrieve a set of likely paths of a person moving in the area under surveillance. In our framework, each camera of the surveillance system locally processes the data and stores video sequences in a storage unit and the metadata for each detected person in the distributed database. A pedestrian's path is formulated as a dynamic Bayesian network (DBN) to model the dependencies between subsequent observations of the person as he makes his way through the camera network. We propose a tool by which the analyst can pose queries about where a certain person appeared while moving in the site during a specified temporal window. The DBN is used in an algorithm that finds potentially relevant metadata records from the distributed databases and then assembles these into probable paths that the person took in the camera network. Finally, the system presents the analyst with the retrieved set of likely paths in ranked order. The computational complexity for our method is quadratic in the number of camera nodes and linear in the number of moving persons. Experiments were carried out on simulated data to test the system with large distributed databases and in a real setting in which six databases store the data from six video cameras. The simulations confirm that our method provides good results with varying numbers of cameras and persons moving in the network. In a real setting, the method reconstructs paths across the camera network with approximatively 75% accuracy at rank 1.
Discovering frequent arrangements of temporal intervals

In this paper we study a new problem in temporal pattern mining: discovering frequent arrangements of temporal intervals. We assume that the database consists of sequences of events, where an event occurs during a time-interval. The goal is to mine arrangements of event intervals that appear frequently in the database. There are many applications where these type of patterns can be useful, including data network, scientific, and financial applications. Efficient methods to find frequent arrangements of temporal intervals using both breadth first and depth first search techniques are described. The performance of the proposed algorithms is evaluated and compared with other approaches on real datasets (American sign language streams and network data) and large synthetic datasets.
Index trees for efficient deformable shape-based retrieval

An improved method for deformable shape-based image indexing and retrieval is described. A pre-computed index tree is used to improve the speed of our previously reported online model fitting method; simple shape features are used as keys in a pre-generated index tree of model instances. A coarse-to-fine indexing scheme is used at different levels of the tree to further improve speed. Experimental results show that the speedup is significant, while the accuracy of shape-based indexing is maintained. A method for shape population-based retrieval is also described. The method allows query formulation based on the population distributions of shapes in each image. Results of population-based queries for a database of blood cell micrographs are shown.
Unconstrained Salient Object Detection via Proposal Subset Optimization

We aim at detecting salient objects in unconstrained images. In unconstrained images, the number of salient objects (if any) varies from image to image, and is not given. We present a salient object detection system that directly outputs a compact set of detection windows, if any, for an input image. Our system leverages a Convolutional-Neural-Network model to generate location proposals of salient objects. Location proposals tend to be highly overlapping and noisy. Based on the Maximum a Posteriori principle, we propose a novel subset optimization framework to generate a compact set of detection windows out of noisy proposals. In experiments, we show that our subset optimization formulation greatly enhances the performance of our system, and our system attains 16-34% relative improvement in Average Precision compared with the state-of-the-art on three challenging salient object datasets.
Real-time structure and motion recovery from two views of a multiplanar scene

This demo addresses the problem of recovering the structure of a 3D scene from two views of a scene composed by textured planar surfaces. In this work, we propose a framework that exploits geometrical and algebraic constraints induced by rigidity and planarity in the scene. We present the 3D reconstruction of multi-planar environments as a multi-linear problem expressed in terms of the planar homographies and the projection of intersections between the observed planes. But, instead of solving a complex multi-linear system, our algorithm iteratively solves several linear problems: (i) coplanar feature segmentation, (ii) planar projective transfer estimation, (iii) the epipole computation, and (iv) finding intersections between planes. Linear methods allow our approach to be fast, hence suitable for online 3D reconstruction and real-time localization of a camera moving within a scene.
Sign Language Spotting with a Threshold Model Based on Conditional Random Fields

Sign language spotting is the task of detecting and recognizing signs in a signed utterance, in a set vocabulary. The difficulty of sign language spotting is that instances of signs vary in both motion and appearance. Moreover, signs appear within a continuous gesture stream, interspersed with transitional movements between signs in a vocabulary and nonsign patterns (which include out-of-vocabulary signs, epentheses, and other movements that do not correspond to signs). In this paper, a novel method for designing threshold models in a conditional random field (CRF) model is proposed which performs an adaptive threshold for distinguishing between signs in a vocabulary and nonsign patterns. A short-sign detector, a hand appearance-based sign verification method, and a subsign reasoning method are included to further improve sign language spotting accuracy. Experiments demonstrate that our system can spot signs from continuous data with an 87.0 percent spotting rate and can recognize signs from isolated data with a 93.5 percent recognition rate versus 73.5 percent and 85.4 percent, respectively, for CRFs without a threshold model, short-sign detection, subsign reasoning, and hand appearance-based sign verification. Our system can also achieve a 15.0 percent sign error rate (SER) from continuous data and a 6.4 percent SER from isolated data versus 76.2 percent and 14.5 percent, respectively, for conventional CRFs.
Online supervised hashing

Fast similarity search is becoming more and more critical given the ever growing sizes of datasets. Hashing approaches provide both fast search mechanisms and compact indexing structures to address this critical need. In image retrieval problems where labeled training data is available, supervised hashing methods prevail over un-supervised methods. However, most supervised hashing methods are batch-learners; this hinders their ability to adapt to changes as a dataset grows and diversifies. In this work, we propose an online supervised hashing technique that is based on Error Correcting Output Codes. Given an incoming stream of training data with corresponding labels, our method learns and adapts its hashing functions in a discriminative manner. Our method makes no assumption about the number of possible class labels, and accommodates new classes as they are presented in the incoming data stream. In experiments with three image retrieval benchmarks, the proposed method yields state-of-the-art retrieval performance as measured in Mean Average Precision, while also being orders-of-magnitude faster than competing batch methods for supervised hashing.
Saliency Detection: A Boolean Map Approach

A novel Boolean Map based Saliency (BMS) model is proposed. An image is characterized by a set of binary images, which are generated by randomly thresholding the image's color channels. Based on a Gestalt principle of figure-ground segregation, BMS computes saliency maps by analyzing the topological structure of Boolean maps. BMS is simple to implement and efficient to run. Despite its simplicity, BMS consistently achieves state-of-the-art performance compared with ten leading methods on five eye tracking datasets. Furthermore, BMS is also shown to be advantageous in salient object detection.
Recursive estimation of motion and planar structure

A specialized formulation of Azarbayejani and Pentland's (1995) framework for recursive recovery of motion, structure and focal length from feature correspondences tracked through an image sequence is presented. The specialized formulation addresses the case where all tracked points lie on a plane. This planarity constraint reduces the dimension of the original state vector, and consequently the number of feature points needed to estimate the state. Experiments with synthetic data and real imagery illustrate the system performance. The experiments confirm that the specialized formulation provides improved accuracy, stability to observation noise, and rate of convergence in estimation for the case where the tracked points lie on a plane.
Object Recognition and Localization Via Spatial Instance Embedding

We propose an approach for improving object recognition and localization using spatial kernels together with instance embedding. Our approach treats each image as a bag of instances (image features) within a multiple instance learning framework, where the relative locations of the instances are considered as well as the appearance similarity of the localized image features. The introduced spatial kernel augments the recognition power of the instance embedding in an intuitive and effective way, providing increased localization performance. We test our approach over two object datasets and present promising results.