nils-herrmann/clean_scirepeval huggingface dataset:
Single viewpoint stereoscopic sensor calibration

Calibration is a classical issue in computer vision. This work presents a camera calibration approach for single viewpoint cameras which can be modeled by the unified spherical projection. The paper extends the method of perspective camera calibration using virtual visual servoing, to central cameras and particularly to omnidirectional cameras. The contribution is the development of simultaneous intrinsic and extrinsic calibration of N central cameras stereo rig.
Statistically robust 2-D visual servoing

A fundamental step toward broadening the use of real-world image-based visual servoing is to deal with the important issue of reliability and robustness. In order to address this issue, a closed-loop control law is proposed that simultaneously accomplishes a visual servoing task and is robust to a general class of image processing errors. This is achieved with the application of widely accepted statistical techniques such as robust M-estimation and LMedS. Experimental results are presented which demonstrate visual servoing tasks that resist severe outlier contamination.
A real-time tracker for markerless augmented reality

Augmented reality has now progressed to the point where real-time applications are required and being considered. At the same time it is important that synthetic elements are rendered and aligned in the scene in an accurate and visually acceptable way. In order to address these issues a real-time, robust and efficient 3D model-based tracking algorithm is proposed for a 'video see through' monocular vision system. The tracking of objects in the scene amounts to calculating the pose between the camera and the objects. Virtual objects can then be projected into the scene using the pose. Here, non-linear pose computation is formulated by means of a virtual visual servoing approach. In this context, the derivation of point-to-curve interaction matrices is given for different features including lines, circles, cylinders and spheres. A local moving edge tracker is used in order to provide real-time tracking of points normal to the object contours. A method is proposed for combining local position uncertainty and global pose uncertainty in an efficient and accurate way by propagating uncertainty. Robustness is obtained by integrating an M-estimator into the visual control law via an iteratively re-weighted least squares implementation. The method presented in this paper has been validated on several complex image sequences including outdoor environments. Results show the method to be robust to occlusion, changes in illumination and mistracking.
3D model based pose estimation for omnidirectional stereovision

Robot vision has a lot to win as well with wide field of view induced by catadioptric cameras as with redundancy brought by stereovision. Merging these two characteristics in a single sensor is obtained by combining a single camera and multiple mirrors. This paper proposes a 3D model tracking algorithm that allows a robust tracking of 3D objects using stereo catadioptric images given by this sensor. The presented work relies on an adapted virtual visual servoing approach, a non-linear pose computation technique. The model take into account central projection and multiple mirrors. Results show robustness in illumination changes, mistracking and even higher robustness with four mirrors than with two.
Model-free augmented reality by virtual visual servoing

This work presents a method based on the virtual visual servoing approach [Marchard, E et al., 2002] to achieve markerless augmented reality applications. This work aims to realize this task using as little prior 3D information as possible. Virtual visual servoing techniques that lead to a non-linear minimization approach allow one to estimate the 2D transformation between two images of a video sequence which permits to achieve augmented reality on this sequence. Thanks to the work that has already been carried out in this domain, the presented method is efficient and robust wrt. noise and occlusions. It allows very realistic augmented videos with minimum knowledge about the real environment.
Augmenting markerless complex 3D objects by combining geometrical and color edge information

This paper presents a method to address the issue of augmenting a markerless 3D object with a complex shape. It relies on a model-based tracker which takes advantage of GPU acceleration and 3D rendering in order to handle the complete 3D model, whose sharp edges are efficiently extracted. In the pose estimation step, we propose to robustly combine geometrical and color edge-based features in the nonlinear minimization process, and to integrate multiple-hypotheses in the geometrical edge-based registration phase. Our tracking method shows promising results for augmented reality applications, with a Kinect-based reconstructed 3D model.
Increasing optical tracking workspace of VR applications using controlled cameras

We propose an approach to greatly increase the tracking workspace of VR applications without adding new sensors. Our approach relies on controlled cameras able to follow the tracked markers all around the VR workspace providing 6DoF tracking data. We designed the proof-of-concept of such approach based on two consumer-grade cameras and a pan-tilt head. The resulting tracking workspace could be greatly increased depending on the actuators' range of motion. The accuracy error and jitter were found to be rather limited during camera motion (resp. 0.3cm and 0.02cm). Therefore, whenever the final VR application does not require a perfect tracking accuracy over the entire workspace, we recommend using our approach in order to enlarge the tracking workspace.
Hybrid tracking algorithms for planar and non-planar structures subject to illumination changes

Augmented reality (AR) aims to fuse a virtual world and a real one in an image stream. When considering only a vision sensor, it relies on registration techniques that have to be accurate and fast enough for on-line augmentation. This paper proposes a real-time, robust and efficient 3D model-based tracking algorithm monocular vision system. A virtual visual servoing approach is used to estimate the pose between the camera and the object. The integration of texture information in the classical non-linear edge-based pose computation provides a more reliable tracker. Several illumination models have been considered and compared to better deal with the illumination change in the scene. The method presented in this paper has been validated on several video sequences for augmented reality applications.
Chasing a moving target from a flying UAV

This paper proposes a vision-based algorithm to autonomously track and chase a moving target with a small-size flying UAV. The challenging constraints associated with the UAV flight led us to consider a density-based representation of the object to track. The proposed approach to estimate the target's position, orientation and scale, is built on a robust color-based tracker using a multi-part representation. This object tracker can handle large displacements, occlusions and account for some image noise due to partial loss of wireless video link, thanks to the use of a particle filter. The information obtained from the visual tracker is then used to control the position and yaw angle of the UAV in order to chase the target. A hierarchical control scheme is designed to achieve the tracking task. Experiments on a quad-rotor UAV following a small moving car are provided to validate the proposed approach.
Colorimetry-based visual servoing

The goal of this paper is to present a way to perform visual servoing tasks from color attributes. This approach can be seen as an extension of our previous papers based on the luminance , . Indeed, as we did for the luminance, color attributes are directly used in the control law avoiding therefore any complex images processing as features extraction or matching. We propose in this paper several potential color features and then a way to select a priori the best choice among them with respect to the scene being observed. Experimental results validate as well the interest of using color attributes as visual features as our selection process.
Fitting 3D Models on Central Catadioptric Images

Increasing the field of view of camera is an important issue practical in robot vision. One solution is to consider catadioptric camera that allows a 360deg field of view. In this paper we propose a 3D model tracking algorithm that allows a fast and reliable tracking of 3D objects within central catadioptric images. The proposed approach relies on the virtual visual servoing approach. All the modeling aspects have been reconsidered to consider the projection model. Results show the method to be robust and efficient.
Deformable random dot markers

We extend planar fiducial markers using random dots  to nonrigidly deformable markers. Because the recognition and tracking of random dot markers are based on keypoint matching, we can estimate the deformation of the markers with nonrigid surface detection from keypoint correspondences. First, the initial pose of the markers is computed from a homography with RANSAC as a planar detection. Second, deformations are estimated from the minimization of a cost function for deformable surface fitting. We show augmentation results of 2D surface deformation recovery with several markers.
Controlling an uninstrumented ROV manipulator by visual servoing

We present a vision-based method to control the displacement of robot arm mounted on an underwater ROV. A closed-loop system based on an eye-to-hand visual servoing approach has been designed to achieve this task. We show that, using such an approach, measuring the manipulator motion with proprioceptive sensors is not required to precisely control the end-effector motion. To maintain the end effector in the field of view, the camera orientation is also controlled. Presented results show the validity of the approach.
A robust model-based tracker combining geometrical and color edge information

This paper focuses on the issue of estimating the complete 3D pose of the camera with respect to a potentially textureless object, through model-based tracking. We propose to robustly combine complementary geometrical and color edge-based features in the minimization process, and to integrate a multiple-hypotheses framework in the geometrical edge-based registration phase. In order to deal with complex 3D models, our method takes advantage of GPU acceleration. Promising results, outperforming classical state-of-art approaches, have been obtained for space robotics applications on various real and synthetic image sequences and using satellite mock-ups as targets.
An optical tracking system based on hybrid stereo/single-view registration and controlled cameras

Optical tracking is widely used in robotics applications such as unmanned aerial vehicle (UAV) localization. Unfortunately, such systems require many cameras and are, consequently, expensive. In this paper, we propose an approach to considerably increase the optical tracking volume without adding cameras. First, when the target becomes no longer visible by at least two cameras we propose a single-view tracking mode which requires only one camera. Furthermore, we propose to rely on controlled cameras able to track the UAV all around the volume to provide 6DoF tracking data through multi-view registration. This is achieved by using a visual servoing scheme. The two methods can be combined in order to maximize the tracking volume. We propose a proof-of-concept of such an optical tracking system based on two consumer-grade cameras and a pan-tilt actuator and we used this approach on UAV localization.
Robust stereo tracking for space applications

This paper proposes a real-time, robust and efficient 3D model-based tracking algorithm for visual servoing. A virtual visual servoing approach is used for 3D tracking. This method is similar to more classical non-linear pose computation techniques. Robustness is obtained by integrating an M- estimator into the virtual visual control law via an iteratively re- weighted least squares implementation. The presented approach is also extended to the use of multiple cameras. Results show the method to be robust to occlusion, changes in illumination and miss-tracking.
Histograms-Based Visual Servoing

In the direct visual servoing methods such as photometric framework, the images as a whole are used to define a control law. This can be opposed to the classical visual servoing approaches that relies on geometric features and where image processing algorithms that extract and track visual features are necessary. In this letter, we propose a generic framework to consider histogram as a visual feature. A histogram is an estimate of the probability distribution of a variable (for example the probability of occurrence in an intensity, color, or gradient orientation in an image). We show that the framework we proposed applies, but is not limited to, a wide set of histograms and allows the definition of efficient control laws. Statistical comparisons are presented from simulation results and real robots experiments including navigation tasks are also provided.
Microassembly of complex and solid 3D MEMS by 3D vision-based control

This paper describes the vision-based methods developed for assembly of complex and solid 3D MEMS (micro electromechanical systems) structures. The microassembly process is based on sequential robotic operations such as planar positioning, gripping, orientation in space and insertion tasks. Each of these microassembly tasks is performed using a pose-based visual control. To be able to control the microassembly process, a 3D model-based tracker is used. This tracker is able to directly provide the 3D micro-object pose at real-time and from only a single view of the scene. The methods proposed in this paper are validated on an automatic assembly of fives silicon microparts of 400 ¿m × 400 ¿m × 100 ¿m on 3-levels. The insertion tolerance (mechanical play) is estimated to 3 ¿m. The precision of this insertion tolerance allows us to obtain solid and complex micro electromechanical structures without any external joining (glue, wending. Promising positioning and orientation accuracies are obtained who can reach 0.3 ¿m in position and 0.2° in orientation.
Accurate real-time tracking using mutual information

In this paper we present a direct tracking approach that uses Mutual Information (MI) as a metric for alignment. The proposed approach is robust, real-time and gives an accurate estimation of the displacement that makes it adapted to augmented reality applications. MI is a measure of the quantity of information shared by signals that has been widely used in medical applications. Since then, and although MI has the ability to perform robust alignment with illumination changes, multi-modality and partial occlusions, few works propose MI-based applications related to object tracking in image sequences due to some optimization problems.
Real-time keypoints matching: application to visual servoing

Many computer vision problems such as recognition, image retrieval, and tracking require matching two images. Currently, ones try to find as reliable as possible matching techniques with a very little constraint of computational time. In this paper, we are interested in applying image matching technique into robotic problems such as visual servoing in which the computational time is a critical element. We propose in this paper a real time keypoint based matching method. The novelties of this method include a fast corner detector, a compact corner descriptor based on principal component analysis (PCA) technique and an efficient matching with help of approximate nearest neighbor (ANN) technique. We show that the method gives a very satisfying result on accuracy as well as the computational time. The matching algorithm is applied to control a robot in a visual servoing application. It works at 10-14Hz and is well robust to variations in 3D viewpoint and illumination.
Using image gradient as a visual feature for visual servoing

Direct photometric visual servoing has proved to be an efficient approach for robot positioning. Instead of using classical geometric features such as points, straight lines, pose or an homography, as it is usually done, information provided by all pixels in the image are considered. In the past mainly luminance information has been considered. In this paper, considering that most of the useful information in an image is located in its high frequency areas (that are contours), we have consider various possible combinations of global visual feature based on luminance and gradient. Experimental results are presented to show the behavior of such features.
Second-Order Optimization of Mutual Information for Real-Time Image Registration

In this paper, we present a direct image registration approach that uses mutual information (MI) as a metric for alignment. The proposed approach is robust and gives an accurate estimation of a set of 2-D motion parameters in real time. MI is a measure of the quantity of information shared by signals. Although it has the ability to perform robust alignment with illumination changes, multimodality, and partial occlusions, few works have proposed MI-based applications related to spatiotemporal image registration or object tracking in image sequences because of some optimization problems, which we will explain. In this paper, we propose a new optimization method that is adapted to the MI cost function and gives a practical solution for real-time tracking. We show that by refining the computation of the Hessian matrix and using a specific optimization approach, the registration results are far more robust and accurate than the existing solutions, with the computation also being cheaper. A new approach is also proposed to speed up the computation of the derivatives and keep the same optimization efficiency. To validate the advantages of the proposed approach, several experiments are performed.
Tracking planes in omnidirectional stereovision

Omnidirectional cameras allow direct tracking and motion estimation of planar regions in images during a long period of time. However, using only one camera leads to plane and trajectory reconstruction up to a scale factor. We propose to develop dense plane tracking based on omnidirectional stereovision to answer this issue. The presented method estimates simultaneously the parameters of several 3D planes along with the camera motion in a spherical model formulation. Results show the efficiency of the approach.
Decoupled mapping and localization for Augmented Reality on a mobile phone

Using Simultaneous Localization And Mapping (SLAM) methods become more and more common in Augmented Reality (AR). To achieve real-time requirement and to cope with scale factor and the lack of absolute positioning issue, we propose to decouple the localization and the mapping step. We explain the benefits of this approach and how a SLAM strategy can still be used in a way that is meaningful for the end user.
6-DoF automatic micropositioning using photometric information

In this paper, an approach for 6-DoF automatic micropositioning is presented. It involves a closed-loop visual servoing scheme in order to achieve eye-to-hand positioning task in micro-scale. Instead of using classical visual features in the servoing scheme, pure image photometric information from the vision sensor is employed to compute the control law for micropositioning. The approach is validated in simulation as well as experimentally on a parallel positioning stage and a digital microscope at low magnification. Experimental results show the accuracy and efficiency of this control scheme.
Calibration of scanning electron microscope using a multi-image non-linear minimization process

In this paper, a novel approach of SEM calibration based on non-linear minimization process is presented. The SEM calibration for the intrinsic parameters are achieved by an iterative non-linear optimization algorithm which minimize the registration error between the current estimated position of the pattern and its observed position. The calibration can be achieved by one image and multiple images of calibration pattern. Perspective and parallel projection models are addressed in this approach. The experimental results show the efficiency and accuracy of the proposed method.
Dense non-rigid visual tracking with a robust similarity function

This paper deals with dense non-rigid visual tracking robust towards global illumination perturbations of the observed scene. The similarity function is based on the sum of conditional variance (SCV). With respect to most approaches that minimize the sum of squared differences, which is poorly robust towards illumination variations in the scene, the choice of SCV as our registration function allows the approach to be naturally robust towards global perturbations. Moreover, a thin-plate spline warping function is considered in order to take into account deformations of the observed template. The proposed approach, after being detailed, is tested in nominal conditions and on scenes where light perturbations occur in order to assess the robustness of the approach.
Experiments with robust estimation techniques in real-time robot vision

The goal of this paper is to present an overview of robust estimation techniques with a special focus on robotic vision applications. In this particular context, constraints due computation time have to be considered in the choice of the estimation algorithm. Among the numerous techniques proposed in the literature to obtained robust estimation we have, not being exhaustive, Hough transform, RANSAC (Random Sample Consensus), the LMedS (Least Median of Squares), the M-estimators, etc. In this overview, we describe these various approaches in the light of a simple example. Finally, we illustrate the use of robust estimation techniques by various examples in real-time robot vision
Eye-in-hand/eye-to-hand cooperation for visual servoing

The use of a camera in a robot control loop can be performed with two types of architecture: for eye-in-hand camera when it rigidly mounted on the robot end-effector; and for eye-to-hand camera when it observes the robot within its work space. These two schemes have technical differences and they can play very complementary parts. Obviously, the eye-in-hand one has a partial but precise sight of the scene whereas the eye-to-hand camera has a less precise but global sight of it. The motivation of our work is to take advantage of both, free-standing and robot-mounted sensors, in a cooperation scheme. The system presented performs two separate tasks: a positioning task to perform in the global image, and a tracking task to perform in the local image. For robustness considerations, the control law stability is proved and several cooperative schemes are studied and compared in experimental results.
Omnidirectional photometric visual servoing

Visual servoing has been based on geometric features for a long time. Recent works have highlighted the interest of taking into account the photometric information of the entire image. This approach was tackled with images of perspective cameras. We propose, in this paper, to adapt this technique to central cameras. This generalization allows to apply this kind of method to wide field of view cameras. We also propose to adapt gradient computation to take into account distorsions of such cameras. Several experiments have been successfully done with a fisheye camera.