nils-herrmann/clean_scirepeval huggingface dataset:
Navigating in virtual environments with 360° omnidirectional rendering

Typical field-of-view (fov) of visual feedback in virtual reality applications is generally limited. In some cases, e.g. in videogames, the provided fov can be artificially increased, using simple perspective projection methods. In this paper, we design and evaluate different visualization techniques, inspired by the cartography domain, for navigating in virtual environments (VE) with a 360° horizontal fov of the scene. We have conducted an evaluation of different methods compared to a rendering method of reference, i.e. a perspective projection, in a basic navigation task. Our results confirm that using any omnidirectional rendering method could lead to more efficient navigation in terms of average task completion time. Among the different 360° projection methods, the subjective preference was significantly given to a cylindrical projection method (equirectangular). Taken together, our results suggest that omnidirectional rendering could be used in virtual reality applications in which fast navigation or full and rapid visual exploration are important. They pave the way to novel kinds of visual cues and visual rendering methods in virtual reality.
A redundancy-based iterative approach for avoiding joint limits: application to visual servoing

We propose new redundancy-based solutions to avoid robot joint limits of a manipulator. We use a control scheme based on the task function approach. We first recall the classical gradient projection approach and then present a far more efficient method that relies on the iterative computation of motion that does not affect the task achievement and ensures the avoidance problem. We apply this new method in a visual servoing application. We demonstrate the validity of the approach on various real experiments as well as on the control of a virtual humanoid.
Particle filter-based direct visual servoing

With respect to classical visual servoing (VS) technics based on geometrical features, the main drawback of direct visual servoing is its limited convergence area. In this paper we propose a new direct visual servoing control law that relies on a particle filter to achieve non-local and non-linear optimization in order to increase this convergence area. Thanks to multi-view geometry and image transfer techniques, a set of particles (which correspond to potential camera velocities) are drawn and evaluated in order to evaluate the best camera trajectory. This new control law is validated on a 6 DOF positioning task performed on a real gantry robot and statistical comparisons are also provided from simulation results.
Computer animation: a new application for image-based visual servoing

Presents an application for image-based visual servoing: computer graphics animation. Indeed, the control of a virtual camera in a virtual environment is not a trivial problem and usually requires skilled operators. Visual servoing, a now well known technique in robotics and computer vision, consists in positioning a camera according to the informations perceived in the images. Using this method within a computer graphics context leads to a very intuitive approach of animation. Furthermore, in that case a full knowledge about the scene is available. It allows us to easily introduce constraints within the control law in order to react automatically to modifications of the environment. We apply this approach in two different contexts: highly reactive applications (virtual reality, video games) and the control of humanoid avatars.
Hybrid automatic visual servoing scheme using defocus information for 6-DoF micropositioning

Direct photometric visual servoing uses only the pure image information as a visual feature, instead of using classic geometric features such as points or lines. It was demonstrated efficiently in 6 degrees of freedom (DoF) positioning. However, in micro-scale, using only image intensity as a visual feature performs unsatisfactorily in cases where the photometric variation is low, such as motions along vision sensor's focal axis under a high magnification. In order to improve the performance and accuracy in those cases, an approach using hybrid visual features is proposed in this paper. Image gradient is employed as a visual feature on z axis while image intensity is used on the other 5 DoFs to control the motion. A 6-DoF micro-positioning task is accomplished by this hybrid visual servoing scheme. The experimental results obtained on a parallel positioning micro-stage under a digital microscope show the robustness and efficiency of the proposed method.
Improving monocular plane-based SLAM with inertial measures

This article presents a solution to the problem of fusing measurements acquired from a monocular camera with inertial data to achieve simultaneous localization and mapping (SLAM) tasks. This paper describes the models used to correctly integrate inertial and vision data in an EKF-SLAM based application, and ways to perform the fusion on low cost hardware. Both synthetic and real sequences show that our method work and greatly enhance classical SLAM estimation.
Using multiple hypothesis in model-based tracking

Classic registration methods for model-based tracking try to align the projected edges of a 3D model with the edges of the image. However, wrong matches at low level can make these methods fail. This paper presents a new approach allowing to retrieve multiple hypothesis on the camera pose from multiple low-level hypothesis. These hypothesis are integrated into a particle filtering framework to guide the particle set toward the peaks of the distribution. Experiments on simulated and real video sequences show the improvement in robustness of the resulting tracker.
Photometry-based visual servoing using light reflexion models

We present in this paper a way to achieve positioning tasks by visual servoing under complex luminance variations. To do that, we use as visual features the luminance of all pixels in the image as we did in our previous work . An important issue of this approach is that it does not rely at all on a any matching nor tracking process, contrary to all the approaches based on geometric visual features (points, straight lines, pose, homography, etc.). However, we consider in this paper a complete illumination model contrary to  where the temporal luminance constancy hypothesis was assumed. The main issue of this paper is thus the analytical computation of the interaction matrix related to the luminance from the Phong illumination model. Experimental results on specular objects validate our approach.
Hybrid tracking approach using optical flow and pose estimation

This paper proposes an hybrid approach to estimate the 3D pose of an object. The integration of texture information based on image intensities in a more classical non-linear edge-based pose estimation computation has proven to highly increase the reliability of the tracker. We propose in this work to exploit the data provided by an optical flow algorithm for a similar purpose. The advantage of using the optical flow is that it does not require any a priori knowledge on the object appearance. The registration of 2D and 3D cues for monocular tracking is performed by a non linear minimization. Results obtained show that using optical flow enables to perform robust 3D hybrid tracking even without any texture model.
Control Camera and Light Source Positions using Image Gradient Information

In this paper, we propose an original approach to control camera position and/or lighting conditions in an environment using image gradient information. Our goal is to ensure a good viewing condition and good illumination of an object to perform vision-based task (recognition, tracking, etc.). Within the visual servoing framework, we propose solutions to two different issues: maximizing the brightness of the scene and maximizing the contrast in the image. Solutions are proposed to consider either a static light and a moving camera, either or a moving light and a static/moving camera. The proposed method is independent of the structure, color and aspect of the objects. Experimental results on both synthetic and real images are finally presented.
Real-time vision-based microassembly of 3D MEMS

Robotic microassembly is a promising way to fabricate micrometric components based three dimensions (3D) compound products where the materials or the technologies are incompatible: structures, devices, Micro Electro Mechanical Systems (MEMS), Micro Opto Electro Mechanical Systems (MOEMS), etc. To date, solutions proposed in the literature are based on 2D visual control because of the lack of accurate and robust 3D measures from the work scene. In this paper the relevance of the real-time 3D visual tracking and control is demonstrated. The 3D poses of the MEMS is supplied by a model-based tracking algorithm in real-time. It is accurate and robust enough to enable a precise regulation toward zero of a 3D error using a visual servoing approach. The assembly of 400 µm × 400 µm × 100 µm parts by their 100 µm × 100 µm × 100 µm notches with a mechanical play of 3 µm is achieved with a rate of 41 seconds per assembly. The control accuracy reaches 0.3 µm in position and 0.2° in orientation.
Photometric Visual Servoing

This paper proposes a new way to achieve robotic tasks by two-dimensional (2-D) visual servoing. Indeed, instead of using classical geometric features such as points, straight lines, pose, or a homography, as is usually done, the luminance of all pixels in the image is considered here. The main advantage of this new approach is that it requires no tracking or matching process. The key point of our approach relies on the analytic computation of the interaction matrix. This computation is based either on a temporal luminance-constancy hypothesis or on a reflection model so that complex illumination changes can be considered. Experimental results on positioning and tracking tasks validate the proposed approach and show its robustness to approximated depths, low-textured objects, partial occlusions, and specular scenes. They also showed that luminance leads to lower positioning errors than a classical visual servoing based on 2-D geometric visual features.
A visual servoing control law that is robust to image outliers

A fundamental step towards broadening the use of real world image-based visual servoing is to deal with the important issues of reliability and robustness. In order to address this issue, a closed loop control law is proposed that simultaneously accomplishes a visual servoing task and is robust to a general class of external errors. This generality allows concurrent consideration of a wide range of errors including: noise from image feature extraction, small scale errors in the tracking and even large scale errors in the matching between current and desired features. This is achieved with the application of widely accepted statistical techniques of robust M-estimation. The M-estimator is integrated by an iteratively re-weighted method. The median absolute deviation is used as an estimate of the standard deviation of the inlier data and is compared with other methods. This combination is advantageous because of its high efficiency, high breakdown point and desirable influence functions. The robustness and stability of the control law is shown to be dependent on a subsequent measure of position uncertainty. Furthermore the convergence criteria of the control law are investigated. Experimental results are presented which demonstrate visual servoing tasks which resist severe outlier contamination.
Autofocusing-based visual servoing: Application to MEMS micromanipulation

In MEMS microassembly areas, different methods of automatic focusing are presented in the literature. All these methods have a common point. Thus, the current autofocusing methods for microscopes need to perform a scanning on all the vertical axis of the microscope in order to find the peak corresponding to the focus (sharpen image). Those methods are time consuming. Therefore, this paper presents an original method of autofocusing based on a velocity control approach which is developed and validated on real experiments.
3D model-based tracking for UAV position control

This paper presents a 3D model-based tracking suitable for indoor position control of an unmanned aerial vehicle (UAV). Given a 3D model of the edges of its environment, the UAV locates itself thanks to a robust multiple hypothesis tracker. The pose estimation is then fused to inertial data to provide the translational velocity required for the control. A hierarchical control is used to achieve positioning tasks. Experiments on a quad-rotor aerial vehicle validate the proposed approach.
Highly precise micropositioning task using a direct visual servoing scheme

This paper demonstrates accurate micropositioning scheme based on a direct visual servoing process. This technique uses only the pure image signal (photometric information) to design the control law. With respect to traditional visual servoing approaches that use geometric visual features (points, lines …), the visual features used in the control law are the pixel intensity. The proposed approach was tested in term of accuracy and robustness in several experimental conditions. The obtained results have demonstrated a good behavior of the control law and very good positioning accuracy. The obtained accuracies are estimated to 14 nm, 89 nm, and 0.001 degrees in the x, y and θ axes of positioning platform, respectively
Vision-based absolute localization for unmanned aerial vehicles

This paper presents a method for localizing an Unmanned Aerial Vehicle (UAV) using georeferenced aerial images. Easily maneuverable and more and more affordable, UAVs have become a real center of interest. In the last few years, their utilization has significantly increased. Today, they are used for multiple tasks such as navigation, transportation or vigilance. Nevertheless, the success of these tasks could not be possible without a highly accurate localization which can, unfortunately be often laborious. Here we provide a multiple usage localization algorithm based on vision only. However, a major drawback with vision-based algorithms is the lack of robustness. Most of the approaches are sensitive to scene variations (like season or environment changes) due to the fact that they use the Sum of Squared Differences (SSD). To prevent that, we choose to use the Mutual Information (MI) which is very robust toward local and global scene variations. However, dense approaches are often related to drift disadvantages. Here, we solve this problem by using georeferenced images. The localization algorithm has been implemented and experimental results are presented demonstrating the localization of a hexarotor UAV fitted with a downward looking camera during real flight tests.
A model free hybrid algorithm for real time tracking

Robustness and accuracy are major issues in real-time tracking. This paper describes a reliable tracking for markerless planar objects based on the fusion of visual cues and on the estimation of a 2D transformation. Its parameters are estimated by a non-linear minimization of an unique criterion that integrates information on both texture and edges. The efficiency and the robustness of the proposed method are tested on image sequences as well as during a robotic application.
Real time active visual reconstruction using the synchronous paradigm

In this paper, we apply the synchronous approach to real time active visual reconstruction. It illustrates the adequateness of SIGNAL, a synchronous data flow programming language, for the specification of a system dealing with various domains such as robot control, computer vision and the programming of hierarchical parallel automata. More precisely, our application consists in the 3D structure estimation of a set of geometrical primitives using a camera mounted on the end effector of a six dof robot. At the level of camera motion control, the visual servoing approach is specified and implemented in SIGNAL as a function from sensor inputs to control outputs. The 3D reconstruction method is based on the "structure from controlled motion" approach. Its specification is made in parallel to visual servoing. We also present a perception strategy for connecting up several estimations, using time intervals and hierarchical structures for task preemption in SIGNAL. The integration of these techniques is validated experimentally by their implementation on a robotic cell.
Mapping and re-localization for mobile augmented reality

Using Simultaneous Localization And Mapping (SLAM) methods become more and more common in Augmented Reality (AR). To achieve real-time requirement and to cope with scale factor and the lack of absolute positioning issue, we propose to decouple the localization and the mapping step. We explain the benefits of this approach and how a SLAM strategy can still be used in a way that is meaningful for the end user. The method we proposed has been fully implemented on various smartphone in order to show its efficiency.
Using the task function approach to avoid robot joint limits and kinematic singularities in visual servoing

We propose in this paper solutions to avoid robot joint limits and kinematic singularities in visual servoing. We use a control scheme based on the task function approach. It combines the regulation of the selected vision based task with the minimization of a secondary cost function, which reflects the manipulability of the robot in the vicinity of internal or external singularities. Several methods are proposed to avoid joint limits and a comparison between them is presented. We have demonstrated on various experiments the validity of our approach.
Robust model-based tracking for robot vision

This paper proposes a real-time, robust and efficient 3D model-based tracking algorithm for visual servoing. A virtual visual servoing approach is used for monocular 3D tracking. This method is similar to more classical nonlinear pose computation techniques. A concise method for derivation of efficient distance-to-contour interaction matrices is described. An oriented edge detector is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating a M-estimator into the virtual visual control law via an iteratively reweighted least squares implementation. The method presented in this paper has been validated on several 2D 1/2 visual servoing experiments considering various objects. Results show the method to be robust to occlusion, changes in illumination and miss-tracking.
3-D Model-Based Tracking for UAV Indoor Localization

This paper proposes a novel model-based tracking approach for 3-D localization. One main difficulty of standard model-based approach lies in the presence of low-level ambiguities between different edges. In this paper, given a 3-D model of the edges of the environment, we derive a multiple hypotheses tracker which retrieves the potential poses of the camera from the observations in the image. We also show how these candidate poses can be integrated into a particle filtering framework to guide the particle set toward the peaks of the distribution. Motivated by the UAV indoor localization problem where GPS signal is not available, we validate the algorithm on real image sequences from UAV flights.
Stereoscopic rendering of virtual environments with wide Field-of-Views up to 360°

In this paper we introduce a novel approach for stereoscopic rendering of virtual environments with a wide Field-of-View (FoV) up to 360°. Handling such a wide FoV implies the use of non-planar projections and generates specific problems such as for rasterization and clipping of primitives. We propose a novel pre-clip stage specifically adapted to geometric approaches for which problems occur with polygons spanning across the projection discontinuities. Our approach integrates seamlessly with immersive virtual reality systems as it is compatible with stereoscopy, head-tracking, and multi-surface projections. The benchmarking of our approach with different hardware setups could show that it is well compliant with real-time constraint, and capable of displaying a wide range of FoVs. Thus, our geometric approach could be used in various VR applications in which the user needs to extend the FoV and apprehend more visual information.
Optimal detection and tracking of feature points using mutual information

This paper proposes a new way to achieve feature point tracking using the entropy of the image. Sum of Squared Differences (SSD) is widely considered in differential trackers such as the KLT. Here, we consider another metric called Mutual Information (MI), which is far less sensitive to changes in the lighting condition and to a wide class of non-linear image transformation. Since mutual-information is used as an energy function to be maximized to track each points, a new feature selection, which is optimal for this metric, is proposed. Results under various complex conditions are presented. Comparison with the classical KLT tracker are proposed.
3D object pose detection using foreground/background segmentation

This paper addresses the challenge of detecting and localizing a poorly textured known object, by initially estimating its complete 3D pose in a video sequence. Our solution relies on the 3D model of the object and synthetic views. The full pose estimation process is then based on foreground/background segmentation and on an efficient probabilistic edge-based matching and alignment procedure with the set of synthetic views, classified through an unsupervised learning phase. Our study focuses on space robotics applications and the method has been tested on both synthetic and real images, showing its efficiency and convenience, with reasonable computational costs.
Entropy-based visual servoing

In this work we propose a new way to achieve visual servoing using directly the information (as defined by Shannon) of the image. A metric derived from information theory, mutual information, is considered. Mutual information is widely used in multi-modal image registration (medical applications) since it is insensitive to changes in the lighting condition and to a wide class of non-linear image transformation. In this paper mutual-information is used as a new visual feature for visual servoing and allows us to build a new control law to control the 6 dof of the robot. Among various advantages, this approach does not require any matching nor tracking step, is robust to large illumination variation and allows to consider, within the same task, different image modalities. Experiments that demonstrate these advantages conclude the paper.
Toward augmenting everything: Detecting and tracking geometrical features on planar objects

This paper presents an approach for detecting and tracking various types of planar objects with geometrical features. We combine traditional keypoint detectors with Locally Likely Arrangement Hashing (LLAH)  for geometrical feature based keypoint matching. Because the stability of keypoint extraction affects the accuracy of the keypoint matching, we set the criteria of keypoint selection on keypoint response and the distance between keypoints. In order to produce robustness to scale changes, we build a non-uniform image pyramid according to keypoint distribution at each scale. In the experiments, we evaluate the applicability of traditional keypoint detectors with LLAH for the detection. We also compare our approach with SURF and finally demonstrate that it is possible to detect and track different types of textures including colorful pictures, binary fiducial markers and handwritings.