nils-herrmann/clean_scirepeval huggingface dataset:
Monogenic phase based myocardial motion analysis from cardiac ultrasound with transverse oscillations

We present a motion estimation algorithm for cardiac ultrasound images with transverse oscillations after baseband demodulation in the axial direction (B-mode TO). Our algorithm computes the motion by solving the optical-flow equation locally on a sliding spatial window. Conservation of the monogenic phase is assumed in lieu of the traditional pixel brightness. A local affine model accounts for typical cardiac motions as contraction/expansion and shearing. A coarse-to-fine B-spline scheme allows a robust and effective computation of the model parameters. To improve the precision of the estimate, we adopt a pyramidal refinement scheme based on the multiband nature of B-mode TO images: the lowest harmonic is used to produce a rough estimate of the displacement; higher harmonics are then used to refine it thanks to their higher level of detail. Performance was evaluated on realistic simulated cardiac ultrasound sequences and compared to a well-established algorithm. With respect to the latter, our algorithm produced an average reduction of 40.8% of the endpoint error. Also, the contribution of the higher lateral harmonics typical of B-mode TO images was shown to considerably improve the estimation accuracy.
2D-3D camera fusion for visual odometry in outdoor environments

Accurate estimation of camera motion is very important for many robotics applications involving SfM and visual SLAM. Such accuracy is attempted by refining the estimated motion through nonlinear optimization. As many modern robots are equipped with both 2D and 3D cameras, it is both highly desirable and challenging to exploit data acquired from both modalities to achieve a better localization. Existing refinement methods, such as Bundle adjustment and loop closing, may be employed only when precise 2D-to-3D correspondences across frames are available. In this paper, we propose a framework for robot localization that benefits from both 2D and 3D information without requiring such accurate correspondences to be established. This is carried out through a 2D-3D based initial motion estimation followed by a constrained nonlinear optimization for motion refinement. The initial motion estimation finds the best possible 2D-to-3D correspondences and localizes the cameras with respect the 3D scene. The refinement step minimizes the projection errors of 3D points while preserving the existing relationships between images. The problems of occlusion and that of missing scene parts are handled by comparing the image-based reconstruction and 3D sensor measurements. The effect of data inaccuracies is minimized using an M-estimator based technique. Our experiments have demonstrated that the proposed framework allows to obtain a good initial motion estimate and a significant improvement through refinement.
Aerial sequence registration for vehicle detection

In this paper we present a system that aims at detecting and tracking moving objects from aerial video streams. These sequences are obtained from a camera mounted on an aerial vehicle which flies over roads and highways. In order to compensate the motion introduced by the dynamic behavior of the camera, we have to estimate geometrical transformations which stabilize consecutive frames. Our system is based on the use of Mutual Information as a criterion to find an affine model which relates a frame to a reference one. Then, moving regions are extracted using residual motion. The tracking module takes as input these regions to eliminate false alarms and to track vehicles all over the time.
An original approach for automatic plane extraction by omnidirectional vision

Whereas some methods for plane extraction have been proposed, this problem still remains an open issue due to the complexity of the task. This paper especially focuses on the extraction of points lying on a plane (such as the ground and buildings walls) in sequences acquired by a central omnidirectional camera. Our approach is based on the epipolar constraint for planar scenes (i.e. homography) on a pair of omnidirectional images to detect some interest points belonging to a plane. Our main contribution is the introduction of a new method, called “2-point algorithm for homography”, that imposes some constraints on the homography using vanishing point (VP) information. Compared to the widely used DLT (4-point) algorithm, experiments on real data demonstrated that the proposed “2-point algorithm for homography” is more robust to noise and false matching, even when the plane to extract is not dominant in the image. Finally, we show that our system provides key clues for ground segmentation by GrabCut.
Improvement of feature matching in catadioptric images using gyroscope data

Most of vision-based algorithms for motion and localization estimation requires matching some interest points in a pair of images. After building feature correspondence, it is possible to estimate camera motion/localization using epipolar geometry. However feature matching is still a challenging problem because of time constraint or image variability for example. In several robotic applications, the camera rotation may be known thanks to a gyroscope or another orientation sensor. Therefore, in this paper, we aim to answer the following question: can the knowledge of rotation from a gyroscope be used to improve feature matching. To analyze this new approach of camera and gyroscope data fusion, we proceed in two steps. First, we rotationally align the images using rotation information of the gyroscope. And second, we compare the quality of feature matching in the original and rotationally aligned images. Experimental results on a real catadioptric sequence show that gyroscope data permits to sensibly improve the number of inliers according to epipolar geometry.
Rectangle Extraction in Catadioptric Images

Nowadays, robotic systems are more and more equipped with catadioptric cameras. However several problems associated to catadioptric vision have been studied only slightly. Especially algorithms for detecting rectangles in catadioptric images have not yet been developed whereas it is required in diverse applications such as building extraction in aerial images. We show that working in the equivalent sphere provides an appropriate framework to detect lines, parallelism, orthogonality and therefore rectangles. Finally, we present experimental results on synthesized and real data.
A navigation system based on an ominidirectional vision sensor

In this paper we present a dynamic localization system which allows a mobile robot to evolve autonomously in a structured environment. Our system is based on the use of two sensors: an odometer and an omnidirectional vision system which gives a reference in connection with a set of natural beacons. Our navigation algorithm gives a reliable position estimation due to a systematic dynamic resetting. To merge the data obtained we use the extended Kalman filter. Our proposed method allows us to treat efficiently the noise problems linked to the primitive extraction, which contributes to the robustness of our system. Thus, we have developed a reliable and quick navigation system which can deals with the constraints of moving the robots in an industrial environment. We give the experimental results obtained from a mission realized in an a priori known environment.
Accurate scale estimation based on unsynchronized camera network

In this paper we present an unsynchronized camera network able to estimate the motion and the structure with accurate absolute scale. The proposed algorithm requires at least three frames: two frames from one camera and a frame from a neighbouring camera. The relative camera poses are estimated with classical Structure-from-Motion and the absolute scales between views are computed by assuming straight trajectories between consecutive views of one camera. We propose a final optimisation step to refine only the scale and the 3D points. Our method is evaluated in real conditions on the KITTI dataset. We show quantitative evaluation through comparisons against GPS/INS ground truth.
A robust top-down approach for rotation estimation and vanishing points extraction by catadioptric vision in urban environment

A key requirement for unmanned aerial vehicles (UAV) applications is the attitude stabilization of the aircraft, which requires the knowledge of its orientation. It is now well established that traditional navigation equipments, like GPS or INS, suffer from several disadvantages. That is why some works have suggested a vision-based approach of the problem. Especially, catadioptric vision is more and more used since it permits to gather much more information from the environment, compared to traditional perspective cameras, and therefore the robustness of the UAV attitude estimation is improved. Rotation estimation from conventional and catadioptric images has been extensively studied. Whereas interesting results can be obtained, the existing methods have non-negligible limitations such as difficult features matching (e.g. repeated texture, blurring or illumination changing) or a high computational cost (e.g. vanishing point extraction or analyze in frequency domain). In order to overcome these limitations, this paper presents a top-down approach for estimating the rotation and extracting the vanishing points in catadioptric images. This new framework is accurate and can run in real-time. To obtain the ground truth data, we also calibrate our catadioptric camera with a gyroscope. Finally, experimental results on a real video sequence are presented and compared to the ground truth data obtained by the gyroscope.
Extrinsic calibration of non-overlapping camera-laser system using structured environment

In this paper are presented simple and practical solutions to extrinsic calibration between a camera and a 2D laser sensor, without overlap. Previous methods utilized a plane or an intersecting line of two planes as a geometric constraint with enough common field-of-view. These required additional sensors to calibrate non-overlapping systems. In this paper, we present two methods for solving the problem - one utilizes a plane; the other utilizes an intersecting line of two planes. For each method, an initial solution of the relative positions of a non-overlapping camera and a laser sensor, was computed by adopting a reasonable assumption about geometric structures. Then we refined it via non-linear optimization, even if the assumption was not perfectly satisfied. Both simulation results and experiments using real data showed that the proposed methods provided reliable results compared to ground-truth, and similar or better results than those provided by a conventional method.
Dynamic programming and skyline extraction in catadioptric infrared images

Unmanned Aerial Vehicles (UAV) are the subject of an increasing interest in many applications and a key requirement for autonomous navigation is the attitude/position stabilization of the vehicle. Some previous works have suggested using catadioptric vision, instead of traditional perspective cameras, in order to gather much more information from the environment and therefore improve the robustness of the UAV attitude/position estimation. This paper belongs to a series of recent publications of our research group concerning catadioptric vision for UAVs. Currently, we focus on the extraction of skyline in catadioptric images since it provides important information about the attitude/position of the UAV. For example, the DEM-based methods can match the extracted skyline with a Digital Elevation Map (DEM) by process of registration, which permits to estimate the attitude and the position of the camera. Like any standard cameras, catadioptric systems cannot work in low luminosity situations because they are based on visible light. To overcome this important limitation, in this paper, we propose using a catadioptric infrared camera and extending one of our methods of skyline detection towards catadioptric infrared images. The task of extracting the best skyline in images is usually converted in an energy minimization problem that can be solved by dynamic programming. The major contribution of this paper is the extension of dynamic programming for catadioptric images using an adapted neighborhood and an appropriate scanning direction. Finally, we present some experimental results to demonstrate the validity of our approach.
Globally optimal line clustering and vanishing point estimation in Manhattan world

The projections of world parallel lines in an image intersect at a single point called the vanishing point (VP). VPs are a key ingredient for various vision tasks including rotation estimation and 3D reconstruction. Urban environments generally exhibit some dominant orthogonal VPs. Given a set of lines extracted from a calibrated image, this paper aims to (1) determine the line clustering, i.e. find which line belongs to which VP, and (2) estimate the associated orthogonal VPs. None of the existing methods is fully satisfactory because of the inherent difficulties of the problem, such as the local minima and the chicken-and-egg aspect. In this paper, we present a new algorithm that solves the problem in a mathematically guaranteed globally optimal manner and can inherently enforce the VP orthogonality. Specifically, we formulate the task as a consensus set maximization problem over the rotation search space, and further solve it efficiently by a branch-and-bound procedure based on the Interval Analysis theory. Our algorithm has been validated successfully on sets of challenging real images as well as synthetic data sets.
Rotation and translation estimation for a wide baseline fisheye-stereo at crossroads based on traffic flow analysis

Many vision-based systems have been proposed for intersection monitoring, but few of them use wide angle cameras. In this paper, we present a fisheye-stereo monitoring system that will be installed especially in rural areas. Our goal is to estimate the extrinsic calibration decoupled into a rotation followed by a translation. In the context of rural intersections, methods that purely rely on the geometry of the scene are difficult to apply. In fact, the presence of vegetation and the large baseline between the cameras make the task particularly challenging. We present instead an approach based on vehicles' motion analysis on the main road, along with the generation of virtual lines joining the front wheels. We adopt the spherical camera model and use the assumption of planar motion. The approach is validated on a test drive field with a baseline larger than 13 m. We get errors below 3 degrees for the rotation and less than 0.5 percent for the translation at scale. Our method allows a reliable initialization for 3D intersection monitoring.
Summarizing large scale 3D point cloud for navigation tasks

Democratization of 3D sensor devices makes 3D maps building easier especially in long term mapping and autonomous navigation. In this paper we present a new method for summarizing a 3D map (dense cloud of 3D points). This method aims to extract a summary map facilitating the use of this map by navigation systems with limited resources (smartphones, cars, robots…). This Vision-based summarizing process is applied in a fully automatic way using the photometric, geometric and semantic information of the studied environment.
A Branch-and-Bound Approach to Correspondence and Grouping Problems

Data correspondence/grouping under an unknown parametric model is a fundamental topic in computer vision. Finding feature correspondences between two images is probably the most popular application of this research field, and is the main motivation of our work. It is a key ingredient for a wide range of vision tasks, including three-dimensional reconstruction and object recognition. Existing feature correspondence methods are based on either local appearance similarity or global geometric consistency or a combination of both in some heuristic manner. None of these methods is fully satisfactory, especially in the presence of repetitive image textures or mismatches. In this paper, we present a new algorithm that combines the benefits of both appearance-based and geometry-based methods and mathematically guarantees a global optimization. Our algorithm accepts the two sets of features extracted from two images as input, and outputs the feature correspondences with the largest number of inliers, which verify both the appearance similarity and geometric constraints. Specifically, we formulate the problem as a mixed integer program and solve it efficiently by a series of linear programs via a branch-and-bound procedure. We subsequently generalize our framework in the context of data correspondence/grouping under an unknown parametric model and show it can be applied to certain classes of computer vision problems. Our algorithm has been validated successfully on synthesized data and challenging real images.
Towards a qualitative spatial model for road traffic in urban environment

This paper presents preliminary steps towards the development of a general spatial model, based on graph theory, to visualize and reason about the road traffic in an urban environment. This model includes qualitative, in addition to quantitative, data which improves its computation and makes it robust to quantitative errors. The paper also describes different levels of abstraction which define distinct points of view of the environment and hence, allow for the acquisition of heterogeneous data. The graph consists of set of entities and corresponding spatial relations, the semantics behind which differ according to the level of abstraction. The ideas presented in this paper bring together the research done by Geomatics and Robotics/Perception communities.
A Vision Algorithm for Dynamic Detection of Moving Vehicles with a UAV

This paper presents a vision system for road traffic surveillance from sequences acquired from an unmanned aerial vehicle (UAV). This UAV is able to follow a path considered as the surveillance area and defined by a set of ordered GPS points. During the navigation of the UAV, the vision system acquires sequences which are treated in real-time in order to detect vehicles. This detection allows to perform a traffic estimation or to track a pointed out vehicle. The detection of vehicles is based on the spatiotemporal grouping of primitives formulated as a normalized cuts problem. A verification step based on the Dempster-Shafer theory is also proposed in order to recognize the vehicles.
Localization of 2D Cameras in a Known Environment Using Direct 2D-3D Registration

In this paper we propose a robust and direct 2D-to-3D registration method for localizing 2D cameras in a known 3D environment. Although the 3D environment is known, localizing the cameras remains a challenging problem that is particularly undermined by the unknown 2D-3D correspondences, outliers, scale ambiguities and occlusions. Once the cameras are localized, the Structure-from-Motion reconstruction obtained from image correspondences is refined by means of a constrained nonlinear optimization that benefits from the knowledge of the scene. We also propose a common optimization framework for both localization and refinement steps in which projection errors in one view are minimized while preserving the existing relationships between images. The problem of occlusion and that of missing scene parts are handled by employing a scale histogram while the effect of data inaccuracies is minimized using an M-estimator-based technique.
Omnidirectional vision on UAV for attitude computation

Unmanned aerial vehicles (UAVs) are the subject of an increasing interest in many applications. Autonomy is one of the major advantages of these vehicles. It is then necessary to develop particular sensors in order to provide efficient navigation functions. In this paper, we propose a method for attitude computation catadioptric images. We first demonstrate the advantages of the catadioptric vision sensor for this application. In fact, the geometric properties of the sensor permit to compute easily the roll and pitch angles. The method consists in separating the sky from the earth in order to detect the horizon. We propose an adaptation of the Markov random fields for catadioptric images for this segmentation. The second step consists in estimating the parameters of the horizon line thanks to a robust estimation algorithm. We also present the angle estimation algorithm and finally, we show experimental results on synthetic and real images captured from an airplane
UAV Attitude estimation by vanishing points in catadioptric images

Unmanned aerial vehicles (UAV) are the subject of an increasing interest in many applications and a key requirement is the stabilization of the vehicle. Some previous works have suggested using catadioptric vision, instead of traditional perspective cameras, in order to gather much more information from the environment and therefore improve the robustness of the UAV attitude estimation. This paper belongs to a series of recent publications of our research group concerning catadioptric vision for UAVs. Currently, we focus on the estimation of the complete attitude of a UAV flying in urban environment. In order to avoid the limitations of horizon-based approaches, the difficulties of traditional epipolar methods (such as rotation-translation ambiguity, lack of features, retrieving motion parameters from matrix decomposition, etc..) and improve UAV dynamic control, we suggest computing infinite homography. We show how catadioptric vision plays a key role to: first, extract a large number of lines, second robustly estimate the associated vanishing points and third, track them even during long video sequences. Therefore it is not only possible to estimate the relative rotation between consecutive frames but also compute the absolute rotation between two distant frames without error accumulation. Finally, we present some experimental results with ground truth data to demonstrate the accuracy and the robustness of our method.
A direct approach for face detection on omnidirectional images

Catadioptric sensors offer abilities unexploited so far. This is especially true for face detection, and more generally, object detection. This paper presents our results of a direct approach to tackle face detection on catadioptric images. Despite no geometrical transformations, we are able to successfully apply our detector on distorted images. We expose a new method to synthesize large omnidirectional images database. Inspired from regular face detection training schemes, our method makes use of newly introduced polygonal Haar-like features. First tests demonstrated that our approach gives good performance and at the same time speeds up the detection process.
Local path planning in a complex environment for self-driving car

This paper introduces an local path planning algorithm for the self-driving car in a complex environment. The proposed algorithm is composed of three parts: the novel path representation, the collision detection and the path modification using a voronoi cell. The novel path representation provides convenience for checking the collision and modifying the path and continuous control input for steering wheel rather than way point navigation. The proposed algorithm were applied to the self-driving car, EureCar(KAIST) and its applicability and feasibility of real time use were validated.
Adaptative Markov Random Fields for Omnidirectional Vision

Images obtained with catadioptric sensors contain significant deformations which prevent the direct use of classical image treatments. Thus, Markov random fields (MRF) whose usefulness is now obvious for projective image processing, cannot be used directly on catadioptric images because of the inadequacy of the neighborhood. In this paper, we propose to define a new neighborhood for MRF by using the equivalence theorem developed for central catadioptric sensors. We show the importance of this adaptation for a motion detection application