nils-herrmann/clean_scirepeval huggingface dataset:
Object proposal using 3D point cloud for DRC-HUBO+

We present an object proposal method which utilizes the 3D data obtained from a depth sensor as well as the color information of images. Our object proposal method is designed to improve the performance of the object detection for a mobile robot equipped with a camera and a laser scanner. Compared to traditional object proposal methods using only 2D images, the proposed method provides much less number of candidate windows for object detection. We show less than 100 object proposal windows per image using the proposed method result in high recall tested on the public dataset. Our method presents object proposals in 3D space as well as in 2D image thus it can further be applied to following tasks for mobile robots such as 3D location and pose estimation of the target object after successful object detection. We validate our method using the real-world object detection dataset for outdoor mobile robots captured during the DRC Finals 2015 and the public dataset for comparison with the previous methods.
Image Decomposition With Multilabel Context: Algorithms and Applications

Most research on image decomposition, e.g., image segmentation and image parsing, has predominantly focused on the low-level visual clues within a single image and neglected the contextual information across images. In this paper, we present a new perspective to image decomposition piloted by the multilabel context associated with each individual image. Observing that the contextual information (i.e., local label representations of the same label are similar while those from different labels are dissimilar) exists across images, we propose to perform image decomposition in a collective way and obtain an optimal representation for each label from a set of multilabeled images. We formulate the problem as an optimization problem which maximizes inter-label difference while minimizing the intra-label difference of the target label representations and propose two ways to solve this problem. Such a contextual image decomposition has a wide variety of applications, among which two exemplary ones-multilabel image annotation and label ranking, are presented and evaluated with different classification techniques. Extensive experiments on two benchmark datasets demonstrate promising results.
Depth from accidental motion using geometry prior

We present a method to reconstruct dense 3D points from small camera motion. We begin with estimating sparse 3D points and camera poses by Structure from Motion (SfM) method with homography decomposition. Although the estimated points are optimized via bundle adjustment and gives reliable accuracy, the reconstructed points are sparse because it heavily depends on the extracted features of a scene. To handle this, we propose a depth propagation method using both a color prior from the images and a geometry prior from the initial points. The major benefit of our method is that we can easily handle the regions with similar colors but different depths by using the surface normal estimated from the initial points. We design our depth propagation framework into the cost minimization process. The cost function is linearly designed, which makes our optimization tractable. We demonstrate the effectiveness of our approach by comparing with a conventional method using various real-world examples.
Visual SLAM by Single-Camera Catadioptric Stereo

In this paper, we present a robust simultaneous localization and mapping (SLAM) framework using a single camera catadioptric stereo system composed of vertically aligned two hyperboloidal mirrors and a CCD camera. In the SLAM, conventional stereo cameras have some problems due to their narrow field of view, for example, the data association problem which occurs in the featureless homogeneous regions or dynamic environments where moving persons or objects exist and error accumulation when the robot moves for a long time. However, a single camera catadioptric stereo system which gives not only a full horizontal field of view but the 3D locations of the landmarks helps to solve the above problems. For more accurate motion estimation, we propose the outlier detection algorithm to eliminate mismatched or incorrectly tracked features. We also propose the rectification algorithm that makes the mirrors and a camera parallel each other to satisfy single viewpoint (SVP). We analyze the proposed methodology through various experiments and have shown the robustness of the proposed SLAM algorithm
Accurate Motion Estimation and High-Precision 3D Reconstruction by Sensor Fusion

The CCD camera and the 2D laser range finder are widely used for motion estimation and 3D reconstruction. With their own strengths and weaknesses, low-level fusion of these two sensors complements each other. We combine these two sensors to perform motion estimation and 3D reconstruction simultaneously and precisely. We develop a motion estimation scheme appropriate for this sensor system. In the proposed method, the motion between two frames is estimated using three points among the scan data, and refined by nonlinear optimization. We validate the accuracy of the proposed method using real images. The results show that the proposed system is a practical solution for motion estimation as well as for 3D reconstruction.
Finding and tracking road lanes using "line-snakes"

This paper presents a method for finding and tracking road lanes. The method extracts and tracks lane boundaries for vision-guided vehicle navigation by combining the Hough transform and the "active line model (ALM)". The Hough transform can extract vanishing points of the road, which can be used as a good estimation of the vehicle heading. For the curved road, however, the estimation may be too crude to be used for navigation. Therefore, the Hough transform is used to obtain an initial position estimation of the lane boundaries on the road. The line snake-ALM-then improves the initial approximation to an accurate configuration of the lane boundaries. Once the line snake is initialized in the first image, it tracks the road lanes using the external and internal forces computed from images and a proposed boundary refinement technique. Specifically, an image region is divided into a few subregions along the vertical direction. The Hough transform is then performed for each sub-region and candidate lines of road lanes in each sub-region are extracted. Among candidate fines, a most prominent fine is found by the ALM that minimizes a defined snake energy. The external energy of ALM is a normalized sum of image gradients along the line. The internal deformation energy ensures the continuity of two neighboring lines by using the angle-difference between two adjacent lines and the distance between the two lines. A search method based on the dynamic programming reduces the computational cost. The proposed method gives a unified framework for detecting, refining and tracking the road lane. Experimental results using images of a real road scene are presented.
Autonomous homing based on laser-camera fusion system

Building maps of unknown environments is a critical factor for autonomous navigation and homing, and this problem is especially challenging in large-scale environments. Recently, sensor fusion systems such as combinations of cameras and laser sensors have become popular in the effort to ensure a general level of performance in this task. In this paper, we present a new homing method in a large-scale environment using a laser-camera fusion system. Instead of fusing data to form a single map builder, we adaptively select sensor data to handle environments which contain ambiguity. For autonomous homing, we propose a new mapping strategy for building a hybrid map and a return strategy for selecting the next target waypoints efficiently. The experimental results demonstrate that the proposed algorithm enables the autonomous homing of a robot in a large-scale indoor environments in real time.
Contextual decomposition of multi-label images

Most research on image decomposition, e.g. image segmentation and image parsing, has predominantly focused on the low-level visual clues within single image and neglected the contextual information across different images. In this paper, we present a new perspective to image decomposition piloted by the multi-labels associated with individual images. Observing that the context information (i.e., local label representations of the same label are similar while those from different labels are dissimilar) exists across different images, we propose to perform image decomposition in a collective way, and then the image decomposition problem is formulated as an optimization which maximizes inter-label difference and at the same time minimizes intra-label difference of the target label representations. Such contextual image decomposition has a wide variety of applications, among which the two exemplary ones are: 1) multi-label image annotation in which the sparse coding of a query image over the bases consisting of all learned label representations naturally produces the multi-label annotation, and 2) label ranking in which the annotated labels are re-ordered according to the sparse coding coefficients on those learned label representations. It is worth noting that these two applications can be performed simultaneously via the label propagation process in sparse coding.
A Taxonomy and Evaluation of Dense Light Field Depth Estimation Algorithms

This paper presents the results of the depth estimation challenge for dense light fields, which took place at the second workshop on Light Fields for Computer Vision (LF4CV) in conjunction with CVPR 2017. The challenge consisted of submission to a recent benchmark [7], which allows a thorough performance analysis. While individual results are readily available on the benchmark web page http://www.lightfield-analysis.net, we take this opportunity to give a detailed overview of the current participants. Based on the algorithms submitted to our challenge, we develop a taxonomy of light field disparity estimation algorithms and give a report on the current state-ofthe- art. In addition, we include more comparative metrics, and discuss the relative strengths and weaknesses of the algorithms. Thus, we obtain a snapshot of where light field algorithm development stands at the moment and identify aspects with potential for further improvement.
Haze removal on superpixel domain

Haze and fog cause poor visibility for vision sensor, and make computer vision problems challenging. In this paper, we propose a new de-hazing method for single image on super-pixel domain. One of widely used approaches for de-hazing is to introduce dark channel prior. However, in the previous researches, dark channel prior map is estimated with the fixed size and rectangle patch, and its size affects the performance of de-hazing algorithm. By replacing the conventional fixed size patch to the super-pixel, transmission map can be estimated with more specific details, which allow obtaining non-saturated haze free results. We demonstrate our preliminary results by comparisons between our method and one of the state-of-the-art algorithm with varying patch size on transmission map and restored images.
Fast multiple objects detection and tracking fusing color camera and 3D LIDAR for intelligent vehicles

For many robotics and intelligent vehicle applications, detection and tracking multiple objects (DATMO) is one of the most important components. However, most of the DATMO applications have difficulty in applying real-world applications due to high computational complexity. In this paper, we propose an efficient DATMO framework that fully employs the complementary information from the color camera and the 3D LIDAR. For high efficiency, we present a segmentation scheme by using both 2D and 3D information which gives accurate segments very quickly. In our experiments, we show that our framework can achieve the faster speed (~4Hz) than the state-of-the-art methods reported in KITTI benchmark (>1Hz).
EureCar turbo: A self-driving car that can handle adverse weather conditions

Autonomous driving technology has made significant advances in recent years. In order for self-driving cars to become practical, they are required to operate safely and reliably even under adverse driving conditions. However, most current autonomous driving cars have only been shown to be operational under amiable weather conditions, i.e., on sunny days on dry roads. In order to enable autonomous cars to handle adverse driving conditions such as rain and wet roads, the algorithm must be able to detect roads within a tolerable margin of error using sensors such as cameras and laser scanners. In this paper, we propose a sensor fusion algorithms that is able to operate under a variety of weather conditions, including rain. Our algorithm was validated when a strong shower occurred during the 2014 Hyundai Motor Company's Autonomous Car Competition. In this paper, we present the competition results that were collected on the same course on both sunny and rainy days. Based on the comparison, we propose the future directions to improve the autonomous driving capability under adverse environmental conditions.
Color edge detection using the Skellam distribution as a sensor noise model

Conventional edge detectors suffer from inherent image noise and threshold determination. In this paper, we propose a noble edge detector based on the noise distribution for CCD or CMOS cameras. By assuming the dominant photon noise, we model the distribution of intensity differences between two neighborhood pixels. Since it is well known that photon noise follows a Poisson distribution, we introduce a Skellam distribution, which is the difference of two Poisson random variables. We show experimentally that the Skellam distribution can be used to model the noise distribution of pixels that are captured from the same scene radiance. For estimating the noise distribution given a single pixel, we find the important property that the Skellam parameters are linearly related to the intensity value of pixels. This linearity enables us to determine noise parameters according to the intensity value. In addition, parameters of the line are preserved under illumination, scene and camera setting changes except for only a gain change. Based on the noise distributions, we calculate intensity allowances of three channels for each pixel given a confidence interval. We propose a noble edge detector by skipping a pre-processing step of conventional Gaussian smoothing which is the main obstacle for robust and accurate edge detection. If the difference of intensity exceeds the intensity allowance at least in a single channel, the in- between pixel is marked as an edge pixel. We demonstrate that without conventional Gaussian smoothing the noise-model based approach can automatically extract the fine details of image structures, such as edge and corners, independent of camera setting.
Motion prior based on general motion statistics

Motion estimation plays an important role in scene matching, SLAM (Simultaneously Localization And Mapping) in computer vision field. In BMA (Block Matching Algorithm) for the video compression, motion estimation is also considerable. Motion prior which could be an initial guess for the next motion has an effect on how well motion is estimated. In many applications, motion model is assumed to be Gaussian model with constant velocity. In this paper, we propose a new motion model based on general motion statistics and demonstrate proposed motion model helps motion estimation become more accurate.
Accurate depth map estimation from a lenslet light field camera

This paper introduces an algorithm that accurately estimates depth maps using a lenslet light field camera. The proposed algorithm estimates the multi-view stereo correspondences with sub-pixel accuracy using the cost volume. The foundation for constructing accurate costs is threefold. First, the sub-aperture images are displaced using the phase shift theorem. Second, the gradient costs are adaptively aggregated using the angular coordinates of the light field. Third, the feature correspondences between the sub-aperture images are used as additional constraints. With the cost volume, the multi-label optimization propagates and corrects the depth map in the weak texture regions. Finally, the local depth map is iteratively refined through fitting the local quadratic function to estimate a non-discrete depth map. Because micro-lens images contain unexpected distortions, a method is also proposed that corrects this error. The effectiveness of the proposed algorithm is demonstrated through challenging real world examples and including comparisons with the performance of advanced depth estimation algorithms.
Vehicle segmentation using evidential reasoning

This paper proposes a segmentation algorithm by means of an evidential reasoning to segment moving vehicles in front of our moving car in a road traffic scene. Generally, an evidential reasoning finds the perceptually known evidences of a target and updates a probabilistic expectation for the target to be in an image. Since a noise image produces unreliable features and degrades the detection and localization, selecting image primitives which are less sensitive to noise and well represent the evidences is important. We carry out this task by the probabilistic integration of image features based on maximum a posteriori (MAP) probability that combines the prior and likelihood probabilities using Bayes' rule.
Geometrically inspired MRF for moving object detection from mobile stereo camera

Detecting moving objects from an image sequence is challenging, especially when the camera is moving and the background varies significantly in every frame. In addition, classifying moving objects using only their appearances creates ambiguities in complex scenes. In this sense a Markov random field (MRF) approach is proposed incorporating a stereo vision-based structure-from-motion scheme in order to robustly detect the moving objects from image sequences. In this MRF formulation, the new energy terms of a high-order likelihood and a temporal pairwise potential are added to improve the detection performance further. The performance of the proposed method is demonstrated from publicly available datasets.
Probabilistic matching of line segments for their homography

Finding correspondences of two images taken from largely different camera configuration is a challenging problem because appearance information such as color, intensity and edge orientation histogram cannot be used. A probabilistic approach to match line segments in the images is proposed for calculating their homography. A membership matrix to represent contribution of every match to the homography is employed. A relaxed version of forward stepwise regression is derived and the test statistic is optimized. An alternating scheme for optimizing the membership and homography is provided. The simulation results on synthetic images validate the proposed method.
Real-time motion detection based on Discrete Cosine Transform

We present a motion detection algorithm by a change detection filter matrix derived from Discrete Cosine Transform. Recently, a Fourier reconstruction scheme shows good results for motion detection. However, its computational cost is a major drawback. We revisit the problem and achieve two orders of magnitude faster than the previous algorithm with better performance. The proposed algorithm runs at about 800 frames per second for VGA resolution images on a consumer hardware by using only integer matrix multiplication and the symmetric property of the change detection filter matrix. In addition, our algorithm is fundamentally robust to sudden illumination changes because it works based on edge information. We verify our algorithm with challenging datasets that contain strong and sudden illumination changes.
Obstacle detection and self-localization without camera calibration using projective invariants

In this paper, we propose two new vision-based methods for indoor mobile robot navigation. One is a self-localization algorithm using projective invariant and the other is a method for obstacle detection by simple image difference and relative positioning. For a geometric model of corridor environment, we use natural features formed by floor, walls, and door frames. Using the cross-ratios of the features can be effective and robust in building and updating model-base, and image matching. We predefine a risk zone without obstacles for a robot, and store the image of the risk zone, which will be used to detect obstacles inside the zone by comparing the stored image with the current image of a new risk zone. The position of the robot and obstacles are determined by relative positioning. The robustness and feasibility of our algorithms have been demonstrated through experiments in corridor environments using the KASIRI-II indoor mobile robot.
Euclidean Structure from Confocal Conics: Theory and Application to Camera Calibration

Plane-based calibration is now a very popular procedure because of its flexibility. One key step consists in detecting a set of coplanar features, from which the Euclidean structure of the corresponding 3D plane has to be computed. We suggest to use confocal conics as calibration targets, as they offer undeniable advantages over other ones (e.g., points or lines) in terms of detection and estimation, especially in the presence of partial occlusion. We introduce important projective and Euclidean properties of the linear family of conics (i.e., the confocal conic range), spanned by two confocal conics. In particular, we rely on the fact that the circular point-envelope - a rank-2 conic that encodes the 2D Euclidean structure - is a degenerate member of any confocal conic range. This allows us to give closed-form solutions in three cases: one conic with known foci, two confocal conics with known product of ratios of semi axes, and two unknown confocal conics. The performances of the proposed algorithms (consisting of a few lines of Matlab-like code) show up high accuracies for both intrinsic and extrinsic camera parameters. In addition to experiments with synthetic data, a video sequence is processed, showing off the interest of using confocal conics as calibration targets, for augmented reality purposes.
StairNet: Top-Down Semantic Aggregation for Accurate One Shot Detection

One-stage object detectors such as SSD or YOLO already have shown promising accuracy with small memory footprint and fast speed. However, it is widely recognized that one-stage detectors have difficulty in detecting small objects while they are competitive with two-stage methods on large objects. In this paper, we investigate how to alleviate this problem starting from the SSD framework. Due to their pyramidal design, the lower layer that is responsible for small objects lacks strong semantics(e.g contextual information). We address this problem by introducing a feature combining module that spreads out the strong semantics in a top-down manner. Our final model StairNet detector unifies the multi-scale representations and semantic distribution effectively. Experiments on PASCAL VOC 2007 and PASCAL VOC 2012 datasets demonstrate that Stair-Net significantly improves the weakness of SSD and outperforms the other state-of-the-art one-stage detectors.