nils-herrmann/clean_scirepeval huggingface dataset:
Capturing city-level scenes with a synchronized camera-laser fusion sensor

In this paper, we present a sensor fusion system of cameras and 2D laser sensors for 3D reconstruction. The proposed system is designed to capture data on a fast-moving ground vehicle. The system consists of six cameras and one 2D laser sensor. In order to capture data at high speed, we synchronized all sensors by detecting the laser ray at a specific angle and generating a trigger signal for the cameras. Reconstruction of 3D structures is done by estimating frame-by-frame motion and accumulating vertical laser scans. The difference between the proposed system and the previous works using two 2D laser sensors is that we do not assume 2D motion. The motion of the system in 3D space (including absolute scale) is estimated accurately by data-level fusion of images and range data. The problem of error accumulation is solved by loop closing, not by GPS. The moving objects are detected by utilizing the depth information provided by the laser sensor. The experimental results show that the estimated path is successfully overlayed on the satellite images.
Object Recognition Using a Generalized Robust Invariant Feature and Gestalt&#146;s Law of Proximity and Similarity

In this paper, we propose a new context-based method for object recognition. We first introduce a neurophysiologically motivated visual part detector. We found that the optimal form of the visual part detector is a combination of a radial symmetry detector and a corner-like structure detector. A general context descriptor, named GRIF (Generalized-Robust Invariant Feature), is then proposed, which encodes edge orientation, edge density and hue information in a unified form. Finally, a context-based voting scheme is proposed. This proposed method is inspired by the function of the human visual system, called figure-ground discrimination. We use the proximity and similarity between features to support each other. The contextual feature descriptor and contextual voting method, which use contextual information, enhance the recognition performance enormously in severely cluttered environments.
Line meets as-projective-as-possible image stitching with moving DLT

We propose a spatially varying stitching method with line correspondences. We are motivated by the observation that point features could be spatially biased or not matched in practice, e.g., repeated textures or homogeneous regions of man-made structures. In this scenario, line matches can provide strong correspondences as well as supplement cues, such as the structure preserving property. With these advantages, we adopt a feature fusion method that combines point and line correspondences into a unified framework for spatially varying stitching. We then estimate the balancing parameter between the point and line terms using geometric error. Our experiments show accurate alignment for challenging but common cases.
Contextual Bag-of-Words for Visual Categorization

Bag-of-words (BOW), which represents an image by the histogram of local patches on the basis of a visual vocabulary, has attracted intensive attention in visual categorization due to its good performance and flexibility. Conventional BOW neglects the contextual relations between local patches due to its Naïve Bayesian assumption. However, it is well known that contextual relations play an important role for human beings to recognize visual categories from their local appearance. This paper proposes a novel contextual bag-of-words (CBOW) representation to model two kinds of typical contextual relations between local patches, i.e., a semantic conceptual relation and a spatial neighboring relation. To model the semantic conceptual relation, visual words are grouped on multiple semantic levels according to the similarity of class distribution induced by them, accordingly local patches are encoded and images are represented. To explore the spatial neighboring relation, an automatic term extraction technique is adopted to measure the confidence that neighboring visual words are relevant. Word groups with high relevance are used and their statistics are incorporated into the BOW representation. Classification is taken using the support vector machine with an efficient kernel to incorporate the relational information. The proposed approach is extensively evaluated on two kinds of visual categorization tasks, i.e., video event and scene categorization. Experimental results demonstrate the importance of contextual relations of local patches and the CBOW shows superior performance to conventional BOW.
Automatic Trimap Generation and Consistent Matting for Light-Field Images.

In this paper, we introduce an automatic approach to generate trimaps and consistent alpha mattes of foreground objects in a light-field image. Our method first performs binary segmentation to roughly segment a light-field image into foreground and background based on depth and color. Next, we estimate accurate trimaps through analyzing color distribution along the boundary of the segmentation using guided image filter and KL-divergence. In order to estimate consistent alpha mattes across sub-images, we utilize the epipolar plane image (EPI) where colors and alphas along the same epipolar line must be consistent. Since EPI of foreground and background are mixed in the matting area, we propagate the EPI from definite foreground/background regions to unknown regions by assuming depth variations within unknown regions are spatially smooth. Using the EPI constraint, we derive two solutions to estimate alpha when color samples along epipolar line are known, and unknown. To further enhance consistency, we refine the estimated alpha mattes by using the multi-image matting Laplacian with an additional EPI smoothness constraint. In experimental evaluations, we have created a dataset where the ground truth alpha mattes of light-field images were obtained by using the blue screen technique. A variety of experiments show that our proposed algorithm produces both visually and quantitatively high-quality alpha mattes for light-field images.
Learning Image Representations by Completing Damaged Jigsaw Puzzles

In this paper, we explore methods of complicating selfsupervised tasks for representation learning. That is, we do severe damage to data and encourage a network to recover them. First, we complicate each of three powerful self-supervised task candidates: jigsaw puzzle, inpainting, and colorization. In addition, we introduce a novel complicated self-supervised task called "Completing damaged jigsaw puzzles" which is puzzles with one piece missing and the other pieces without color. We train a convolutional neural network not only to solve the puzzles, but also generate the missing content and colorize the puzzles. The recovery of the aforementioned damage pushes the network to obtain robust and general-purpose representations. We demonstrate that complicating the self-supervised tasks improves their original versions and that our final task learns more robust and transferable representations compared to the previous methods, as well as the simple combination of our candidate tasks. Our approach achieves state-of-the-art performance in transfer learning on PASCAL classification and semantic segmentation.
Local-driven semi-supervised learning with multi-label

In this paper, we present a local-driven semi-supervised learning framework to propagate the labels of the training data (with multi-label) to the unlabeled data. Instead of using each datum as a vertex of graph, we encode each extracted local feature descriptor as a vertex, and then the labels for each vertex from the training data are derived based on the context among different training data, finally the decomposed labels on each vertex are further propagated to the unlabeled vertices based on the similarities measured according to the features extracted at each local regions. With the learnt local descriptor graph we can predict the semantic labels for not only the test local features but also the test images. The experiments on multi-label image annotation demonstrate the encouraging results from our proposed framework of semi-supervised learning.
Visual tracking for non-rigid objects using Rao-Blackwellized particle filter

Particle filters have been used for visual tracking during long periods because they enable effective estimation for non-linear and non-Gaussian distributions. However, particle filter-based tracking approaches suffer from occlusion and deformation of the target objects, which result in the large difference between the current observations and the target model. Thus, we present a Rao-Blackwellized particle filter (RBPF)-based tracking algorithm that effectively estimates the joint distribution for the target state and the target model; in the proposed method, the target object is tracked by using the particle filter while the target model is simultaneously updated on the basis of the on-line approximation of a mixture of Gaussians. To ensure the robustness to occlusion, we represent the target model by 16 orientation histograms that are spatially divided, and individually update each histogram through a video sequence. We demonstrate the robustness of the proposed method under occlusion and deformation of the target objects.
Learning a Deep Convolutional Network for Light-Field Image Super-Resolution

Commercial Light-Field cameras provide spatial and angular information, but its limited resolution becomes an important problem in practical use. In this paper, we present a novel method for Light-Field image super-resolution (SR) via a deep convolutional neural network. Rather than the conventional optimization framework, we adopt a datadriven learning method to simultaneously up-sample the angular resolution as well as the spatial resolution of a Light-Field image. We first augment the spatial resolution of each sub-aperture image to enhance details by a spatial SR network. Then, novel views between the sub-aperture images are generated by an angular super-resolution network. These networks are trained independently but finally finetuned via end-to-end training. The proposed method shows the state-of-the-art performance on HCI synthetic dataset, and is further evaluated by challenging real-world applications including refocusing and depth map estimation.
Auto-adjusting camera exposure for outdoor robotics using gradient information

We present a new method to auto-adjust camera exposure for outdoor robotics. In outdoor environments, scene dynamic range may be wider than the dynamic range of the cameras due to sunlight and skylight. This can results in failures of vision-based algorithms because important image features are missing due to under-/over-saturation. To solve the problem, we adjust camera exposure to maximize image features in the gradient domain. By exploiting the gradient domain, our method naturally determines the proper exposure needed to capture important image features in a manner that is robust against illumination conditions. The proposed method is implemented using an off-the-shelf machine vision camera and is evaluated using outdoor robotics applications. Experimental results demonstrate the effectiveness of our method, which improves the performance of robot vision algorithms.
Non-contact terrain classification for autonomous mobile robot

In this paper we introduce a method for classifying terrains and for predicting friction coefficient on terrains by applying visual information. Coefficient of friction on a terrain is very important for autonomous mobile robots in driving on road and traversing over obstacle. Our algorithm is based on terrain classification for visual image. To predict friction coefficient from given image, we divide an image into homogeneous regions which have same material composition. The proposed method, non-contacting approach, has advantage over other methods that extract material characteristic of road by sensors contacting road surface. Obtained information about friction coefficient before such terrain is entered can be very useful for path planning and avoiding slippery areas. We form a group of each terrain type. So, when new terrain is entered into a system, the data of new terrain are classified into each group. To improve accuracy of the result of classifying terrains, images are compensated by using contrast enhancement techniques. By grouping each terrain to use the same regression coefficients, we can reduce the amount of processing time for terrain recognition. The proposed method will be verified by real outdoor environment with real vehicles.
Estimating Intrinsic Parameters of Cameras using Two Arbitrary Rectangles

In this paper, we propose new camera calibration methods assuming a static camera. Two corresponding imaged rectangles whose aspect ratios are unknown are sufficient to calibrate a camera. By warping the images properly, we show that the information from the imaged rectangles can be transformed to the form of camera constraints. Based on these results, we propose two methods, one for three or more images and the other for only two images. The proposed methods are verified with synthetic and real images, and the results are comparable with less assumption on cameras and on scenes
High-Quality Depth from Uncalibrated Small Motion Clip

We propose a novel approach that generates a high-quality depth map from a set of images captured with a small viewpoint variation, namely small motion clip. As opposed to prior methods that recover scene geometry and camera motions using pre-calibrated cameras, we introduce a self-calibrating bundle adjustment tailored for small motion. This allows our dense stereo algorithm to produce a high-quality depth map for the user without the need for camera calibration. In the dense matching, the distributions of intensity profiles are analyzed to leverage the benefit of having negligible intensity changes within the scene due to the minuscule variation in viewpoint. The depth maps obtained by the proposed framework show accurate and extremely fine structures that are unmatched by previous literature under the same small motion configuration.
Pushing the Envelope of Modern Methods for Bundle Adjustment

In this paper, we present results and experiments with several methods for bundle adjustment, producing the fastest bundle adjuster ever published in terms of computation and convergence. From a computational perspective, the fastest methods naturally handle the block-sparse pattern that arises in a reduced camera system. Adapting to the naturally arising block-sparsity allows the use of BLAS3, efficient memory handling, fast variable ordering, and customized sparse solving, all simultaneously. We present two methods; one uses exact minimum degree ordering and block-based LDL solving and the other uses block-based preconditioned conjugate gradients. Both methods are performed on the reduced camera system. We show experimentally that the adaptation to the natural block sparsity allows both of these methods to perform better than previous methods. Further improvements in convergence speed are achieved by the novel use of embedded point iterations. Embedded point iterations take place inside each camera update step, yielding a greater cost decrease from each camera update step and, consequently, a lower minimum. This is especially true for points projecting far out on the flatter region of the robustifier. Intensive analyses from various angles demonstrate the improved performance of the presented bundler.
A Real-Time Augmented Reality System to See-Through Cars

One of the most hazardous driving scenario is the overtaking of a slower vehicle, indeed, in this case the front vehicle (being overtaken) can occlude an important part of the field of view of the rear vehicle's driver. This lack of visibility is the most probable cause of accidents in this context. Recent research works tend to prove that augmented reality applied to assisted driving can significantly reduce the risk of accidents. In this paper, we present a real-time marker-less system to see through cars. For this purpose, two cars are equipped with cameras and an appropriate wireless communication system. The stereo vision system mounted on the front car allows to create a sparse 3D map of the environment where the rear car can be localized. Using this inter-car pose estimation, a synthetic image is generated to overcome the occlusion and to create a seamless see-through effect which preserves the structure of the scene.
Geometric Calibration of Micro-Lens-Based Light Field Cameras Using Line Features

We present a novel method for the geometric calibration of micro-lens-based light field cameras. Accurate geometric calibration is the basis of various applications. Instead of using sub-aperture images, we directly utilize raw images for calibration. We select appropriate regions in raw images and extract line features from micro-lens images in those regions. For the entire process, we formulate a new projection model of a micro-lens-based light field camera, which contains a smaller number of parameters than previous models. The model is transformed into a linear form using line features. We compute the initial solution of both the intrinsic and the extrinsic parameters by a linear computation and refine them via non-linear optimization. Experimental results demonstrate the accuracy of the correspondences between rays and pixels in raw images, as estimated by the proposed method.
Identigram/watermark removal using cross-channel correlation

We introduce a method to repair an image which has been stamped by an identigram or a watermark. Our method is based on the cross-channel correlation which assures the co-occurrence of image discontinuities and correlation of color distributions across different color channels of an image. Using blind source separation, we find the transformation of color space which separates the structures of identigram and that of the original image into two different individual color channels. To repair the image contents in the corrupted channel, we formulate the problem using Bayes' rule where the prior and the likelihood probabilities are defined based on the cross-channel correlation assumption. We compare our results with results from inpainting and texture synthesis-based hole filling techniques. Our results are pleasable for real-world examples and have the maximum PSNR for synthetic examples.
A fusion approach for robust visual object tracking in crowd scenes

The visual object tracking problem in a crowd scene has many challenges such as occlusion, similar objects and complex motion. This study presents a system of which modules are composed of feature tracking and detection methods. The proposed system fuses the two modules by converting the incomparable responses into a same metric domain. According to an explicit combining rule, the results of the modules are combined and learned only when the two modules produce consistent results. The performance of the proposed algorithm was quantitatively validated and was compared with other modern visual trackers on i-Lids dataset.
Modeling photo composition and its application to photo re-arrangement

We introduce a learning based photo composition model and its application on photo re-arrangement. In contrast to previous approaches which evaluate quality of photo composition using the rule of thirds or the golden ratio, we train a normalized saliency map from visually pleasurable photos taken by professional photographers. We use Principal Component Analysis (PCA) to analyze training data and build a Gaussian mixture model (GMM) to describe the photo composition model. Our experimental results show that our approach is reliable and our trained photo composition model can be used to improve photo quality through photo re-arrangement.
Recognition-based indoor topological navigation using robust invariant features

In this paper, we present a recognition-based autonomous navigation system for mobile robots. The system is based on our previously proposed robust invariant feature (RIF) detector. This detector extracts highly robust and repeatable features based on the key idea of tracking multi-scale interest points and selecting unique representative local structures with the strongest response in both spatial and scale domains. Weighted Zernike moments are used as the feature descriptor and applied to the place recognition. The navigation system is composed of on-line and off-line two stages. In the off-line learning stage, we train the robot in its workspace by just taking several images of representative places as landmarks. Then, in the on-line navigation stage, the robot recognizes scenes, obtains robust feature correspondences, and navigates the environment autonomously using the iterative pose converging (IPC) algorithm which is based on the idea of the visual servoing technique. The experimental results and the performance evaluation show that the proposed navigation system can achieve excellent performance in complex indoor environments.
Exploiting Shading Cues in Kinect IR Images for Geometry Refinement

In this paper, we propose a method to refine geometry of 3D meshes from the Kinect fusion by exploiting shading cues captured from the infrared (IR) camera of Kinect. A major benefit of using the Kinect IR camera instead of a RGB camera is that the IR images captured by Kinect are narrow band images which filtered out most undesired ambient light that makes our system robust to natural indoor illumination. We define a near light IR shading model which describes the captured intensity as a function of surface normals, albedo, lighting direction, and distance between a light source and surface points. To resolve ambiguity in our model between normals and distance, we utilize an initial 3D mesh from the Kinect fusion and multi-view information to reliably estimate surface details that were not reconstructed by the Kinect fusion. Our approach directly operates on a 3D mesh model for geometry refinement. The effectiveness of our approach is demonstrated through several challenging real-world examples.
Multi lidar system for fast obstacle detection

In recent year, much progress has been made in outdoor obstacle detection. However, for fast moving robotic platform, high-speed obstacle detection is still a daunting challenge. This paper describes laser based system for the fast obstacle detection. To do this, we introduce how to configure laser range finders by using a plane ruler for outdoor robotic platform. For high-speed obstacle detection, we use the gradient of points. We evaluate processing time and accuracy of our system by testing on real drive track including off-road course.
Robust object tracking using an adaptive color model

In this paper we present a new robust face tracking method based on the condensation algorithm that uses a sampling based density representation. A two-dimensional color model is used to approximate the face color. We modified the condensation algorithm to provide color adaptability to the abrupt change of illumination and to the tracking of differently colored people. According to the face size and location uncertainty, the searching range is automatically determined and it makes the algorithm extremely robust and efficient. The tracker operates at real-time and actively controls a camera pan-tilt in order to locate a person's face in the center of the image. Experimental results show the algorithm's robustness to the agile motion of face and to the dramatic change of illumination in the presence of complex background.