nils-herrmann/clean_scirepeval huggingface dataset:
Gradient-Based Camera Exposure Control for Outdoor Mobile Platforms

We introduce a novel method to automatically adjust camera exposure for image processing and computer vision applications on mobile robot platforms. Because most image processing algorithms rely heavily on low-level image features that are based mainly on local gradient information, we consider that gradient quantity can determine the proper exposure level, allowing a camera to capture the important image features in a manner robust to illumination conditions. We then extend this concept to a multi-camera system and present a new control algorithm to achieve both brightness consistency between adjacent cameras and a proper exposure level for each camera. We implement our prototype system with off-the-shelf machine-vision cameras and demonstrate the effectiveness of the proposed algorithms on practical applications, including pedestrian detection, visual odometry, surround-view imaging, panoramic imaging, and stereo matching.
A biprism-stereo camera system

In this paper we propose a novel and practical stereo camera system that uses only one camera and a biprism placed in front of the camera. The equivalent of a stereo pair of images is formed as the left and right halves of a single CCD image using a biprism. The system is therefore cheap and extremely easy to calibrate since it requires only one CCD camera. An additional advantage of the geometrical set-up is that corresponding features lie on the same scanline automatically. The single camera and biprism have led to a simple stereo system for which correspondence is very easy and which is accurate for nearby objects in a small field of view. Since we we only, a single lens, calibration of the system is greatly simplified. This is due to the fact that we need to estimate only one focal length and one center of projection. Given the parameters in the biprism-stereo camera system, we can recover the depth of the object using only the disparity between the corresponding points.
Complementary Sets of Shutter Sequences for Motion Deblurring

In this paper, we present a novel multi-image motion deblurring method utilizing the coded exposure technique. The key idea of our work is to capture video frames with a set of complementary fluttering patterns to preserve spatial frequency details. We introduce an algorithm for generating a complementary set of binary sequences based on the modern communication theory and implement the coded exposure video system with an off-the-shelf machine vision camera. The effectiveness of our method is demonstrated on various challenging examples with quantitative and qualitative comparisons to other computational image capturing methods used for image deblurring.
UAV global pose estimation by matching forward-looking aerial images with satellite images

A global pose estimation method of an Unmanned Aerial Vehicle (UAV) by matching forward-looking aerial images from the UAV flying at low altitude with down-looking images from a satellite is proposed. To overcome the limitation of significantly different camera viewpoints and characteristics, we use buildings as a cue of matching. We extract buildings from aerial images and construct a 3D model of buildings, using the fundamental matrix. We estimate the global pose of the vehicle by matching 3D structure of buildings with satellite images, using a particle filter. Experimental results show that the proposed approach is a promising method to the global pose estimation of the UAV with forward-looking vision data.
Extrinsic Calibration of 2-D Lidars Using Two Orthogonal Planes

This paper describes a new methodology for estimating the relative pose between two 2-D lidars. Scanned points of 2-D lidars do not have enough feature information for correspondence matching. For this reason, additional image sensors or artificial landmarks at known locations have been used to find the relative pose. We propose a novel method of estimating the relative pose between 2-D lidars without any additional sensors or artificial landmarks. By scanning two orthogonal planes, we utilize the coplanarity of the scan points on each plane and the orthogonality of the plane normals. Even if we capture planes which are not exactly orthogonal, the method provides good results using nonlinear optimization. Experiments with both synthetic and real data show the validity of the proposed method. We also derive two degenerate cases: one related to plane poses, and the other caused by the relative pose. To the best of our knowledge, this study provides the first solution for the problem.
Visual recognition of planar panels with line features

The authors introduce a new method for visual recognition of panels with stiffeners for the sub-assembly process of ship-building. Recognition with existing CAD data is to find the location/orientation of stiffeners and panels, panel ID numbers and approximate welding lines in sequence. The method extracts line features from original images and uses five-line invariants for the recognition. The existing invariant approaches have been proved successful for planar object recognition, but the large search space with combinatorial explosion has been the main hurdle in real application. To reduce this search space, the authors propose a new method with line convex hull (LCH) and indexing logic filter (ILF). In addition, a hierarchical database is constructed to retrieve corresponding model features directly. They have performed experiments using fifteen different welding panels, and demonstrate the feasibility of their method for the recognition of planar objects.
Determination of Color Space for Accurate Change Detection

Most change detection methods are based on gray-level images. A gray-level image is regarded as a 1-D projection of three channels of color images. Therefore, more precise change detection results are expected by utilizing color information. We previously developed a change detection scheme using color images. In this paper, we determine which color space should be selected for accurate change detection based on our previous detection scheme. Our method can be applied to various color spaces, including gray-level images. Then we can measure the expected number of error pixels in order to select an appropriate color space which gives the best result among various color spaces. The experiments show that selecting a color space based on measurements results in the fewest error pixels.
L1-based photometric stereo via augmented lagrange multiplier method

Recently, the sparsity model has been applied to photometric stereo by modeling non-Lambertian artifacts as sparse components. As one of these efforts, we present l1-based photometric stereo for the non-Lambertian corruptions. A solution method was derived using the Augmented Lagrange Multiplier (ALM) method, which effectively solves the constrained problem by solving the sub-problems for surface normal and sparse corruptions iteratively. Experiments demonstrate the applicability of our method by comparing with the Least Square method and the l1 baseline method.
Dichromatic-based color constancy using dichromatic slope and dichromatic line space

In this paper, we present a new dichromatic-based method for estimating illuminant chromaticity using a single image. In the proposed method, specular pixels are roughly extracted from an input image and projected into the three-dimensional dichromatic line space according to their image chromaticities and dichromatic slopes. Reliable dichromatic lines are then obtained by detecting local maxima in the dichromatic line space. Illuminant chromaticity is finally estimated by finding the intersections of the resulting dichromatic lines. The proposed method has some advantages. The proposed method can deal with highly textured images since it does not require precise color segmentation. In addition, the proposed method does not suffer from nearly parallel dichromatic lines and it can estimate artificial illuminant chromaticity.
High quality depth map upsampling for 3D-TOF cameras

This paper describes an application framework to perform high quality upsampling on depth maps captured from a low-resolution and noisy 3D time-of-flight (3D-ToF) camera that has been coupled with a high-resolution RGB camera. Our framework is inspired by recent work that uses nonlocal means filtering to regularize depth maps in order to maintain fine detail and structure. Our framework extends this regularization with an additional edge weighting scheme based on several image features based on the additional high-resolution RGB input. Quantitative and qualitative results show that our method outperforms existing approaches for 3D-ToF upsampling. We describe the complete process for this system, including device calibration, scene warping for input alignment, and even how the results can be further processed using simple user markup.
Multi-scale pyramid pooling for deep convolutional representation

Compared to image representation based on low-level local descriptors, deep neural activations of Convolutional Neural Networks (CNNs) are richer in mid-level representation, but poorer in geometric invariance properties. In this paper, we present a straightforward framework for better image representation by combining the two approaches. To take advantages of both representations, we extract a fair amount of multi-scale dense local activations from a pre-trained CNN. We then aggregate the activations by Fisher kernel framework, which has been modified with a simple scale-wise normalization essential to make it suitable for CNN activations. Our representation demonstrates new state-of-the-art performances on three public datasets: 80.78% (Acc.) on MIT Indoor 67, 83.20% (mAP) on PASCAL VOC 2007 and 91.28% (Acc.) on Oxford 102 Flowers. The results suggest that our proposal can be used as a primary image representation for better performances in wide visual recognition tasks.
Partial Sum Minimization of Singular Values in Robust PCA: Algorithm and Applications

Robust Principal Component Analysis (RPCA) via rank minimization is a powerful tool for recovering underlying low-rank structure of clean data corrupted with sparse noise/outliers. In many low-level vision problems, not only it is known that the underlying structure of clean data is low-rank, but the exact rank of clean data is also known. Yet, when applying conventional rank minimization for those problems, the objective function is formulated in a way that does not fully utilize a priori target rank information about the problems. This observation motivates us to investigate whether there is a better alternative solution when using rank minimization. In this paper, instead of minimizing the nuclear norm, we propose to minimize the partial sum of singular values, which implicitly encourages the target rank constraint. Our experimental analyses show that, when the number of samples is deficient, our approach leads to a higher success rate than conventional rank minimization, while the solutions obtained by the two approaches are almost identical when the number of samples is more than sufficient. We apply our approach to various low-level vision problems, e.g., high dynamic range imaging, motion edge detection, photometric stereo, image alignment and recovery, and show that our results outperform those obtained by the conventional nuclear norm rank minimization method.
Graph-based robust shape matching for robotic application

Shape is one of the useful information for object detection. The human visual system can often recognize objects based on the 2-D outline shape alone. In this paper, we address the challenging problem of shape matching in the presence of complex background clutter and occlusion. To this end, we propose a graph-based approach for shape matching. Unlike prior methods which measure the shape similarity without considering the relation among edge pixels, our approach uses the connectivity of edge pixels by generating a graph. A group of connected edge pixels, which is represented by an “edge” of the graph, is considered together and their similarity cost is defined for the “edge” weight by explicit comparison with the corresponding template part. This approach provides the key advantage of reducing ambiguity even in the presence of background clutter and occlusion. The optimization is performed by means of a graph-based dynamic algorithm. The robustness of our method is demonstrated for several examples including long video sequences. Finally, we applied our algorithm to our grasping robot system by providing the object information in the form of prompt hand-drawn templates.
Weakly- and Self-Supervised Learning for Content-Aware Deep Image Retargeting

This paper proposes a weakly- and self-supervised deep convolutional neural network (WSSDCNN) for contentaware image retargeting. Our network takes a source image and a target aspect ratio, and then directly outpues a retargeted image. Retargeting is performed through a shift reap, which is a pixet-wise mapping from the source to the target grid. Our method implicitly learns an attention map, which leads to r content-aware shift map for image retargeting. As a result, discriminative parts in an image are preserved, while background regions are adjusted seamlessly. In the training phase, pairs of an image and its image-level annotation are used to compute content and structure tosses. We demonstrate the effectiveness of our proposed method for a retargeting application with insightful analyses.
Low-Cost Synchronization for Multispectral Cameras

In this paper, we introduce a low-cost multicamera synchronization approach. Our system is low-cost to make, easy to handle and convenient to use. Proposed system can be employed in single- and multi- spectral various cameras, and also used in any devices which support the external trigger. As a result, our system shows a good performance comparing with hand-eye synchronization, and we also shows that synchronized images are enough to use in ADAS systems.
Multi-Image Deblurring Using Complementary Sets of Fluttering Patterns

We present a novel coded exposure video technique for multi-image motion deblurring. The key idea of this paper is to capture video frames with a set of complementary fluttering patterns, which enables us to preserve all spectrum bands of a latent image and recover a sharp latent image. To achieve this, we introduce an algorithm for generating a complementary set of binary sequences based on the modern communication theory and implement the coded exposure video system with an off-the-shelf machine vision camera. To demonstrate the effectiveness of our method, we provide in-depth analyses of the theoretical bounds and the spectral gains of our method and other state-of-the-art computational imaging approaches. We further show deblurring results on various challenging examples with quantitative and qualitative comparisons to other computational image capturing methods used for image deblurring, and show how our method can be applied for protecting privacy in videos.
Radiometric calibration by transform invariant low-rank structure

We present a robust radiometric calibration method that capitalizes on the transform invariant low-rank structure of sensor irradiances recorded from a static scene with different exposure times. We formulate the radiometric calibration problem as a rank minimization problem. Unlike previous approaches, our method naturally avoids over-fitting problem; therefore, it is robust against biased distribution of the input data, which is common in practice. When the exposure times are completely unknown, the proposed method can robustly estimate the response function up to an exponential ambiguity. The method is evaluated using both simulation and real-world datasets and shows a superior performance than previous approaches.
Relative scale estimation between two camera motions

In structure from motion, if two metric structures are given, the unknown scale between them can be resolved by constraining the rigidity of the metric space. There exist two well-known approaches. The first one is the pose estimation which aims to find the pose of a camera for known 3D points. The second one is the scale estimation whose goal is to resolve the scale after estimating the motion between cameras. Recently, the former way is preferred because of the vulnerability of the scale estimation, the weakness to the image noise. In this paper, we thus propose a robust method which can overcome the weakness of the scale estimation by considering the uncertainty of reconstructed 3D points. Additionally, the rotation matrix is directly corrected under the structural consistency constraint. To illustrate the performance of the method, we demonstrate some examples of large-scale reconstructions and compare the results.
Fast object recognition using salient line groups

This paper presents an effective recognition method based on perceptual organization of low level features detected in an image. The method uses a dynamic programming (DP) based formulation to represent various line groups such as convex, concave, and more complex patterns consisting of convex and concave shapes. The essential features of perceptual organization such as endpoint proximity, collinearity, parallelism, and connectivity of lines, are incorporated into the DP based formulation as energy terms. As endpoint proximity, we detect two line junctions from image lines. We then search for junction groups by using collinearity constraint between the junctions. A DP-based search algorithm is used to detect a junction chain similar to the model chain, based on a local comparison. The proposed system is able to find line groups from images with broken lines and strong background clutters. We demonstrate the feasibility of our DP-based matching method based on perceptual organization using real images.
Ambiguous Surface Defect Image Classification of AMOLED Displays in Smartphones

In this paper, we propose a classification approach for ambiguously shaped defects found on the surface of a type of display panel module that is widely used in the field of mobile displays. These types of surface defects are difficult to properly distinguish due to defect similarity and diversity. In such cases, defect types can only be determined using cumbersome human visual inspection. To solve the problem of ambiguous surface defect classification, we introduce a novel filtering method that effectively separates the foreground defective regions from the background, which has structured patterns, local illumination variation, and different light conditions for each of several cameras in an inspection system. Applying the proposed filter method to defect images, we select important features by adopting a wrapper-based feature selection method using a random forest as a learning algorithm. Successful classification results using the presented model are obtained using challenging real-world defect image data gathered from a smart phone display module inspection line in an industrial plant.
Color Transfer Using Probabilistic Moving Least Squares

This paper introduces a new color transfer method which is a process of transferring color of an image to match the color of another image of the same scene. The color of a scene may vary from image to image because the photographs are taken at different times, with different cameras, and under different camera settings. To solve for a full nonlinear and nonparametric color mapping in the 3D RGB color space, we propose a scattered point interpolation scheme using moving least squares and strengthen it with a probabilistic modeling of the color transfer in the 3D color space to deal with mis-alignments and noise. Experiments show the effectiveness of our method over previous color transfer methods both quantitatively and qualitatively. In addition, our framework can be applied for various instances of color transfer such as transferring color between different camera models, camera settings, and illumination conditions, as well as for video color transfers.
Video-Story Composition via Plot Analysis

We address the problem of composing a story out of multiple short video clips taken by a person during an activity or experience. Inspired by plot analysis of written stories, our method generates a sequence of video clips ordered in such a way that it reflects plot dynamics and content coherency. That is, given a set of multiple video clips, our method composes a video which we call a video-story. We define metrics on scene dynamics and coherency by dense optical flow features and a patch matching algorithm. Using these metrics, we define an objective function for the video-story. To efficiently search for the best video-story, we introduce a novel Branch-and-Bound algorithm which guarantees the global optimum. We collect the dataset consisting of 23 video sets from the web, resulting in a total of 236 individual video clips. With the acquired dataset, we perform extensive user studies involving 30 human subjects by which the effectiveness of our approach is quantitatively and qualitatively verified.
Object Categorization Robust to Surface Markings using Entropy-guided Codebook

Visual categorization is fundamentally important for autonomous mobile robots to get intelligence such as novel object acquisition and topological place recognition. The main difficulty of visual categorization is how to reduce the large intra-class variations. In this paper, we present a new method made robust to that problem by using intermediate blurring and entropy-guided codebook selection in a bag-of-words framework. Intermediate blurring can reduce the high frequency of surface markings and provide dominant shape information. Entropy of a hypothesized codebook can provide the necessary amount of repetition among training exemplars. A generative optimal codebook for each category is learned using the MDL (minimum description length) principle guided by entropy information. Finally, a discriminative codebook is learned using the discriminative method guided by the inter-category entropy of the codebook. We validate the effect of the proposed method using a Caltech-101 DB, which has large intra-class variations
Cost-aware depth map estimation for Lytro camera

Since commercial light field cameras became available, the light field camera has aroused much interest from computer vision and image processing communities due to its versatile functions. Most of its special features are based on an estimated depth map, so reliable depth estimation is a crucial step. However, estimating depth on real light field cameras is a challenging problem due to noise and short baselines among sub-aperture images. We propose a depth map estimation method for light field cameras by exploiting correspondence and focus cues. We aggregate costs among all the sub-aperture images on cost volume to alleviate noise effects. With efficiency of the cost volume, cost-aware depth estimation is quickly achieved by discrete-continuous optimization. In addition, we analyze each property of correspondence and focus cues and utilize them to select reliable anchor points. A well reconstructed initial depth map from the anchors is shown to enhance convergence. We show our method outperforms the state-of-the-art methods by validating it on real datasets acquired with a Lytro camera.