nils-herrmann/clean_scirepeval huggingface dataset:
Sensor noise modeling using the Skellam distribution: Application to the color edge detection

In this paper, we introduce the Skellam distribution as a sensor noise model for CCD or CMOS cameras. This is derived from the Poisson distribution of photons that determine the sensor response. We show that the Skellam distribution can be used to measure the intensity difference of pixels in the spatial domain, as well as in the temporal domain. In addition, we show that Skellam parameters are linearly related to the intensity of the pixels. This property means that the brighter pixels tolerate greater variation of intensity than the darker pixels. This enables us to decide automatically whether two pixels have different colors. We apply this modeling to detect the edges in color images. The resulting algorithm requires only a confidence interval for a hypothesis test, because it uses the distribution of image noise directly. More importantly, we demonstrate that without conventional Gaussian smoothing the noise model-based approach can automatically extract the fine details of image structures, such as edges and corners, independent of camera setting.
Learning Optimal Compact Codebook for Efficient Object Categorization

Representation of images using the distribution of local features on a visual codebook is an effective method for object categorization. Typically, discriminative capability of the codebook can lead to a better performance. However, conventional methods usually use clustering algorithms to learn codebooks without considering this. This paper presents a novel approach of learning optimal compact codebooks by selecting a subset of discriminative codes from a large codebook. Firstly, the Gaussian models of object categories based on a single code are learned from the distribution of local features within each image. Then two discriminative criteria, i.e. likelihood ratio and Fisher, are introduced to evaluate how each code contributes to the categorization. We evaluate the optimal codebooks constructed by these two criteria on Caltech-4 dataset, and report superior performance of object categorization compared with traditional K-means method with the same size of codebook.
One-day outdoor photometric stereo via skylight estimation

We present an outdoor photometric stereo method using images captured in a single day. We simulate a sky hemisphere for each image according to its GPS and timestamp, and parameterize the obtained sky hemisphere into a quadratic skylight and a Gaussian sunlight distribution. Unlike previous works which usually model outdoor illumination as a sum of constant ambient light and a distant point light, our method models natural illumination according to a popular sky model and thus provides sufficient constraints for shape reconstruction from one day images. We generate pixel profiles of uniformly sampled unit vectors for the corresponding time of captures and evaluate them using correlation with the actual pixel profiles. The estimated surface normal is refined by MRF optimization. We have tested our method to recover objects and scenes of various sizes in real-world outdoor daylight.
A Multi-view Structured-Light System for Highly Accurate 3D Modeling

We present a multi-view structured-light system which uses geometric and photometric information observed at multiple viewpoints. In our method, a highly accurate geometry reconstruction is achieved by exploiting our multi-view constraint on unwrapped phase images instead of using photo-consistency between intensity images. Also to express fine 3D information, we directly use photometric normal to infer non-linear shape between each point spacing, rather than fusing it by refining the geometry. Our system is built upon the off-the-shelf structured-light system, so that the proposed algorithm can be applied to existing systems including commercial 3D scanners to improve the accuracy without any system modification. We evaluate the performance of our method on both synthetic and real data, and demonstrate superior accuracy, which surpasses the accuracy of the state-of-the-art commercial products.
Fast Randomized Singular Value Thresholding for Low-Rank Optimization

Rank minimization can be converted into tractable surrogate problems, such as Nuclear Norm Minimization (NNM) and Weighted NNM (WNNM). The problems related to NNM, or WNNM, can be solved iteratively by applying a closed-form proximal operator, called Singular Value Thresholding (SVT), or Weighted SVT, but they suffer from high computational cost of Singular Value Decomposition (SVD) at each iteration. We propose a fast and accurate approximation method for SVT, that we call fast randomized SVT (FRSVT), with which we avoid direct computation of SVD. The key idea is to extract an approximate basis for the range of the matrix from its compressed matrix. Given the basis, we compute partial singular values of the original matrix from the small factored matrix. In addition, by developping a range propagation method, our method further speeds up the extraction of approximate basis at each iteration. Our theoretical analysis shows the relationship between the approximation bound of SVD and its effect to NNM via SVT. Along with the analysis, our empirical results quantitatively and qualitatively show that our approximation rarely harms the convergence of the host algorithms. We assess the efficiency and accuracy of the proposed method on various computer vision problems, e.g., subspace clustering, weather artifact removal, and simultaneous multi-image alignment and rectification.
Relative attributes with deep Convolutional Neural Network

Our work is based on the idea of relative attributes, aiming to provide more descriptive information to the images. We propose the model that integrates relative-attribute framework with deep Convolutional Neural Networks (CNN) to increase the accuracy of attribute comparison. In addition, we analyzed the role of each network layer in the process. Our model uses features extracted from CNN and is learned by Rank SVM method with these feature vectors. As a result, our model outperforms the original relative attribute model in terms of significant improvement in accuracy.
PR-SLAM in particle filter framework

Simultaneous localization and mapping is an important task for autonomous mobile robot. To let the robot explore a new environment without any prior map, real-time estimation of the geometrical relation between the robot and the environment is necessary. Extended Kalman filter (EKF)-based approaches are the most common. However, they always have the risk of collapse where the assumption of Gaussian distribution is not applicable. It is well known that state estimation with a particle filter is very robust against clutter in dynamic and noisy environments because of its ability to represent non-Gaussian distributions. Unfortunately, particle-based posterior representation in high dimensions is extremely expensive. We propose an approach, named partitioned recursive SLAM, that overcomes the complexity problem arising in adopting a particle filter in SLAM. By partitioning the state and alternating the turns for the state update, the computational capacity required to process SLAM is reduced to scale linearly with the number of landmarks in the map.
System-on-Chip Solution of Video Stabilization for CMOS Image Sensors in Hand-Held Devices

Majority of CMOS image sensors in consumer market utilize a rolling shutter to increase sensitivity. However, it causes severe distortions, such as jitter, wobble, or skew. Since most of these kinds of sensors are used in hand-held devices, the approach of undistorting and generating stabilized images is restricted to resource limited systems. It has also been one of the major challenges to have a mathematical representation of CMOS rolling effect depicting the practical scenario, while keeping accuracy and stability. We propose that a CMOS sensor can be modeled by a section-wise charge-coupled devices model which has multiple homographies and exploit the observation that rolling shutter mechanism gives close relationships among them. We present a CMOS seven-parameter model, and show video stabilization algorithm by the iterative parameter estimation technique. We address four issues while accelerating our stabilization algorithm within resource limited environment: accuracy, stability, computation time, and resource utilization. We developed cache based optimization techniques to meet the requirement of the memory bandwidth and computational time for the iterative parameter estimation and final output image interpolation, and also proposed the incremental form of the seven-parameter model to greatly reduce resource consumption while maintaining the same results as the previous. The validity and effectiveness of our approach is demonstrated by experiments for different types of camera motions. The cache based optimization technique can be used to accelerate other types of iterative vision algorithms that require repetitive memory access: feature tracking, motion estimation, motion compensation, various types of image distortion correction, and also image warping and scaling.
Robust and Globally Optimal Manhattan Frame Estimation in Near Real Time

Most man-made environments, such as urban and indoor scenes, consist of a set of parallel and orthogonal planar structures. These structures are approximated by the Manhattan world assumption, in which notion can be represented as a Manhattan frame (MF). Given a set of inputs such as surface normals or vanishing points, we pose an MF estimation problem as a consensus set maximization that maximizes the number of inliers over the rotation search space. Conventionally, this problem can be solved by a branch-and-bound framework, which mathematically guarantees global optimality. However, the computational time of the conventional branch-and-bound algorithms is rather far from real-time. In this paper, we propose a novel bound computation method on an efficient measurement domain for MF estimation, i.e., the extended Gaussian image (EGI). By relaxing the original problem, we can compute the bound with a constant complexity, while preserving global optimality. Furthermore, we quantitatively and qualitatively demonstrate the performance of the proposed method for various synthetic and real-world data. We also show the versatility of our approach through three different applications: extension to multiple MF estimation, 3D rotation based video stabilization, and vanishing point estimation (line clustering).
Robust High Dynamic Range Imaging by Rank Minimization

This paper introduces a new high dynamic range (HDR) imaging algorithm which utilizes rank minimization. Assuming a camera responses linearly to scene radiance, the input low dynamic range (LDR) images captured with different exposure time exhibit a linear dependency and form a rank-1 matrix when stacking intensity of each corresponding pixel together. In practice, misalignments caused by camera motion, presences of moving objects, saturations and image noise break the rank-1 structure of the LDR images. To address these problems, we present a rank minimization algorithm which simultaneously aligns LDR images and detects outliers for robust HDR generation. We evaluate the performances of our algorithm systematically using synthetic examples and qualitatively compare our results with results from the state-of-the-art HDR algorithms using challenging real world examples.
Robust visual lock-on and simultaneous localization for an unmanned aerial vehicle

We present a method for simultaneously locking on to a ground target and estimating the position of an unmanned aerial vehicle (UAV) under countermeasure (CM) conditions, where sensors are prevented from successfully tracking a target. Owing to the limited payload and power of the UAVs, we employ a monocular camera and a global positioning system (GPS) to carry out vision-based simultaneous localization and mapping (SLAM) using both an unscented Kalman filter and a Kalman filter. Since this approach estimates the state of the UAV and the location of the target, we can estimate the position of the target in the image, even in the presence of CMs. Our experiments show that the proposed method successfully locks on to the target and estimates the state of the UAV.
Accurate Camera Calibration Robust to Defocus Using a Smartphone

We propose a novel camera calibration method for defocused images using a smartphone under the assumption that the defocus blur is modeled as a convolution of a sharp image with a Gaussian point spread function (PSF). In contrast to existing calibration approaches which require well-focused images, the proposed method achieves accurate camera calibration with severely defocused images. This robustness to defocus is due to the proposed set of unidirectional binary patterns, which simplifies 2D Gaussian deconvolution to a 1D Gaussian deconvolution problem with multiple observations. By capturing the set of patterns consecutively displayed on a smartphone, we formulate the feature extraction as a deconvolution problem to estimate feature point locations in sub-pixel accuracy and the blur kernel in each location. We also compensate the error in camera parameters due to refraction of the glass panel of the display device. We evaluate the performance of the proposed method on synthetic and real data. Even under severe defocus, our method shows accurate camera calibration result.
How human visual systems recognize objects - a novel computational model

This paper presents a novel computational model of 3D object recognition based on human visual system. Conventional schemes have feed forward structure based on the bottom-up process of human vision. However, psychological and physiological evidence suggests that top-down process and feature binding by visual attention are also important. Hence, we propose a method to integrate these facts under statistical framework, Markov chain Monte Carlo. In this scheme, object recognition is regarded as parameter optimization problem. The bottom-up process is used to initialize parameters and top-down process is used to optimize them. On both processes, feature map binding is performed by spatial attention mechanism. Experimental results show that the proposed computational model is feasible for 3D object recognition.
Difference-Based Image Noise Modeling Using Skellam Distribution

By the laws of quantum physics, pixel intensity does not have a true value, but should be a random variable. Contrary to the conventional assumptions, the distribution of intensity may not be an additive Gaussian. We propose to directly model the intensity difference and show its validity by an experimental comparison to the conventional additive model. As a model of the intensity difference, we present a Skellam distribution derived from the Poisson photon noise model. This modeling induces a linear relationship between intensity and Skellam parameters, while conventional variance computation methods do not yield any significant relationship between these parameters under natural illumination. The intensity-Skellam line is invariant to scene, illumination, and even most of camera parameters. We also propose practical methods to obtain the line using a color pattern and an arbitrary image under natural illumination. Because the Skellam parameters that can be obtained from this linearity determine a noise distribution for each intensity value, we can statistically determine whether any intensity difference is caused by an underlying signal difference or by noise. We demonstrate the effectiveness of this new noise model by applying it to practical applications of background subtraction and edge detection.
Robust self-localization of mobile robots using artificial and natural landmarks

This paper presents a novel localization paradigm for mobile robots based on artificial and natural landmarks. A model-based object recognition method detects natural landmarks and conducts the global and topological localization. In addition, a metric localization method using artificial landmarks is fused to complement the deficiency of topology map and guide to action behavior. The recognition algorithm uses a modified local Zernike moments and a probabilistic voting method for the robust detection of objects in cluttered indoor environments. An artificial landmark is designed to have a three-dimensional multi-colored structure and the projection distortion of the structure encodes the distance and viewing direction of the robot. We demonstrate the feasibility of the proposed system through real world experiments using a mobile robot KASIRI-III.
A Unified Approach of Multi-scale Deep and Hand-Crafted Features for Defocus Estimation

In this paper, we introduce robust and synergetic hand-crafted features and a simple but efficient deep feature from a convolutional neural network (CNN) architecture for defocus estimation. This paper systematically analyzes the effectiveness of different features, and shows how each feature can compensate for the weaknesses of other features when they are concatenated. For a full defocus map estimation, we extract image patches on strong edges sparsely, after which we use them for deep and hand-crafted feature extraction. In order to reduce the degree of patch-scale dependency, we also propose a multi-scale patch extraction strategy. A sparse defocus map is generated using a neural network classifier followed by a probability-joint bilateral filter. The final defocus map is obtained from the sparse defocus map with guidance from an edge-preserving filtered input image. Experimental results show that our algorithm is superior to state-of-the-art algorithms in terms of defocus estimation. Our work can be used for applications such as segmentation, blur magnification, all-in-focus image generation, and 3-D estimation.
Video^M: Multi-video Synopsis

Conventional video representation methods focus predominantly on a single video, aiming at reducing the space-time redundancy as much as possible, while this paper describes a novel approach to simultaneously presenting dynamics of multiple videos, aiming at a less intrusive viewing experience. Given a main video and multiple supplementary videos, the proposed approach automatically constructs a synthesized multi-video synopsis, called VideoM, by integrating the supplementary videos into the most suitable space-time portions within the main video. We formulate the problem of VideoM as a maximum a posterior (MAP) problem which maximizes the desired properties related to less intrusive viewing experience, i.e., informativeness, consistency, visual naturalness, and stability. This problem is solved by the Viterbi beam search algorithm to optimally find the suitable integration between the main video and supplementary videos.
Extracting topographic features for outdoor mobile robots

Methods are presented for building high-level terrain descriptions, referred as topographic maps, by extracting terrain features like peaks, pits, ridges, and ravines from the contour map. The resulting topographic map contains the location and type of terrain features as well as the ground topography. The authors develop new definitions for those topographic features based on the contour map. They build a contour map from an elevation map and generate the connectivity tree of all regions separated by the contours. The authors use this connectivity tree, called a topographic change tree, to extract the topographic features. Experimental results on a digital elevation model support the definitions for topographic features and the approach.<<ETX>>
Dense Depth and Albedo from a Single-Shot Structured Light

Single-shot structured light scanning has been actively investigated as it can recover accurate geometrical shape even on a dynamic scene. Since many single-shot approaches focus on improving depth accuracy, recovering the intrinsic properties of the scene such as albedo and shading are also valuable. In this paper, we propose a novel method that reconstructs not only the metric depth but also the intrinsic properties from a single structured light image. We extend the conventional color structured light model to embrace the Lambertian shading model. By using a color phase-shifting pattern, we parameterize the captured image with only two variables, albedo and depth. For an initial solution, a simple but powerful method to decompose sinusoids from the input image is presented. We formulate a non-linear cost function and jointly optimize albedo and depth efficiently by calculating analytic Jacobian. We demonstrate that our algorithm reasonably works on various real-world objects which exhibit challenging surface reflectance and albedo.
A semantic region descriptor for local feature based image categorization

Region descriptor has proved to be very important for local feature based image categorization. Previous region descriptors are usually based on the statistics of low level features, such as intensity, edge response, and etc. In this paper a novel descriptor named local texton statistics (LTS) that explores the high level semantic statistical characteristics of image regions is presented. Perceptual information is obtained by applying Gaussian filter banks and the image regions are described by the statistics of different 'texton's. Using the bag of words as classification algorithm, experiments show that the proposed descriptor is superior to the previous popular SIFT descriptors on the Wang dataset. The combination of these two descriptors shows high performance for categorization on both the Wang dataset and the fifteen scene categories dataset.
Artificial landmark tracking based on the color histogram

For the fast and accurate self-localization of mobile robots, landmarks can be used very efficiently in the complex workspace. In this paper, we propose a simple color landmark model for self-localization and a fast landmark detection and tracking algorithm based on the proposed landmark model. We develop a color landmark with symmetric and repetitive structures, which shows invariant color histogram characteristics under some geometric distortions. Detection and tracking of the model are accomplished by a factored sampling technique in which color similarity is estimated by the color histogram intersection. We also use the color similarity to update the color histogram model of the landmark model for robust tracking under illumination change. We demonstrate the feasibility of the proposed technique through experiments in cluttered indoor environments.
Deltille Grids for Geometric Camera Calibration

The recent proliferation of high resolution cameras presents an opportunity to achieve unprecedented levels of precision in visual 3D reconstruction. Yet the camera calibration pipeline, developed decades ago using checkerboards, has remained the de facto standard. In this paper, we ask the question: are checkerboards the optimal pattern for high precision calibration? We empirically demonstrate that deltille grids (regular triangular tiling) produce the highest precision calibration of the possible tilings of Euclidean plane. We posit that they should be the new standard for high-precision calibration and present a complete ecosystem for calibration using deltille grids including: (1) a highly precise corner detection algorithm based on polynomial surface fitting; (2) an indexing scheme based on polarities extracted from the fitted surfaces; and (3) a 2D coding system for deltille grids, which we refer to as DelTags, in lieu of conventional matrix barcodes. We demonstrate state-of-the-art performance and apply the full calibration ecosystem through the use of 3D calibration objects for multiview camera calibration.