nils-herrmann/clean_scirepeval huggingface dataset:
High dynamic range image rendering with a retinex-based adaptive filter

We propose a new method to render high dynamic range images that models global and local adaptation of the human visual system. Our method is based on the center-surround Retinex model. The novelties of our method is first to use an adaptive filter, whose shape follows the image high-contrast edges, thus reducing halo artifacts common to other methods. Second, only the luminance channel is processed, which is defined by the first component of a principal component analysis. Principal component analysis provides orthogonality between channels and thus reduces the chromatic changes caused by the modification of luminance. We show that our method efficiently renders high dynamic range images and we compare our results with the current state of the art
A practical method for measuring the spatial frequency response of light field cameras

The spatial frequency response (SFR) is one of the most important and unbiased image quality measures of a digital camera. It evaluates to which extent a lens/sensor combination can resolve scene details. In this paper, we propose a simple and practical method to measure the SFR of microlens-based light field cameras. The particularity of such cameras resides in their ability to capture both spatial and angular information of the incoming light field thanks to an array of microlenses located in front of the sensor. Existing methods for measuring the SFR of conventional cameras are thus no longer applicable as the interaction between the main lens and the micro-lenses results in different resolving powers over the image plane that depend on the scene depths. By using a 3-dimensional target made of vertical lines printed on an inclined planar surface, we are able to measure the SFR across multiple depths in a single exposure. Our method allows SFR measurements from the raw light field itself as captured by the camera, and is thus independent of subsequent post-processing algorithms such as image reconstruction, digital refocusing or depth estimation. Our experimental results are consistent with theoretical bounds and reproducible.
Appearance-based keypoint clustering

We present an algorithm for clustering sets of detected interest points into groups that correspond to visually distinct structure. Through the use of a suitable colour and texture representation, our clustering method is able to identify keypoints that belong to separate objects or background regions. These clusters are then used to constrain the matching of keypoints over pairs of images, resulting in greatly improved matching under difficult conditions. We present a thorough evaluation of each component of the algorithm, and show its usefulness on difficult matching problems.
Saliency Detection using regression trees on hierarchical image segments

The currently best performing state-of-the-art saliency detection algorithms incorporate heuristic functions to evaluate saliency. They require parameter tuning, and the relationship between the parameter value and visual saliency is often not well understood. Instead of using parametric methods we follow a machine learning approach, which is parameter free, to estimate saliency. Our method learns data-driven saliency-estimation functions and exploits the contributions of visual properties on saliency. First, we over-segment the image into superpixels and iteratively connect them to form hierarchical image segments. Second, from these segments, we extract biologically-plausible visual features. Finally, we use regression trees to learn the relationship between the feature values and visual saliency. We show that our algorithm outperforms the most recent state-of-the-art methods on three public databases.
Multispectral interest points for RGB-NIR image registration

This paper explores the use of joint colour and near-infrared (NIR) information for feature based matching and image registration. In particular, we investigate multispectral generalisations of two popular interest point detectors (Harris and difference of Gaussians), and show that these give a marked improvement in performance when the extra NIR channel is available. We also look at the problem of multimodal RGB to NIR registration, and propose a variant of the SIFT descriptor that gives improved performance.
SLIC Superpixels Compared to State-of-the-Art Superpixel Methods

Computer vision applications have come to rely increasingly on superpixels in recent years, but it is not always clear what constitutes a good superpixel algorithm. In an effort to understand the benefits and drawbacks of existing methods, we empirically compare five state-of-the-art superpixel algorithms for their ability to adhere to image boundaries, speed, memory efficiency, and their impact on segmentation performance. We then introduce a new superpixel algorithm, simple linear iterative clustering (SLIC), which adapts a k-means clustering approach to efficiently generate superpixels. Despite its simplicity, SLIC adheres to boundaries as well as or better than previous methods. At the same time, it is faster and more memory efficient, improves segmentation performance, and is straightforward to extend to supervoxel generation.
Near-infrared guided color image dehazing

Near-infrared (NIR) light has stronger penetration capability than visible light due to its long wavelengths and is thus less scattered by particles in the air. This makes it desirable for image dehazing to unveil details of distant objects in landscape photographs. In this paper, we propose an improved image dehazing scheme using a pair of color and NIR images, which effectively estimates the airlight color and transfers details from the NIR. A two-stage dehazing method is proposed by exploiting the dissimilarity between RGB and NIR for airlight color estimation, followed by a dehazing procedure through an optimization framework. Experiments on captured haze images show that our method can achieve substantial improvements on the detail recovery and the color distribution over the existing image dehazing algorithms.
Learning to see through reflections

Pictures of objects behind a glass are difficult to interpret and understand due to the superposition of two real images: a reflection layer and a background layer. Separation of these two layers is challenging due to the ambiguities in assigning texture patterns and the average color in the input image to one of the two layers. In this paper, we propose a novel method to reconstruct these layers given a single input image by explicitly handling the ambiguities of the reconstruction. Our approach combines the ability of neural networks to build image priors on large image regions with an image model that accounts for the brightness ambiguity and saturation. We find that our solution generalizes to real images even in the presence of strong reflections. Extensive quantitative and qualitative experimental evaluations on both real and synthetic data show the benefits of our approach over prior work. Moreover, our proposed neural network is computationally and memory efficient.
Superpixels and Polygons Using Simple Non-iterative Clustering

We present an improved version of the Simple Linear Iterative Clustering (SLIC) superpixel segmentation. Unlike SLIC, our algorithm is non-iterative, enforces connectivity from the start, requires lesser memory, and is faster. Relying on the superpixel boundaries obtained using our algorithm, we also present a polygonal partitioning algorithm. We demonstrate that our superpixels as well as the polygonal partitioning are superior to the respective state-of-the-art algorithms on quantitative benchmarks.
Saliency detection for content-aware image resizing

Content aware image re-targeting methods aim to arbitrarily change image aspect ratios while preserving visually prominent features. To determine visual importance of pixels, existing re-targeting schemes mostly rely on grayscale intensity gradient maps. These maps show higher energy only at edges of objects, are sensitive to noise, and may result in deforming salient objects. In this paper, we present a computationally efficient, noise robust re-targeting scheme based on seam carving by using saliency maps that assign higher importance to visually prominent whole regions (and not just edges). This is achieved by computing global saliency of pixels using intensity as well as color features. Our saliency maps easily avoid artifacts that conventional seam carving generates and are more robust in the presence of noise. Also, unlike gradient maps, which may have to be recomputed several times during a seam carving based re-targeting operation, our saliency maps are computed only once independent of the number of seams added or removed.
Correlation-based deblurring leveraging multispectral chromatic aberration in color and near-infrared joint acquisition

Joint acquisition of color and near-infrared (NIR) images is of growing interest due to various applications that make use of the additional spectral information. An obstacle to this acquisition is the wavelength-dependent blurring caused by the chromatic aberration of optical lenses. When one of the spectral channels, for example the green channel, is in focus on the sensor plane, the images of the other channels, especially NIR, are blurred. This paper presents a study of spectral-spatial correlations between color and NIR channels and proposes a method to correct for chromatic aberrations. The algorithm we introduce leverages axial chromatic aberration to deblur the NIR image when the color image is in focus. The proposed technique improves image sharpness by 48.8% on average compared to state-of-the-art results. Moreover, our method generates an NIR image that has a larger depth-of-field compared to an NIR image originally captured in focus.
Compression of multispectral images: Color (RGB) plus near-infrared (NIR)

We propose a compression framework for four-channel images, composed of color (RGB) and near-infrared (NIR) channels, which exploits the correlation between the visible and the NIR information. The high-frequency components of both visible and NIR scene representations are strongly correlated. By encoding only the DCT components that differ above a chosen threshold, we significantly improve compression ratios for a given quality level. To evaluate our proposed method, we compare our results with standard JPEG compression, as well as PCA-based approaches that are often employed to compress multispectral images. Our experiments show that applying our proposed method yields the same quality at a lower bit-rate, compared to conventional JPEG and PCA-based algorithms.
Linear demosaicing inspired by the human visual system

There is an analogy between single-chip color cameras and the human visual system in that these two systems acquire only one limited wavelength sensitivity band per spatial location. We have exploited this analogy, defining a model that characterizes a one-color per spatial position image as a coding into luminance and chrominance of the corresponding three colors per spatial position image. Luminance is defined with full spatial resolution while chrominance contains subsampled opponent colors. Moreover, luminance and chrominance follow a particular arrangement in the Fourier domain, allowing for demosaicing by spatial frequency filtering. This model shows that visual artifacts after demosaicing are due to aliasing between luminance and chrominance and could be solved using a preprocessing filter. This approach also gives new insights for the representation of single-color per spatial location images and enables formal and controllable procedures to design demosaicing algorithms that perform well compared to concurrent approaches, as demonstrated by experiments.
Dynamic Texture Synthesis: Compact Models Based on Luminance-Chrominance Color Representation

Dynamic textures are sequences of images showing temporal regularity. Examples can be found in videos representing smoke, flames, ocean waves, wind-shaken forests, etc. Dynamic texture modelling and synthesis has usually been done considering RGB color images. In this paper, we analyze the use of different color encodings, which permit to model luminance and chrominance information separately. We find that this separation is more appropriate, since it takes advantage of the spatial and temporal characteristics of the color channels and leads to more flexible and compact representations. We show that compared to RGB, similar synthesis performance can be achieved using YCbCr or Lab color encodings, using half of the model coefficients and less computational power.
Multi-spectral SIFT for scene category recognition

We use a simple modification to a conventional SLR camera to capture images of several hundred scenes in colour (RGB) and near-infrared (NIR). We show that the addition of near-infrared information leads to significantly improved performance in a scene-recognition task, and that the improvements are greater still when an appropriate 4-dimensional colour representation is used. In particular we propose MSIFT–a multispectral SIFT descriptor that, when combined with a kernel based classifier, exceeds the performance of state-of-the-art scene recognition techniques (e.g., GIST) and their multispectral extensions. We extensively test our algorithms using a new dataset of several hundred RGB-NIR scene images, as well as benchmarking against Torralba's scene categorization dataset.
Spatio-chromatic decorrelation by shift-invariant filtering

In this paper we derive convolutional filters for colour image whitening and decorrelation. Whilst whitening can be achieved via eigendecomposition of the image patch co-variance, this operation is neither efficient nor biologically plausible. Given the shift invariance of image statistics, the covariance matrix contains repeated information which can be eliminated by solving directly for a per pixel linear operation (convolution). We formulate decorrelation as a shift and rotation invariant filtering operation and solve directly for the filter shape via non-linear least squares. This results in opponent-colour lateral inhibition filters which resemble those found in the human visual system. We also note the similarity of these filters to current interest point detectors, and perform an experimental evaluation of their use in this context.
Non-Parametric Blur Map Regression for Depth of Field Extension

Real camera systems have a limited depth of field (DOF) which may cause an image to be degraded due to visible misfocus or too shallow DOF. In this paper, we present a blind deblurring pipeline able to restore such images by slightly extending their DOF and recovering sharpness in regions slightly out of focus. To address this severely ill-posed problem, our algorithm relies first on the estimation of the spatially varying defocus blur. Drawing on local frequency image features, a machine learning approach based on the recently introduced regression tree fields is used to train a model able to regress a coherent defocus blur map of the image, labeling each pixel by the scale of a defocus point spread function. A non-blind spatially varying deblurring algorithm is then used to properly extend the DOF of the image. The good performance of our algorithm is assessed both quantitatively, using realistic ground truth data obtained with a novel approach based on a plenoptic camera, and qualitatively with real images.
Keyword-based image color re-rendering with semantic segmentation

Keyword-based image color re-rendering is a convenient way to enhance the color of images. Most methods only focus on the color characteristics of the image while ignoring the semantic meaning of different regions. We propose to incorporate semantic information into the color re-rendering pipeline through semantic segmentation. Using semantic segmentation masks, we first generate more accurate correlations between keywords and color characteristics than the state-of-the-art approach. Such correlations are then adopted for rerendering the color of the input image, where the segmentation masks are used to indicate the regions for color rerendering. Qualitative comparisons show that our method generates visually better results than the state-of-the-art approach. We further validate with a psychophysical experiment, where the participants prefer the results of our method.
Consistent image-based measurement and classification of skin color

Little prior image processing work has addressed estimation and classification of skin color in a manner that is independent of camera and illuminant. To this end, we first present new methods for 1) fast, easy-to-use image color correction, with specialization toward skin tones, and 2) fully automated estimation of facial skin color, with robustness to shadows, specularities, and blemishes. Each of these is validated independently against ground truth, and then combined with a classification method that successfully discriminates skin color across a population of people imaged with several different cameras. We also evaluate the effects of image quality and various algorithmic choices on our classification performance. We believe our methods are practical for relatively untrained operators, using inexpensive consumer equipment.
GUI-aided NIR and color image blending

A near-infrared (NIR) image of a scene contains different information than the corresponding visible spectrum (RGB) image, due to physical phenomena related to scattering, absorption, and reflectance of the different radiation bands. It can be desirable to combine the most relevant image content from both NIR and RGB to improve image quality. In this paper, we present two schemes of combining NIR and RGB information to obtain visually better quality images. The first is an automatic approach that performs segment-based NIR-blending by local and global entropy maximization. The second uses a GUI-aided approach where the user chooses the segments and manually controls blending. To minimize artifacts at the segment boundaries, three different boundary smoothing methods are introduced and compared in terms of speed and quality of output.
Face image enhancement using 3D and spectral information

This paper presents a novel method of enhancing image quality of face pictures using 3D and spectral information. Most conventional techniques directly work on the image data, shifting the skin color to a predefined skin tone, and thus do not take into account the effects of shape and lighting. The proposed method first recovers the 3D shape of a face in an input image using a 3D morphable model. Then, using color constancy and inverse rendering techniques, specularities and the true skin color, i.e., its spectral reflectance, are recovered. The quality of the input image is improved by matching the skin reflectance to a predefined reference and reducing the amount of specularities. The method realizes the enhancement in a more physically accurate manner compared to previous ones. Subjective experiments on image quality demonstrate the validity of the proposed method.
Saliency detection using maximum symmetric surround

Detection of visually salient image regions is useful for applications like object segmentation, adaptive compression, and object recognition. Recently, full-resolution salient maps that retain well-defined boundaries have attracted attention. In these maps, boundaries are preserved by retaining substantially more frequency content from the original image than older techniques. However, if the salient regions comprise more than half the pixels of the image, or if the background is complex, the background gets highlighted instead of the salient object. In this paper, we introduce a method for salient region detection that retains the advantages of such saliency maps while overcoming their shortcomings. Our method exploits features of color and luminance, is simple to implement and is computationally efficient. We compare our algorithm to six state-of-the-art salient region detection methods using publicly available ground truth. Our method outperforms the six algorithms by achieving both higher precision and better recall. We also show application of our saliency maps in an automatic salient object segmentation scheme using graph-cuts.
Simultaneous Geometric and Radiometric Calibration of a Projector-Camera Pair

We present a novel method that allows for simultaneous geometric and radiometric calibration of a projector-camera pair. It is simple and does not require specialized hardware. We prewarp and align a specially designed projection pattern onto a printed pattern of different colorimetric properties. After capturing the patterns in several orientations, we perform geometric calibration by estimating the corner locations of the two patterns in different color channels. We perform radiometric calibration of the projector by using the information contained inside the projected squares. We show that our method performs on par with current approaches that all require separate geometric and radiometric calibration, while being more efficient and user friendly.
Revisiting radiometric calibration for color computer vision

We present a study of radiometric calibration and the in-camera imaging process through an extensive analysis of more than 10,000 images from over 30 cameras. The goal is to investigate if image values can be transformed to physically meaningful values and if so, when and how this can be done. From our analysis, we show that the conventional radiometric model fits well for image pixels with low color saturation but begins to degrade as color saturation level increases. This is due to the color mapping process which includes gamut mapping in the in-camera processing that cannot be modeled with conventional methods. To this end, we introduce a new imaging model for radiometric calibration and present an effective calibration scheme that allows us to compensate for the nonlinear color correction to convert non-linear sRGB images to CCD RAW responses.
Webly Supervised Semantic Segmentation

We propose a weakly supervised semantic segmentation algorithm that uses image tags for supervision. We apply the tags in queries to collect three sets of web images, which encode the clean foregrounds, the common backgrounds, and realistic scenes of the classes. We introduce a novel three-stage training pipeline to progressively learn semantic segmentation models. We first train and refine a class-specific shallow neural network to obtain segmentation masks for each class. The shallow neural networks of all classes are then assembled into one deep convolutional neural network for end-to-end training and testing. Experiments show that our method notably outperforms previous state-of-the-art weakly supervised semantic segmentation approaches on the PASCAL VOC 2012 segmentation benchmark. We further apply the class-specific shallow neural networks to object segmentation and obtain excellent results.
Mirror Match: Reliable Feature Point Matching without Geometric Constraints

Many algorithms have been proposed to solve the problem of matching feature points in two or more images using geometric assumptions to increase the robustness of the matching. However, these assumptions do not always hold, in particular, few methods address the problem of reliable matching in cases where it is unknown whether the images have any corresponding areas or objects in the first place. We propose two algorithms for matching feature points without the use of geometric constraints. The first relies on the fact that any match between two images should be better than all possible matches within a single image. The second algorithm extends this idea by using community structure in the similarity graph of feature points to find reliable correspondences. To evaluate the algorithms experimentally, we introduce a simple method to generate a large amount of test cases based on a set of image pairs with viewpoint changes. Our results show that the proposed algorithm is generally superior to traditional approaches in finding correct correspondences.