nils-herrmann/clean_scirepeval huggingface dataset:
Improved block level adaptive quantization for high efficiency video coding

As the concept of block level adaptivity becomes an important feature in recent video CODECs, block level adaptive quantization (BLAQ) is being considered in the High Efficiency Video Coding (HEVC) standard. The BLAQ is based on the assumption that each block should have its own quantization parameter (QP), which can adapt to the local content of video sequences much better, and hence the video encoder with adaptive QP can perform a better perceptual quality. However, in the HEVC reference software, the BLAQ is required to obtain a proper QP for each block by the rate distortion optimization (RDO) scheme and so the computational complexity of the encoder increases significantly. In this paper, an improved BLAQ algorithm is proposed to obtain the adaptive QP for each block. The simulation results show that the proposed method can save more bits as well as require lower computational complexity, compared to the traditional method.
An Efficient Frame-Content Based Intra Frame Rate Control for High Efficiency Video Coding

Rate control plays an important role in the rapid development of high-fidelity video services. As the High Efficiency Video Coding (HEVC) standard has been finalized, many rate control algorithms are being developed to promote its commercial use. The HEVC encoder adopts a new R-lambda based rate control model to reduce the bit estimation error. However, the R-lambda model fails to consider the frame-content complexity that ultimately degrades the performance of the bit rate control. In this letter, a gradient based R-lambda (GRL) model is proposed for the intra frame rate control, where the gradient can effectively measure the frame-content complexity and enhance the performance of the traditional R-lambda method. In addition, a new coding tree unit (CTU) level bit allocation method is developed. The simulation results show that the proposed GRL method can reduce the bit estimation error and improve the video quality in HEVC all intra frame coding.
High quality image construction from multiple low quality copies

In this paper, the authors proposed to construct a high quality image based on multiple low quality input images. The relationship between one pixel and its neighbourhood should be consistent between different degraded images. Therefore, the reconstruction method is proposed by enforcing the pixel consistency property, which is ensured by estimating the parameters of piecewise image model for each pixel. Subsequently, the reconstructed coefficients are regularized within a reasonable range. Experimental results on multiple images (with different distortions of different levels) have demonstrated that the proposed method can effectively alleviate the noises meanwhile preserve the detailed information. Better quality images in terms of both objective and subjective measurements can be generated.
Color fidelity of chromatic distributions by triad illuminant comparison

Performance measures for quantifying human color constancy and computational color constancy are very different. The former relate to measurements on individual object colors whereas the latter relate to the accuracy of the estimated illuminant. To bridge this gap, we propose a psychophysical method in which observers judge the global color fidelity of the visual scene rendered under different illuminants. In each experimental trial, the scene is rendered under three illuminants, two chromatic test illuminants and one neutral reference illuminant. Observers indicate which of the two test illuminants leads to better color fidelity in comparison to the reference illuminant. Here we study multicolor scenes with chromatic distributions that are differently oriented in color space, while having the same average chromaticity. We show that when these distributions are rendered under colored illumination they lead to different perceptual estimates of the color fidelity.
A Statistical Method for 2-D Facial Landmarking

Many facial-analysis approaches rely on robust and accurate automatic facial landmarking to correctly function. In this paper, we describe a statistical method for automatic facial-landmark localization. Our landmarking relies on a parsimonious mixture model of Gabor wavelet features, computed in coarse-to-fine fashion and complemented with a shape prior. We assess the accuracy and the robustness of the proposed approach in extensive cross-database conditions conducted on four face data sets (Face Recognition Grand Challenge, Cohn-Kanade, Bosphorus, and BioID). Our method has 99.33% accuracy on the Bosphorus database and 97.62% accuracy on the BioID database on the average, which improves the state of the art. We show that the method is not significantly affected by low-resolution images, small rotations, facial expressions, and natural occlusions such as beard and mustache. We further test the goodness of the landmarks in a facial expression recognition application and report landmarking-induced improvement over baseline on two separate databases for video-based expression recognition (Cohn-Kanade and BU-4DFE).
Selection and fusion of color models for feature detection

The choice of a color space is of great importance for many computer vision algorithms (e.g. edge detection and object recognition). It induces the equivalence classes to the actual algorithms. However, the problem is how to automatically select the color space that produces the best result for a particular task. The subsequent difficulty then is how to obtain a proper weighting scheme for the algorithms so that the results are combined in an optimal setting. To achieve proper color space selection and fusion of feature detectors, in this paper, we propose a method that exploits non-perfect correlation between the color models derived from the principles of diversification. As a consequence, the weighting scheme yields maximal color discrimination. The method is verified experimentally for two different feature detectors. The experimental results show that the model provides feature detection results having a discriminative power of 30 percent higher than the standard weighting scheme.
Image retrieval and segmentation based on color invariants

We will demonstrate our CVPR2000 paper "Measurement of Color Invariants" for the cases of image retrieval based on query by example and for color image segmentation. Both are of importance in content based access of image and video data. We demonstrate the usefulness of the proposed color invariants in image retrieval by example systems. We show that an image retrieval query should include the type of invariance expected in the result. We demonstrate such queries by using the "ImageSurf" retrieval system. Segmentation of images based on the proposed color invariants is demonstrated by the "PicToVision" system. The system provides image processing functionality through the world wide web, and is publicly accessible at www.science. uva.nl/research/isis.pictovision.html.
Combining Priors, Appearance, and Context for Road Detection

Detecting the free road surface ahead of a moving vehicle is an important research topic in different areas of computer vision, such as autonomous driving or car collision warning. Current vision-based road detection methods are usually based solely on low-level features. Furthermore, they generally assume structured roads, road homogeneity, and uniform lighting conditions, constraining their applicability in real-world scenarios. In this paper, road priors and contextual information are introduced for road detection. First, we propose an algorithm to estimate road priors online using geographical information, providing relevant initial information about the road location. Then, contextual cues, including horizon lines, vanishing points, lane markings, 3-D scene layout, and road geometry, are used in addition to low-level cues derived from the appearance of roads. Finally, a generative model is used to combine these cues and priors, leading to a road detection method that is, to a large degree, robust to varying imaging conditions, road types, and scenarios.
Expression-Invariant Age Estimation Using Structured Learning

In this paper, we investigate and exploit the influence of facial expressions on automatic age estimation. Different from existing approaches, our method jointly learns the age and expression by introducing a new graphical model with a latent layer between the age/expression labels and the features. This layer aims to learn the relationship between the age and expression and captures the face changes which induce the aging and expression appearance, and thus obtaining expression-invariant age estimation. Conducted on three age-expression datasets (FACES  , Lifespan [2] and NEMO  ), our experiments illustrate the improvement in performance when the age is jointly learnt with expression in comparison to expression-independent age estimation. The age estimation error is reduced by 14.43, 37.75 and 9.30 percent for the FACES, Lifespan and NEMO datasets respectively. The results obtained by our graphical model, without prior-knowledge of the expressions of the tested faces, are better than the best reported ones for all datasets. The flexibility of the proposed model to include more cues is explored by incorporating gender together with age and expression. The results show performance improvements for all cues.
Color Constancy Using Natural Image Statistics and Scene Semantics

Existing color constancy methods are all based on specific assumptions such as the spatial and spectral characteristics of images. As a consequence, no algorithm can be considered as universal. However, with the large variety of available methods, the question is how to select the method that performs best for a specific image. To achieve selection and combining of color constancy algorithms, in this paper natural image statistics are used to identify the most important characteristics of color images. Then, based on these image characteristics, the proper color constancy algorithm (or best combination of algorithms) is selected for a specific image. To capture the image characteristics, the Weibull parameterization (e.g., grain size and contrast) is used. It is shown that the Weibull parameterization is related to the image attributes to which the used color constancy methods are sensitive. An MoG-classifier is used to learn the correlation and weighting between the Weibull-parameters and the image attributes (number of edges, amount of texture, and SNR). The output of the classifier is the selection of the best performing color constancy method for a certain image. Experimental results show a large improvement over state-of-the-art single algorithms. On a data set consisting of more than 11,000 images, an increase in color constancy performance up to 20 percent (median angular error) can be obtained compared to the best-performing single algorithm. Further, it is shown that for certain scene categories, one specific color constancy algorithm can be used instead of the classifier considering several algorithms.
Combining Facial Dynamics With Appearance for Age Estimation

Estimating the age of a human from the captured images of his/her face is a challenging problem. In general, the existing approaches to this problem use appearance features only. In this paper, we show that in addition to appearance information, facial dynamics can be leveraged in age estimation. We propose a method to extract and use dynamic features for age estimation, using a person's smile. Our approach is tested on a large, gender-balanced database with 400 subjects, with an age range between 8 and 76. In addition, we introduce a new database on posed disgust expressions with 324 subjects in the same age range, and evaluate the reliability of the proposed approach when used with another expression. State-of-the-art appearance-based age estimation methods from the literature are implemented as baseline. We demonstrate that for each of these methods, the addition of the proposed dynamic features results in statistically significant improvement. We further propose a novel hierarchical age estimation architecture based on adaptive age grouping. We test our approach extensively, including an exploration of spontaneous versus posed smile dynamics, and gender-specific age estimation. We show that using spontaneity information reduces the mean absolute error by up to 21%, advancing the state of the art for facial age estimation.
Age estimation under changes in image quality: An experimental study

In this paper, we investigate the influence of image quality on the performance of aging features. Age estimation systems used or designed a number of aging features to capture the aging cues from the face such as skin texture and wrinkles. These aging cues are sensitive to small changes in the imaging conditions which suggests considering the imaging quality when extracting such information. Although interesting performances are reported on various datasets, the effect of image quality has not been addressed. We introduce a scheme to explore the influence of image quality on the performance of appearance aging features. A number of datasets are experimented on where artifacts resulted from different types of noise are considered. Finally, we propose a method to automatically apply the most suitable features based on the quality of the image. The results show that better or comparable performance is obtained when automatically applying different features, based on image quality, in comparison to a single (best) feature type.
Edge-Based Color Constancy

Color constancy is the ability to measure colors of objects independent of the color of the light source. A well-known color constancy method is based on the gray-world assumption which assumes that the average reflectance of surfaces in the world is achromatic. In this paper, we propose a new hypothesis for color constancy namely the gray-edge hypothesis, which assumes that the average edge difference in a scene is achromatic. Based on this hypothesis, we propose an algorithm for color constancy. Contrary to existing color constancy algorithms, which are computed from the zero-order structure of images, our method is based on the derivative structure of images. Furthermore, we propose a framework which unifies a variety of known (gray-world, max-RGB, Minkowski norm) and the newly proposed gray-edge and higher order gray-edge algorithms. The quality of the various instantiations of the framework is tested and compared to the state-of-the-art color constancy methods on two large data sets of images recording objects under a large number of different light sources. The experiments show that the proposed color constancy algorithms obtain comparable results as the state-of-the-art color constancy methods with the merit of being computationally more efficient.
Color edge detection by photometric quasi-invariants

Photometric invariance is used in many computer vision applications. The advantage of photometric invariance is the robustness against shadows, shading and illumination conditions. However, the drawbacks of photometric invariance are the loss of discriminative power and the inherent instabilities caused by the nonlinear transformations to compute the invariants. In this paper, we propose a new class of derivatives which we refer to as photometric quasi-invariants. These quasi-invariants share with full invariants the nice property that they are robust against photometric edges, such as shadows or specular edges. Further, these quasi-invariants do not have the inherent instabilities of full photometric invariants. We will apply these quasi-invariant derivatives in the context of photometric invariant edge detection and classification. Experiments show that the quasi-invariant derivatives are stable and they significantly outperform the full invariant derivatives in discriminative power.
Evaluation of Color STIPs for Human Action Recognition

This paper is concerned with recognizing realistic human actions in videos based on spatio-temporal interest points (STIPs). Existing STIP-based action recognition approaches operate on intensity representations of the image data. Because of this, these approaches are sensitive to disturbing photometric phenomena such as highlights and shadows. Moreover, valuable information is neglected by discarding chromaticity from the photometric representation. These issues are addressed by Color STIPs. Color STIPs are multi-channel reformulations of existing intensity-based STIP detectors and descriptors, for which we consider a number of chromatic representations derived from the opponent color space. This enhanced modeling of appearance improves the quality of subsequent STIP detection and description. Color STIPs are shown to substantially outperform their intensity-based counterparts on the challenging UCF~sports, UCF11 and UCF50 action recognition benchmarks. Moreover, the results show that color STIPs are currently the single best low-level feature choice for STIP-based approaches to human action recognition.
PicToSeek: combining color and shape invariant features for image retrieval

We aim at combining color and shape invariants for indexing and retrieving images. To this end, color models are proposed independent of the object geometry, object pose, and illumination. From these color models, color invariant edges are derived from which shape invariant features are computed. Computational methods are described to combine the color and shape invariants into a unified high-dimensional invariant feature set for discriminatory object retrieval. Experiments have been conducted on a database consisting of 500 images taken from multicolored man-made objects in real world scenes. From the theoretical and experimental results it is concluded that object retrieval based on composite color and shape invariant features provides excellent retrieval accuracy. Object retrieval based on color invariants provides very high retrieval accuracy whereas object retrieval based entirely on shape invariants yields poor discriminative power. Furthermore, the image retrieval scheme is highly robust to partial occlusion, object clutter and a change in the object's pose. Finally, the image retrieval scheme is integrated into the PicToSeek system on-line at http://www.wins.uva.nl/research/isis/PicToSeek/ for searching images on the World Wide Web.
Vision-based road detection using road models

Vision-based road detection is very challenging since the road is in an outdoor scenario imaged from a mobile platform. In this paper, a new top-down road detection algorithm is proposed. The method is based on scene (road) classification which provides the probability that an image contains certain type of road geometry (straight, left/right curve, etc.). During the training of the classifier a road probability map is also learned for each road geometry. Then, the proper pixel-based method is selected and fused to provide an improved road detection approach. From experiments it is concluded that the proposed method outperforms state-of-the-art algorithms in a frame by frame context.
Adaptive Image Segmentation by Combining Photometric Invariant Region and Edge Information

An adaptive image segmentation scheme is proposed employing the Delaunay triangulation for image splitting. The tessellation grid of the Delaunay triangulation is adapted to the semantics of the image data by combining region and edge information. To achieve robustness against imaging conditions (e.g. shading, shadows, illumination and highlights), photometric invariant similarity measures and edge computation are proposed. Experimental results on synthetic and real images show that the segmentation method is robust to edge orientation, partially weak object boundaries and noisy-but-homogeneous regions. Furthermore, the method is robust, to a large degree, to varying imaging conditions.
Invariance in content-based retrieval

The paper discusses the role of invariance in content-based image retrieval, and discusses future directions. In the repertoire of images under consideration, a narrow domain is a set of images containing a limited variety in the object's appearances. In a narrow domain, the recording circumstances typically are similar or standardized over the whole domain. On the other end of the spectrum is a broad domain with a large variety in recording circumstances and appearances. The sensory gap between the object in the world and the information captured from a recording causes unwanted variance in the description of an object. We list conditions of variance, and we review some of features which are invariant under these conditions.
Improving Color Constancy by Photometric Edge Weighting

Edge-based color constancy methods make use of image derivatives to estimate the illuminant. However, different edge types exist in real-world images, such as material, shadow, and highlight edges. These different edge types may have a distinctive influence on the performance of the illuminant estimation. Therefore, in this paper, an extensive analysis is provided of different edge types on the performance of edge-based color constancy methods. First, an edge-based taxonomy is presented classifying edge types based on their photometric properties (e.g., material, shadow-geometry, and highlights). Then, a performance evaluation of edge-based color constancy is provided using these different edge types. From this performance evaluation, it is derived that specular and shadow edge types are more valuable than material edges for the estimation of the illuminant. To this end, the (iterative) weighted Gray-Edge algorithm is proposed in which these edge types are more emphasized for the estimation of the illuminant. Images that are recorded under controlled circumstances demonstrate that the proposed iterative weighted Gray-Edge algorithm based on highlights reduces the median angular error with approximately 25 percent. In an uncontrolled environment, improvements in angular error up to 11 percent are obtained with respect to regular edge-based color constancy.
Color Constancy using Image Regions

Color constancy is important for various applications such as image segmentation, object recognition and image retrieval where object color features are extracted invariant to the illumination conditions. Different color constancy methods have been proposed. These methods, in general, compute color constancy based on all image colors. However, not all pixels contain relevant information for color constancy. Eventually, biased pixel values may decrease the performance of color constancy methods. To this end, in this paper, we propose a method based on low-level image features using subsets of pixels. Hence, instead of using the entire pixel set for estimating the illuminant, only relevant pixels in the image are used. Therefore, prior segmentation is performed to learn for different image categories (e.g. open country, street, indoor) which pixel set (i.e. image parts) is most appropriate for a reliable estimation. Based on large scale experiments on real-world scenes, it can be derived that for certain categories, like open country and street, the estimation is far more accurate using image parts than when using the entire image.