nils-herrmann/clean_scirepeval huggingface dataset:
Blind Image Quality Assessment Using Local Consistency Aware Retriever and Uncertainty Aware Evaluator

Blind image quality assessment (BIQA) aims to automatically predict the perceptual quality of a digital image without accessing its pristine reference. Previous studies mainly focus on extracting various quality-relevant image features. By contrast, the explorations on highly efficient learning model are still very limited. Motivated by the fact that it is difficult to approximate a complex and large data set via a global parametric model, we propose a novel local learning method for BIQA to improve quality prediction performance. More specifically, we search for the perceptually similar neighbors of a test image to serve as its unique training set. Unlike the widely used  ${k}$  nearest neighbors principle, which only measures the similarity between the testing and training samples, the local consistency of the selected training data is also considered to generate smoother sample space. The image quality is estimated via a sparse Gaussian process. As an additional benefit, the uncertainty of the predicted score is jointly inferred, which can subsequently drive more robust perceptual image processing applications, such as deblocking investigated in this paper. Extensive experiments demonstrate that the proposed learning model leads to consistent quality prediction improvements over many state-of-the-art BIQA algorithms.
Image Cosegmentation by Incorporating Color Reward Strategy and Active Contour Model

The design of robust and efficient cosegmentation algorithms is challenging because of the variety and complexity of the objects and images. In this paper, we propose a new cosegmentation model by incorporating a color reward strategy and an active contour model. A new energy function corresponding to the curve is first generated with two considerations: the foreground similarity between the image pairs and the background consistency in each of the image pair. Furthermore, a new foreground similarity measurement based on the rewarding strategy is proposed. Then, we minimize the energy function value via a mutual procedure which uses dynamic priors to mutually evolve the curves. The proposed method is evaluated on many images from commonly used databases. The experimental results demonstrate that the proposed model can efficiently segment the common objects from the image pairs with generally lower error rate than many existing and conventional cosegmentation methods.
Region-based image retargeting quality assessment

The tremendous growth in mobile devices has resulted in huge generation and consumption of digital images. The image quality evaluation is hence an important issue for mobile media applications. In this work, we focus on the quality evaluation of images under content-aware retargeting, in which the sizes of images may differ from each other. Shape distortion and content information loss are two main factors in the evaluation of retargeting quality . We have proposed a region-based framework to evaluate shape distortion and information loss caused by retargeting. The proposed algorithm is validated on the public database RetargetMe . Experimental results demonstrate that this region-based quality assessment algorithm outperforms others based on pixels.
Hybrid cascade of active/lazy boosting

In this paper, we present an active boosting algorithm to learn the object detector. This algorithm is to find good features from a confidential map instead of brute-force searching the predefined feature set. The confidential map is computed from the importance re-sampled data. A new feature is created by the linear combination of blocks that are selected from different segmented regions. In addition, lazy boosting associated with the hybrid cascade is developed to speed up the object detection. Experimental results demonstrate the effectiveness of our proposed method that can achieve good performance for the face detection.
Foreground/background video coding scheme

This paper presents the use of segmentation to improve the subjective quality of the sequence produced by very low bit rate video coding system. Through this approach, each frame of the source sequence is first segmented into two non-overlapping regions, namely foreground and background. These two regions are then encoded using the same coder but with different quantization step-sizes. In this way, the image quality of the foreground region can be improved at the expense of encoding the unimportant background region at lower quality. Currently, our work focuses on integrating this approach into the H.263 coder, primarily for the videotelephony application. In this paper, we describe the working implementation, and also demonstrate the improved subjective quality of the coded sequence achieved.
Object-based disparity estimation for stereoscopic images

Disparity estimation has been a critical procedure in stereoscopic or multiview video processing. In this paper, we present an object-based disparity estimation algorithm capable of providing an accurate disparity map from a stereo pair effectively. The algorithm involves a preprocessing step, which identifies the foreground object, and a predictive hierarchical block matching with different parameters for foreground and background. Left-to-right disparity map is calculated, the right image is used as a reference and the left image is reconstructed at the receiver end. The algorithm has been tested and the quality of the reconstruction using the disparity map demonstrates its effectiveness.
Visual quality metric for perceptual video coding

The visual quality assessment (VQA) becomes prevailing in the studies of image and video coding. It assesses the quality of image or video more accurately than mean square error (MSE) with respect to the human visual system (HVS). Toward perceptual video coding, MSE is weighted spatially and temporally to simulate the HVS response to visual signal in this paper. Firstly, the image content is depicted by edge strength to compose spatial weighting factors. Secondly, the motion strength calculated from motion vector of each block gives temporal weighting factors. Thirdly, the motion trajectory based saliency map for video signal is integrated as another weighting factor of MSE. The proposed VQM not only efficiently model HVS but also relate to quantization parameter (QP) capable of guiding perceptual video coding. A perceptual rate distortion optimization (RDO) is established on the proposed VQM. The experimental results indicate that the proposed VQM is consistent well with HVS. In addition, the better rate-distortion efficiency and accurate bit rate control can be achieved by the proposed visual quality control algorithm.
Error probabilities for sequency majority multiplexing in frequency-nonselective, slowly fading channels. 2. Numerical analysis

In this paper we have analysed the error performance of both the sequency majority multiplexing (SMM) and the concatenated sequency majority multiplexing (CSMM) schemes in the traditional AWGN channel and in a selection of frequency-nonselective, slowly fading channels. Our analysis extended the current knowledge on how these sequency multiplexing schemes perform in fading channels-an environment pertinent to the operations of wireless communication systems. We also looked at the improved performance of the CSMM scheme and revealed how such a simple improvement can further extend the performance gains of the SMM scheme by 2 to 3 folds. Our analysis highlighted a few attractive features of both the SMM and CSMM schemes which make them possible contenders for regulating multiple access in a multi-carrier spread spectrum scenario.
Low-Delay Rate Control for Consistent Quality Using Distortion-Based Lagrange Multiplier

Video quality fluctuation plays a significant role in human visual perception, and hence, many rate control approaches have been widely developed to maintain consistent quality for video communication. This paper presents a novel rate control framework based on the Lagrange multiplier in high-efficiency video coding. With the assumption of constant quality control, a new relationship between the distortion and the Lagrange multiplier is established. Based on the proposed distortion model and buffer status, we obtain a computationally feasible solution to the problem of minimizing the distortion variation across video frames at the coding tree unit level. Extensive simulation results show that our method outperforms the rate control used in HEVC Test Model (HM) by providing a more accurate rate regulation, lower video quality fluctuation, and stabler buffer fullness. The average peak signal-to-noise ratio (PSNR) and PSNR deviation improvements are about 0.37 dB and 57.14% in the low-delay (P and B) video communication, where the complexity overhead is ~4.44%.
Pre-And Post-Shift Filtering for Removing Blocking Effects in Downsizing Transcoding

When the high definition video is delivered via a limited bandwidth network, one solution is to reduce the spatial resolution of the bitstream. Downsizing transcoding is adopted as an efficient tool to reduce the spatial resolution of the pre-coded bitstream. Since the downsizing process and the coding process apply a low pass filter to the picture, the reconstructed video may present serious blocking artifact. In this paper, pre-and post-shift filters are considered in the transcoding process. The block boundary is shifted between the downsizing process and the coding process. This reduces the blocking effect in downsizing transcoding. Experimental results show that the proposed scheme can efficiently reduce the blocking effect without introducing high computation.
Using inter-frame dependence history to select intra-refresh blocks

To prevent error propagation in predictive video coding, intra-refresh methods are often employed. Here, a new algorithm for macroblock intra-refresh that is based on pixel level inter-frame dependence history is proposed. This new method keeps track of the coding mode of every pixel and refreshes those macroblocks that contain pixels with a long history record. Simulation results show that the new algorithm performs better than the existing methods while maintaining the same bit rate.
Face segmentation using skin-color map in videophone applications

This paper addresses our proposed method to automatically segment out a person's face from a given image that consists of a head-and-shoulders view of the person and a complex background scene. The method involves a fast, reliable, and effective algorithm that exploits the spatial distribution characteristics of human skin color. A universal skin-color map is derived and used on the chrominance component of the input image to detect pixels with skin-color appearance. Then, based on the spatial distribution of the detected skin-color pixels and their corresponding luminance values, the algorithm employs a set of novel regularization processes to reinforce regions of skin-color pixels that are more likely to belong to the facial regions and eliminate those that are not. The performance of the face-segmentation algorithm is illustrated by some simulation results carried out on various head-and-shoulders test images. The use of face segmentation for video coding in applications such as videotelephony is then presented. We explain how the face-segmentation results can be used to improve the perceptual quality of a videophone sequence encoded by the H.261-compliant coder.
Feature Adaptive Co-Segmentation by Complexity Awareness

In this paper, we propose a novel feature adaptive co-segmentation method that can learn adaptive features of different image groups for accurate common objects segmentation. We also propose image complexity awareness for adaptive feature learning. In the proposed method, the original images are first ranked according to the image complexities that are measured by superpixel changing cue and object detection cue. Then, the unsupervised segments of the simple images are used to learn the adaptive features, which are achieved using an expectation-minimization algorithm combining l 1-regularized least squares optimization with the consideration of the confidence of the simple image segmentation accuracies and the fitness of the learned model. The error rate of the final co-segmentation is tested by the experiments on different image groups and verified to be lower than the existing state-of-the-art co-segmentation methods.
Enhanced shot boundary refinement for post-shot boundary detection

Automatic extraction of video information has become an essential step in the video indexing system. In order to extract information from the special effects incorporated in the sequences, multiple stages are integrated together for effective processing. Shot boundary refinement is the first stage of such a technique. Its purpose is to the exact the transition start and end. The information is used later to determine the types of special effect as well as extracting information within these effects. The algorithm detects the exact shot boundaries by first constructing a reference indicator and picks up the sharp gradient changes within such an indicator. This paper describes the background information and the theory behind such a technique.
Q-DNN: A quality-aware deep neural network for blind assessment of enhanced images

Image enhancement is widely popular due to its capability of producing "better" visual quality for specific applications. Although many enhancement algorithms have been developed in recent years, the studies towards blind assessment of enhanced images are still very lacking. In this paper, we propose a data-driven blind image quality assessment (BIQA) method based on the quality-aware deep neural network (Q-DNN). Unlike the conventional hand-crafted features designed for measuring the degradation level of specific distortion types, a supervised learning model is utilized in our Q-DNN, which is capable of adaptively updating the feature extractor and quality regressor for describing the visual artifacts caused by different image enhancement tasks. Experimental results on two challenging enhanced image databases show that the proposed method is significantly superior to the state-of-the-art BIQA metrics.
Gaze-Based Object Segmentation

This letter addresses the problem of object segmentation with a gaze map and a group of candidate regions. First, we analyze distribution characteristics of gazes when people look at an image. Then, we summarize different cases of the group of candidate regions. Based on our analysis and summary, we develop three measures to evaluate the likelihood of a candidate region belonging to the target object and a pooling method to create a likelihood map of this object. Finally, the measures and pooling method are integrated with a proposed iterative strategy for generating the segmentation result. Experimental results demonstrate that our method can handle different types of gaze maps and different groups of candidate regions, and the overall performance of our method is better than that of the state-of-the-art method.
Global Propagation of Affine Invariant Features for Robust Matching

Local invariant features have been successfully used in image matching to cope with viewpoint change, partial occlusion, and clutters. However, when these factors become too strong, there will be a lot of mismatches due to the limited repeatability and discriminative power of features. In this paper, we present an efficient approach to remove the false matches and propagate the correct ones for the affine invariant features which represent the state-of-the-art local invariance. First, a pair-wise affine consistency measure is proposed to evaluate the consensus of the matches of affine invariant regions. The measure takes into account both the keypoint location and the region shape, size, and orientation. Based on this measure, a geometric filter is then presented which can efficiently remove the outliers from the initial matches, and is robust to severe clutters and non-rigid deformation. To increase the correct matches, we propose a global match refinement and propagation method that simultaneously finds a optimal group of local affine transforms to relate the features in two images. The global method is capable of producing a quasi-dense set of matches even for the weakly textured surfaces that suffer strong rigid transformation or non-rigid deformation. The strong capability of the proposed method in dealing with significant viewpoint change, non-rigid deformation, and low-texture objects is demonstrated in experiments of image matching, object recognition, and image based rendering.
Material segmentation in hyperspectral images with minimal region perimeters

We propose a supervised approach to the classification and segmentation of material regions in hyperspectral imagery. Our algorithm is a two-stage process, combining a pixelwise classification step with a segmentation step aiming to minimise the total perimeters of the resulting regions. Our algorithm is distinctive in its ability to ensure label consistency within local homogeneous areas and to generate material segments with smooth boundaries. Furthermore, we establish a new hyperspectral benchmark dataset to demonstrate the advantages of the proposed approach over several state-of-the-art methods.
Real-Time De-Interlacing for H.264-Coded HD Videos

It is very challenging to de-interlace high-definition videos in real time, as both good visual quality and low complexity should be fulfilled, which, however, are conflicting. To resolve the conflict, a syntax-based de-interlacer is proposed specially for H.264-coded videos, by which the values of the syntax elements (SE) in H.264 bitstreams, such as macroblock-type, intra-prediction modes, and motion vectors (MV), are used to detect edge and motion, estimate the MV, and select proper local de-interlacing algorithms accordingly. A verification mechanism is also proposed to make sure the referred SE values are reliable. The experimental results show that the proposed de-interlacer provides better visual quality than common ones and can de-interlace 1080i sequences in real time on PCs.
Object Co-Segmentation Based on Shortest Path Algorithm and Saliency Model

Segmenting common objects that have variations in color, texture and shape is a challenging problem. In this paper, we propose a new model that efficiently segments common objects from multiple images. We first segment each original image into a number of local regions. Then, we construct a digraph based on local region similarities and saliency maps. Finally, we formulate the co-segmentation problem as the shortest path problem, and we use the dynamic programming method to solve the problem. The experimental results demonstrate that the proposed model can efficiently segment the common objects from a group of images with generally lower error rate than many existing and conventional co-segmentation methods.
Reduced-Reference Video Quality Assessment of Compressed Video Sequences

In this paper, a novel reduced-reference (RR) video quality assessment (VQA) is proposed by exploiting the spatial information loss and the temporal statistical characteristics of the interframe histogram. From the spatial perspective, an energy variation descriptor (EVD) is proposed to measure the energy change of each individual encoded frame, which results from the quantization process. Besides depicting the energy change, EVD can further simulate the texture masking property of the human visual system (HVS). From the temporal perspective, the generalized Gaussian density (GGD) function is employed to capture the natural statistics of the interframe histogram distribution. The city-block distance (CBD) is used to calculate the histogram distance between the original video sequence and the encoded one. For simplicity, the difference image between adjacent frames is employed to characterize the temporal interframe relationship. By combining the spatial EVD together with the temporal CBD, an efficient RR VQA is developed. Evaluation on the subjective quality video database demonstrates that the proposed method outperforms the representative RR video quality metric and the full-reference VQAs, such as peak signal-to-noise ratio and structure similarity index in matching subjective ratings. This means that the proposed metric is more consistent with the HVS perception. Furthermore, as only a small number of RR features are extracted for representing the original video sequence (each frame requires only one parameter for describing EVD and three parameters for recording GGD), the RR features can be embedded into the video sequences or transmitted through the ancillary data channel, which can be used in the video quality monitoring system.
No-Reference Retargeted Image Quality Assessment Based on Pairwise Rank Learning

In this paper, we propose a novel no-reference image quality assessment method for the retargeted image based on the pairwise rank learning approach. Each retargeted image needs to be first represented as a feature vector, which not only captures the image characteristics but also is sensitive to distortions during the retargeting process. As such, we investigate and examine different image representations for their abilities depicting the perceptual quality of retargeted image. Based on the image representations, we resort to the pairwise rank learning approach to discriminate the perceptual quality between the retargeted image pairs. Experimental results demonstrate that the proposed method can effectively depict the perceptual quality of the retargeted image, which can even perform comparably with the full-reference quality assessment methods.
Practical Image Quality Metric Applied to Image Coding

Perceptual image coding requires an effective image quality metric, yet most of the existing metrics are complex and can hardly guide the compression effectively. This paper proposes a practical full-reference metric with consideration of the texture masking effect and contrast sensitivity function. The metric is capable of evaluating typical image impairments in real-world applications and can achieve the comparable performance as the state-of-the-art metrics on the publicly available subjectively-rated image databases. Due to its simplicity, the metric is embedded into JPEG image coding to ensure a better perceptual rate-distortion performance.
Additive Log-Logistic Model for Networked Video Quality Assessment

Modeling subjective opinions on visual quality is a challenging problem, which closely relates to many factors of the human perception. In this paper, the additive log-logistic model (ALM) is proposed to formulate such a multidimensional nonlinear problem. The log-logistic model has flexible monotonic or nonmonotonic partial derivatives and thus is suitable to model various uni-type impairments. The proposed ALM metric adds the distortions due to each type of impairment in a log-logistic transformed space of subjective opinions. The features can be evaluated and selected by classic statistical inference, and the model parameters can be easily estimated. Cross validations on five Telecommunication Standardization Sector of International Telecommunication Union (ITU-T) subjectively-rated databases confirm that: 1) based on the same features, the ALM outper-forms the support vector regression and the logistic model in quality prediction and, 2) the resultant no-reference quality met-ric based on impairment-relevant video parameters achieves high correlation with a total of 27 216 subjective opinions on 1134 video clips, even compared with existing full-reference quality metrics based on pixel differences. The ALM metric wins the model competition of the ITU-T Study Group 12 (where the validation databases are independent with the training databases) and thus is being put forth into ITU-T Recommendation P.1202.2 for the consent of ITU-T.