nils-herrmann/clean_scirepeval huggingface dataset:
No-Reference Quality Assessment of Screen Content Pictures

Recent years have witnessed a growing number of image and video centric applications on mobile, vehicular, and cloud platforms, involving a wide variety of digital screen content images. Unlike natural scene images captured with modern high fidelity cameras, screen content images are typically composed of fewer colors, simpler shapes, and a larger frequency of thin lines. In this paper, we develop a novel blind/no-reference (NR) model for accessing the perceptual quality of screen content pictures with big data learning. The new model extracts four types of features descriptive of the picture complexity, of screen content statistics, of global brightness quality, and of the sharpness of details. Comparative experiments verify the efficacy of the new model as compared with existing relevant blind picture quality assessment algorithms applied on screen content image databases. A regression module is trained on a considerable number of training samples labeled with objective visual quality predictions delivered by a high-performance full-reference method designed for screen content image quality assessment (IQA). This results in an opinion-unaware NR blind screen content IQA algorithm. Our proposed model delivers computational efficiency and promising performance. The source code of the new model will be available at: https://sites.google.com/site/guke198701/publications.
Retina model inspired image quality assessment

We proposed in this paper a retina model based approach for image quality assessment. The retinal model is consisted of an optical modulation transfer module and an adaptive low-pass filtering module. We treat the model as a black box and design the adaptive filter using an information theoretical approach. Since the information rate of visual signals is far beyond the processing power of the human visual system, there must be an effective data reduction stage in human visual brain. Therefore, the underlying assumption for the retina model is that the retina reduces the data amount of the visual scene while retaining as much useful information as possible. For full reference image quality assessment, the original and distorted images pass through the retinal filter before some kind of distance is calculated between the images. Retina filtering can serve as a general preprocessing stage for most existing image quality metrics. We show in this paper that retina model based MSE/PSNR, though being straightforward, has already state of the art performance on several image quality databases.
Exploiting global and local information for image quality assessment with contrast change

Image quality assessment (IQA) has undergone a booming period during the last decade. Contrast change, being an important type of visual alteration for images, has not been seriously treated yet in the current IQA research. To address this problem, in this paper we propose a new reduced-reference (RR) IQA metric for contrast-changed images by exploiting local similarity information and global statistics information (LAGSI). In order to verify the effectiveness of the proposed algorithm, we test LAGSI with a large number of competitors on the recently introduced CID2013 database as well as contrast change related subsets from TID2008 and CSIQ databases. Experimental results demonstrate the superiority of our approach over classical and state-of-the-art IQA metrics.
No-Reference Quality Assessment of Contrast-Distorted Images Based on Natural Scene Statistics

Contrast distortion is often a determining factor in human perception of image quality, but little investigation has been dedicated to quality assessment of contrast-distorted images without assuming the availability of a perfect-quality reference image. In this letter, we propose a simple but effective method for no-reference quality assessment of contrast distorted images based on the principle of natural scene statistics (NSS). A large scale image database is employed to build NSS models based on moment and entropy features. The quality of a contrast-distorted image is then evaluated based on its unnaturalness characterized by the degree of deviation from the NSS models. Support vector regression (SVR) is employed to predict human mean opinion score (MOS) from multiple NSS features as the input. Experiments based on three publicly available databases demonstrate the promising performance of the proposed method.
Visual Saliency Detection With Free Energy Theory

Visual saliency can be thought of as the product of human brain activity. Most existing models were built upon local features or global features or both. Lately, a so-called free energy principle unifies several brain theories within one framework, and tells where easily surprise human viewers in a visual stimulus through a psychological measure. We believe that this “surprise” should be highly related to visual saliency, and thereby introduce a novel computational Free Energy inspired Saliency detection technique (FES). Our method computes the local entropy of the gap between an input image signal and its predicted counterpart that is reconstructed from the input one with a semi-parametric model. Experimental results prove that our algorithm predicts human fixation points accurately and is superior to classical/state-of-the-art competitors.
Multiuser Collaborative Viewport via Temporal Psychovisual Modulation [Applications Corner]

Consider a multiuser visualization scenario. When making or evaluating emergency response plans for a large city, professionals from police, fire, civil defense, health, environment, transportation, and social services departments meet and discuss. As emergency management involves coordinated activities of various stakeholders, all participants desire to have visual representation of location-sensitive data on a common, integrated display. Separate displays for different types of data cause semantic fragmentation; as one's eyes switch between displays to associate related information, mental transformation in cognitive psychology has to take place, reducing an individual's performance on the task. Moreover, separate personal displays create a feeling of isolation from others and hinders face-to-face communication.
Dynamic backlight scaling considering ambient luminance for mobile energy saving

The mobile video playback involves many subsystems of the devices such as computing, rendering and displaying subsystems. Among all subsystems, the displaying subsystem accounts for at least 38% of all consumed power, and it can be up to 68% with the maximum backlight brightness. What is more, lots of people watch videos via mobile devices in various situations, where the ambient luminance condition is different. Therefore, how to save mobile energy and improve the Quality of Experience (QoE) in different situations become significant problems. In this paper, we try to maximally enhance the battery power performance under various ambient luminance conditions through backlight magnitude adjusting, while without negatively impacting users' QoE. In particular, we conduct a series of subject quality assessment experiments to uncover the quantitative relationship among QoE, ambient luminance, video content luminance and backlight level. We first study whether the continuous playback of backlight-scaled shots using the proposed scaling magnitude would cause flicker effect or not. Then motivated by the findings of these subject studies, we implement a Dynamic Backlight Scaling (DBS) strategy. The experiment results demonstrate that the DBS strategy can save more than 40% power at most and can also save 10% power even at a very high ambient luminance.
Quality assessment of contrast-altered images

In image / video systems, the contrast adjustment which manages to enhance the visual quality is nowadays an important research topic. Yet very limited efforts have been devoted to the exploration of image quality assessment (IQA) for contrast adjustment. To address the problem, this paper proposes a novel reduced-reference (RR) IQA metric with the integration of bottom-up and top-down strategies. The former one stems from the recently revealed free energy theory which tells that the human visual system always seeks to understand an input image by the uncertainty removal, while the latter one is towards using the symmetric K-L divergence to compare the histogram of the contrast-altered image with that of the reference image. The bottom-up and top-down strategies are lastly combined to derive the Reduced-reference Contrast-altered Image Quality Measure (RCIQM). A comparison with numerous existing IQA models is conducted on contrast related CID2013, CCID2014, CSIQ, TID2008 and TID2013 databases, and results validate the superiority of the proposed
Perceptual Reduced-Reference Visual Quality Assessment for Contrast Alteration

In image/video systems, contrast adjustment which manages to enhance visual quality is nowadays an important research topic. Yet very limited struggles have been made to the exploration of visual quality assessment for contrast adjustment. To tackle the issue, this paper proposes a novel reduced-reference (RR) quality metric with the integration of bottom-up and top-down strategies. The former one stems from the recently revealed free energy principle that tells that the human visual system seeks to comprehend an input image via uncertainty removal, while the latter one is toward using the symmetric Kullback–Leibler divergence to compare the histogram of the contrast-altered image with that of the pristine image. The bottom-up and top-down strategies are lastly incorporated to derive the RR contrast-altered image quality measure. A comparison using numerous existing IQA models is carried out on five contrast related databases/subsets in CID2013, CCID2014, CSIQ, TID2008, and TID2013, and experimental results validate the superiority of the proposed technique.
MDL context modeling of images with application to denoising

The lately popularized patch-based nonlocal (NL) image processing approach is cast into a framework of statistical context modeling, a thoroughly studied topic in data compression and information theory. The adaptation of image patch (context) to local waveform is crucial to the performance of NL-type of image processing but yet lacks a rigorous study. In this paper we propose a minimum description length (MDL) approach for choosing the size and spatial configuration of the context in which a degraded pixel is to be restored. The MDL criterion of context formation aims to strike an optimal balance between the variance and bias of the errors in fitting a 2D piecewise autoregressive (PAR) model to input image signal. To exemplify the use of the proposed context modeling technique in image processing, an MDL-guided context-based image denoiser is derived and its performance evaluated. Empirical results show that the new context-based denoiser is highly competitive against the current state of the art.
The Analysis of Image Contrast: From Quality Assessment to Automatic Enhancement

Proper contrast change can improve the perceptual quality of most images, but it has largely been overlooked in the current research of image quality assessment (IQA). To fill this void, we in this paper first report a new large dedicated contrast-changed image database (CCID2014), which includes 655 images and associated subjective ratings recorded from 22 inexperienced observers. We then present a novel reduced-reference image quality metric for contrast change (RIQMC) using phase congruency and statistics information of the image histogram. Validation of the proposed model is conducted on contrast related CCID2014, TID2008, CSIQ and TID2013 databases, and results justify the superiority and efficiency of RIQMC over a majority of classical and state-of-the-art IQA methods. Furthermore, we combine aforesaid subjective and objective assessments to derive the RIQMC based Optimal HIstogram Mapping (ROHIM) for automatic contrast enhancement, which is shown to outperform recently developed enhancement technologies.
Visual attention data for image quality assessment databases

Images usually contain areas that particularly attract people's attention and visual attention is an important feature of human visual system (HVS). Visual attention had been shown to be effective in improving performance of existing image quality assessment (IQA) metrics. However, with the quick advancement of IQA research, the booming of open IQA databases calls for associated comprehensive and accurate visual attention dataset. Despite of the large number of existing computational attention/saliency models, the most accurate measure of human attention is still human based. In this research, we first conduct extensive eye tracking experiments for all the pristine images from the seven widely used IQA databases (LIVE, TID2008, CSIQ, Toyama, LIVE Multiply Distortion, IVC and A57 databases). Then we propose a gaze-duration adaptive weighting approach to generate saliency maps from the eye tracking data. When applied on the IQA databases, experimental results suggest that accuracy of benchmark quality metrics, e.g. PSNR and SSIM can be systematically improved, outperforming existing saliency datasets. Both the eye tracking data and the saliency maps in this research will be made publicly available at gvsp.sjtu.edu.cn.
Image deringing using quadtree based block-shift filtering

In this paper, we propose an efficient spatial domain deringing algorithm using quadtree (QT) decomposition and block-shift filtering (BSF). The ringing artifacts are located through the QT decomposition of an image down to the 4times4 block size. The blocks with suspicious ringing are then replaced with the weighted average of itself and pixel-by-pixel shifted neighboring blocks. The block shifting range is up to 9times9 and only those shifted blocks resembling the center one are involved in the averaging stage. Experimental results show that the proposed deringing algorithm can effectively suppress the ringing artifact while preserving image edges and textures well, and the method substantially outperforms most of the existing deringing algorithms reported in the literature, both subjectively and objectively.
Noise Estimation of Natural Images via Statistical Analysis and Noise Injection

We develop a framework for estimating the noise level of a natural image using two important statistics: 1) high kurtosis and 2) scale invariance in transform domain. By exploring the said priors of natural image statistics in 2-D discrete cosine transform (DCT) domain, we reveal the limitations of these statistics for images with highly directional edges or large smooth areas. Then, we derive a novel two-step estimation scheme for noise variance: 1) in preliminary estimation, an integration of wavelet and nondirectional DCT transform is used to alleviate the influence of image's structures and 2) a noise-injection rectification is further devised to deal with the noise-free image contents. A simulation and comparative study demonstrates that this algorithm reliably infers noise variance and its robustness over wide ranges of visual content and noise levels, while outperforming some relevant methods. This paper can significantly improve the performance of existing denoising techniques that require the noise variance as a critical parameter.
Screen image quality assessment incorporating structural degradation measurement

Screen content is typically composed of computer generated text and graphics. The contents shown on the screen exhibit various unnatural properties, such as sharp edges and thin lines with few color variations. In this paper we design a novel structure-induced quality metric (SIQM) for assessing the screen image quality. The proposed SIQM works by weighting the benchmark structural similarity index (SSIM) with the structural degradation measurement that is computed using SSIM as well. Experimental results conducted on the newly released subjective quality database concerning screen images show that on one hand the proposed technique is superior to existing quality measures, and on the other hand our model is able to optimize screen video coding and thus introduce remarkable visual quality improvement.
Self-adaptive scale transform for IQA metric

Recently, an increasing number of image quality assessment (IQA) algorithms have been developed based on multi-scale methods, such as MS-SSIM, IFC, VIF and IW-PSNR/SSIM. Inspired by the achievement of multi-scale type of IQA algorithms, this paper proposes a self-adaptive scale transform based IQA approach. Using image size and viewing distance as input variables, we construct a self-adaptive scale transform function to estimate the suitable scale transform coefficient for the following image quality metrics. Two of the most well-known full-reference IQA methods (PSNR and SSIM), and three publicly-available subjectrated image databases (LIVE, IVC and Toyama-MICT) with clear image size and viewing distance values are used as testing beds in this paper. Experimental results and comparative studies on different combinations of IQA methods and image databases suggest the effectiveness and the robustness of the proposed approach.
Reduced-Reference Image Quality Assessment in Free-Energy Principle and Sparse Representation

The free-energy principle in recent studies of brain theory and neuroscience models the perception and understanding of the outside scene as an active inference process, in which the brain tries to account for the visual scene with an internal generative model. Specifically, with the internal generative model, the brain yields corresponding predictions for its encountered visual scenes. Then, the discrepancy between the visual input and its brain prediction should be closely related to the quality of perceptions. On the other hand, sparse representation has been evidenced to resemble the strategy of the primary visual cortex in the brain for representing natural images. With the strong neurobiological support for sparse representation, in this paper, we approximate the internal generative model with sparse representation and propose an image quality metric accordingly, which is named FSI (free-energy principle and sparse representation-based index for image quality assessment). In FSI, the reference and distorted images are, respectively, predicted by the sparse representation at first. Then, the difference between the entropies of the prediction discrepancies is defined to measure the image quality. Experimental results on four large-scale image databases confirm the effectiveness of the FSI and its superiority over representative image quality assessment methods. The FSI belongs to reduced-reference methods, and it only needs a single number from the reference image for quality estimation.
Spatial Error Concealment With an Adaptive Linear Predictor

In this paper, a novel spatial error concealment (EC) algorithm is proposed. Under the sequential recovery framework, pixels in missing blocks are successively reconstructed based on adaptive linear predictor. The predictor automatically tunes its order and support shape according to local contexts. The predictor order and support shape are determined using Bayesian information criterion, which is able to strike a balance between the bias and variance of the prediction errors. The flexibility of the order-adaptive predictor is able to recover more important features or structures. A novel scan order based on the uncertainty of each pixel is also proposed to alleviate error propagation problem. Compared with the state-of-the-art EC algorithms, experimental results show that the proposed method gives better reconstruction performance in terms of objective and subjective evaluations.
Adaptive Screen Content Image Enhancement Strategy using Layer-based Segmentation

The ubiquitous screen content images (SCIs) play a significant role in various scenarios currently. However, most SCIs captured by consumer devices are frequently corrupted with distortions, especially contrast distortion. Unlike the natural images, SCIs are composed of text, graphics and natural scene pictures so that traditional image enhancement methods are not suitable for these compound images. Therefore, we innovatively proposed an adaptive strategy for enhancing SCIs in this paper. Firstly, we devised a segmentation method to divide SCI into text and pictorial regions. Next, the famous guided image filter (GIF) with big and small kernel sizes served as unsharpness masking for processing different regions adaptively. For verifying performance, the proposed method was tested on recently prevalent SCI datasets including SIQAD, and Webpage Dataset. Experimental results indicate that the proposed approach outperforms state-of-the-art methods in most SCIs with flat background.
HDR2014 - A high dynamic range image quality database

High dynamic range (HDR) imaging has attracted a lot of attention and enthusiasm in the last decades. With the quick advances of sensor technologies, even consumer level digital cameras are capable of capturing HDR images. However, a vast majority of nowadays displays still only have 8-bit color depth, and this leads to the widely studied topic of displaying HDR images on low dynamic range (LDR) devices or tone mapping. On the other hand, the recent emergence of 10-bit display devices brings the possibility of direct visualization of HDR images. So a natural question to ask is whether existing popular image quality metrics designed for and validated on LDR (8-bit) images perform equally well for HDR (10-bit) images. In this paper we propose a new and dedicated High Dynamic Range image quality database (HDR2014). That database is composed of 192 images with four kinds of distortions applied on 6 reference images. More specifically, we use 8 distortion levels for the artifacts of JPEG/JPEG2000 compression, white noise injection and Gaussian blurring. Twenty-five inexperienced viewers were involved in the subjective viewing test. Images were displayed on a pair of carefully calibrated 8-bit LDR and 10-bit HDR monitors and the subjective scores on both of which were recorded. We then tested some ubiquitous and state-of-the-art IQA metrics on the HDR2014 database. Experimental results show that HDR monitor indeed improved perceptual quality of the visual stimuli, as compared to LDR ones. And several existing IQA metrics are still doing well on HDR images, yet performance of some metrics drop significantly.
A fast video stabilization algorithm based on block matching and edge completion

Purpose of video stabilization is to register the frames of a video sequence with relative motions between each other to yield a stable video of higher perceptual quality. In this paper we focus on the problem of fast and robust video stabilization for the same scene based on temporal block matching. We use VoD principle to find the local motion vectors between adjacent frames, and then use statistical analysis to generate the global vibrant motion vector. After motion compensation, we further design an edge completion algorithm incorporating mosaicking and inpainting of neighbour frames, so as to reduce the impact of error propagation. Experimental results and comparative studies will be provided to justify the effectiveness of the proposed algorithm.