nils-herrmann/clean_scirepeval huggingface dataset:
No-Reference Image Sharpness Assessment in Autoregressive Parameter Space

In this paper, we propose a new no-reference (NR)/ blind sharpness metric in the autoregressive (AR) parameter space. Our model is established via the analysis of AR model parameters, first calculating the energy- and contrast-differences in the locally estimated AR coefficients in a pointwise way, and then quantifying the image sharpness with percentile pooling to predict the overall score. In addition to the luminance domain, we further consider the inevitable effect of color information on visual perception to sharpness and thereby extend the above model to the widely used YIQ color space. Validation of our technique is conducted on the subsets with blurring artifacts from four large-scale image databases (LIVE, TID2008, CSIQ, and TID2013). Experimental results confirm the superiority and efficiency of our method over existing NR algorithms, the state-of-the-art blind sharpness/blurriness estimators, and classical full-reference quality evaluators. Furthermore, the proposed metric can be also extended to stereoscopic images based on binocular rivalry, and attains remarkably high performance on LIVE3D-I and LIVE3D-II databases.
Quality enhancement of the multi-view exhibition system

Spatial Psychovisual Modulation (SPVM) is a newly proposed display technology which makes use of the redundancy in display capacity. Many applications (i.e., information security display and simultaneous dual-subtitle exhibition) have been developed based on SPVM. In the implementation of the SPVM systems, the original images are down-sampled line-by-line due to the property of polarization screens. The sampling process causes aliasing and influence negatively on the image quality. In this paper, we add a Lowpass Filter (LPF) into the original system before the sampling to eliminate the aliasing. We compare the improved algorithm and the direct method with objective image quality assessment (IQA). For both natural and computer generated images are used in SPVM, we tested our algorithm on many different databases. Considering the view distance of the SPVM system, we used the SAST model as the metrics of image quality. As a result, the proposed algorithm improves the IQA benchmark, which corresponds to our observation. The test also works as an application and test bed of the SAST IQA model. All the equipments of the multi-view exhibition system is off-the-shelf so it's cheaper to realize. And the enhancement in quality make the system capable for screens with lower PPI.
Progressive Image Denoising Through Hybrid Graph Laplacian Regularization: A Unified Framework

Recovering images from corrupted observations is necessary for many real-world applications. In this paper, we propose a unified framework to perform progressive image recovery based on hybrid graph Laplacian regularized regression. We first construct a multiscale representation of the target image by Laplacian pyramid, then progressively recover the degraded image in the scale space from coarse to fine so that the sharp edges and texture can be eventually recovered. On one hand, within each scale, a graph Laplacian regularization model represented by implicit kernel is learned, which simultaneously minimizes the least square error on the measured samples and preserves the geometrical structure of the image data space. In this procedure, the intrinsic manifold structure is explicitly considered using both measured and unmeasured samples, and the nonlocal self-similarity property is utilized as a fruitful resource for abstracting a priori knowledge of the images. On the other hand, between two successive scales, the proposed model is extended to a projected high-dimensional feature space through explicit kernel mapping to describe the interscale correlation, in which the local structure regularity is learned and propagated from coarser to finer scales. In this way, the proposed algorithm gradually recovers more and more image details and edges, which could not been recovered in previous scale. We test our algorithm on one typical image recovery task: impulse noise removal. Experimental results on benchmark test images demonstrate that the proposed method achieves better performance than state-of-the-art algorithms.
Blind Image Quality Estimation via Distortion Aggravation

Traditional blind image quality assessment (IQA) measures generally predict quality from a sole distorted image directly. In this paper, we first introduce multiple pseudo reference images (MPRIs) by further degrading the distorted image in several ways and to certain degrees, and then compare the similarities between the distorted image and the MPRIs. Via such distortion aggravation, we can have some references to compare with, i.e., the MPRIs, and utilize the full-reference IQA framework to compute the quality. Specifically, we apply four types and five levels of distortion aggravation to deal with the commonly encountered distortions. Local binary pattern features are extracted to describe the similarities between the distorted image and the MPRIs. The similarity scores are then utilized to estimate the overall quality. More similar to a specific pseudo reference image (PRI) indicates closer quality to this PRI. Owning to the availability of the created multiple PRIs, we can reduce the influence of image content, and infer the image quality more accurately and consistently. Validation is conducted on four mainstream natural scene image and screen content image quality assessment databases, and the proposed method is comparable to or outperforms the state-of-the-art blind IQA measures. The MATLAB source code of the proposed measure will be publicly available.
Shifted Window Based Filtering for Alleviating Blocking Artifacts

We propose a computationally simple yet effective image/video deblocking algorithm based on block level postfiltering. Each image block is shifted in the range of half of the block dimension to find the candidate blocks for smoothing. The Mean Absolute Error (MAE) between the central image block and the block indicated by a shifted window is used as a criterion to determine if the corresponding shifted block is used in smoothing. The threshold of the MAE is defined with analyzing the quantization table used in the codec. The proposed algorithm successfully removes the blockiness while preserving the edge features. Another merit of the proposed algorithm is that it is applicable to almost all bit rates without the requirement of detection on blockiness position or estimation on blockiness severity, thus does not impose computational overhead. The experimental results demonstrate that in a wide range of bit rates, the processed image retains higher PSNR and better visual quality.
Effective preprocessing stage in the fourier transform domain for image quality assessment

Image quality assessment (IQA) is currently an important research topic. In fact, the varying viewing distance between audiences and the display seriously affect the IQA accuracy, which has been largely overlooked. To this end, in this paper we take into account the image size and viewing distance as well as the preferential activation of V1 cells by vertical and horizontal contours, and thereby propose an adaptive frequency selection (AFS) model to preprocess input visual signals before IQA metrics are used. Our algorithm works by first applying Fourier Transform (FT) to the reference and distorted images to approximate the low-pass behavior of the human visual system, then extracting proper amount of low frequency components and partial high frequency components corresponding to vertical and horizontal directions, and finally reconstructing spatial images with the inverse FT. We validate the performance of AFS based PSNR and SSIM on the related LIVE, Toyama and IVC databases with clearly specified viewing conditions. Experimental results and comparative studies show the effectiveness of the proposed model.
Color guided thermal image super resolution

IR imaging is playing an important role in industry these days. However, the IR cameras only support low resolution for technical reasons. In this paper, we propose to use the visible camera as a guidance to super resolution the IR images. We setup a prototype of the IR-Color multi-sensor imaging system and construct a dataset including videos collected from different scenes for further research. Then we present our color guided algorithm which is suitable for this kind of super resolution problem. We test our algorithm on our dataset. The experiment results indicate that our algorithm performs well, and avoids the over-texture problems that traditional solutions suffer from. In further work, we will apply our multi-sensor system to more applications in computer vision.
Distinguish True or False 4K Resolution Using Frequency Domain Analysis and Free-Energy Modelling

With the prevalence of Ultra-High Definition (UHD) display terminals, 4K resolution (38402160 pixels) contents are becoming a major selling point for online video media. However, due to the insufficiency of natural UHD contents, a large number of false 4K videos are circulating on the web. Those '4K' contents, usually being upscaled from lower resolutions, often frustrate enthusiastic consumers and are in fact a waste of stringent bandwidth resources. In this paper, we propose to distinguish natural 4K contents from false ones through frequency domain analysis. The basic assumption is that true 4K contents have much more high frequency responses than the upscaled versions. We use the free energy modelling to approximate the Human Visual System so as to minimize the impact of structural complexity of visual contents. We set up a dataset containing more than 1k original 4K frames together with upscaled versions using four widely used interpolation algorithms. Experimental results show that the proposed free-energy-based metric has an accuracy rate higher than 90%.
Learning to integrate local and global features for a blind image quality measure

In this paper, we present a new algorithm for blind/no-reference image quality assessment (BIQA/NR-IQA). Most existing measures are “opinion-aware”, demanding human opinion scored images to map image features to them. The task of obtaining human scores of images is, however, commonly thought to be uneconomical, and thus we focus on “opinion free” (OF) quality metrics in this research. By integrating local and global features, this paper develops a learning-based BIQA approach with three steps by combining local and global features together. In the first step of extracting local features, we use the quality aware clustering with the centroid of each quality level trained by K-means, while we in the second step compute the global features based on the natural scene statistics. Finally, the third step uses the SVR to train a regression module from the above-mentioned local and global features to derive the overall image quality score. Experimental results on LIVE, TID2008, CSIQ, and TID2013 databases validate the effectiveness of our proposed metric (a general framework) as compared to popular no-, reduced- and full-reference IQA approaches.
Demo: DLP based anti-piracy display system

Camcorder piracy has great impact on the movie industry. Although there are many methods to prevent recording in theatre, no recognized technology satisfies the need of defeating camcorder piracy as well as having no effect on the audience. To realize anti-piracy, we uses a new paradigm of information display technology, called temporal psychovisual modulation (TPVM). TPVM exploits the difference in image formation mechanisms of human eyes and imaging sensors. Based on this difference, we build a prototype system on the platform of DLP® LightCrafter 4500™ which features high speed pattern display. The display system serves as a proof-of-concept of anti-piracy system.
Analysis of Distortion Distribution for Pooling in Image Quality Prediction

Image quality assessment (IQA) has been an active research area during last decades. Many existing objective IQA models share a similar two-step structure with measuring local distortion before pooling. Compared with the rapid development for local distortion measurement, seldom effort has been made dedicated to effective pooling schemes. In this paper, we design a new pooling model via the analysis of distortion distribution affected by image content and distortion. That is, distributions of distortion position, distortion intensity, frequency changes, and histogram changes are comprehensively considered to infer an overall quality score. Experimental results conducted on four large-scale image quality databases (LIVE, TID2008, CSIQ, and CCID2014) concluded with three valuable findings. First, the proposed technique leads to consistent improvement in the IQA performance for studied local distortion measures. Second, relative to the traditional pooling, the performance gain of our algorithm is beyond 15% on average. Third, the best overall performance made by the proposed strategy outperforms state-of-the-art competitors.
CVIQD: Subjective quality evaluation of compressed virtual reality images

The 360-degree spherical images/videos, also called Virtual Reality (VR) images/videos, can provide immersive experience of the real-world scenes in some specific systems. This makes it widely employed in concerts/sports events live and VR movies. However, it is difficult to transport, compress or store VR images/videos due to their high resolution. So it is significant to research how the popular coding technologies influence the quality of VR images. To this aim, this paper carries out subjective quality evaluation of compressed VR images and examines the correlation performance of popular objective quality measures in accordance with the aforesaid subjective ratings. We first establish a Compressed VR Image Quality Database (CVIQD), which includes five source VR images and associated 165 compressed images under three prevailing coding technologies. The Single-Stimulus (SS) method is exploited to collect the subjective scores from 20 inexperienced viewers. Next, we implement 10 classical and recent objective quality metrics on the CVIQD database and compute the correlation between each above quality metric and subjective assessment in terms of five commonly used performance indices. Experimental results reveal that multi-scale based MS-SSIM and ADD-SSIM models have lead to high correlation with human visual perception.
2D phase-based matching in uncalibrated images

A novel 2D phase-based matching approach is proposed to resolve the general stereo image matching problem. It is different from the current uncalibrated matching techniques, most of which obtain 2D dense disparity map after the epipolar geometry has been recovered. In this paper, the disparity is directly estimated by simply establishing correspondences between quaternionic phase structures of two QWF (quaternion wavelet filtered) images. Real and short-length biorthogonal wavelet bases are exploited to build linear-phase quaternion wavelet filters (LPQWFs). Once phases are extracted from the QWF image pair, the disparity estimation is formed as a minimization process of a cost function, which is formulated as a similarity measure for comparing quaternion wavelet phases. With regard to the mismatches near phase singularities, phase stability constraints are imposed on cost aggregation stage. And multi-scale matching strategy is introduced to avoid phase wrapping problem and improve convergence speed. The experimental results are encouraging in various image pairs.
Closing the gap: Visual quality assessment considering viewing conditions

Most of existing visual quality assessment algorithms are tested on standard databases that are created in controlled viewing conditions (e.g. display device, viewing distance and lighting). This implies that all the recoded subjective scores are only valid for the specific settings used in the database. However, with the prevalence of mobile devices, the practical viewing environments can significantly vary from moment to moment. It is our daily experience that the same image can look drastically different on dissimilar devices under changed viewing distance and/or lighting conditions. In other words, a gap exists between the eyes and the visual contents behind the screen in current research of quality assessment. Therefore, in this work, we perform subjective quality evaluation with varied actual viewing conditions. To make the research reproducible, we build a prototype system to record what the eyes really see from the screen and construct the viewing environment-changed image database. The database will be made available to the public. Meanwhile we design a dedicated effective environment-assessing algorithm. We believe that this work will benefit the research of visual quality assessment towards more practical applications.
No-Reference Quality Metric of Contrast-Distorted Images Based on Information Maximization

The general purpose of seeing a picture is to attain information as much as possible. With it, we in this paper devise a new no-reference/blind metric for image quality assessment (IQA) of contrast distortion. For local details, we first roughly remove predicted regions in an image since unpredicted remains are of much information. We then compute entropy of particular unpredicted areas of maximum information via visual saliency. From global perspective, we compare the image histogram with the uniformly distributed histogram of maximum information via the symmetric Kullback–Leibler divergence. The proposed blind IQA method generates an overall quality estimation of a contrast-distorted image by properly combining local and global considerations. Thorough experiments on five databases/subsets demonstrate the superiority of our training-free blind technique over state-of-the-art full- and no-reference IQA methods. Furthermore, the proposed model is also applied to amend the performance of general-purpose blind quality metrics to a sizable margin.
Cross-Dimensional Perceptual Quality Assessment for Low Bit-Rate Videos

Most studies in the literature for video quality assessment have been focused on the evaluation of quantized video sequences at fixed and high spatial and temporal resolutions. Only limited work has been reported for assessing video quality under different spatial and temporal resolutions. In this paper, we consider a wider scope of video quality assessment in the sense of considering multiple dimensions. In particular, we address the problem of evaluating perceptual visual quality of low bit-rate videos under different settings and requirements. Extensive subjective view tests for assessing the perceptual quality of low bit-rate videos have been conducted, which cover 150 test scenarios and include five distinctive dimensions: encoder type, video content, bit rate, frame size, and frame rate. Based on the obtained subjective testing results, we perform thorough statistical analysis to study the influence of different dimensions on the perceptual quality and some interesting observations are pointed out. We believe such a study brings new knowledge into the topic of cross-dimensional video quality assessment and it has immediate applications in perceptual video adaptation for scalable video over mobile networks.
Hybrid image interpolation with soft-decision kernel regression

Parametric linear autoregressive (AR) model has been widely used in image processing but is known to induce unstable results. The recently emerged nonparametric kernel regression is an effective structural method for forestalling outliers but often brings over-smoothed output. This paper introduces a hybrid algorithm for image interpolation through combining the strength of parametric and nonparametric modeling techniques. More specifically, it is a soft-decision kernel regression (SKR) method in which the soft-decision AR model is embedded into the adaptive kernel regression framework. Compared with the state-of-the-art interpolation methods, simulation results show that the proposed SKR algorithm achieves comparative or better results in terms of objective and subjective quality.
Web image interpolation via weighted total least squares regression

Although ordinary least squares (OLS) regression achieves great success in clean image interpolation, its effectiveness is questionable in the scenario of web images which are usually compressed beforehand. The inherent flaw of OLS is that it is asymmetric, the perturbation is only confined on the right side of the linear system. It is not reasonable for web images. Considering the drawback of OLS, in this paper, we propose an efficient web image interpolation algorithm based on total least squares (TLS) regression. In the proposed method, small perturbations are allowed in both side of the system, which are optimized by TLS in a patch-based manner. Furthermore, we develop a weighted version of TLS to consider contribution diversity of different samples and patches in model estimation, which can efficiently remove the influence of outliers in regression. Experimental results on benchmark test images demonstrate the efficiency of our method.
Visual attention on human face

Human faces are always the focus of visual attention since faces can provide plenty of information. Although some visual attention models incorporating face cues work better in scenes containing faces, no visual attention model is particularly designed for faces. On faces, many high-level factors will influence visual attention distribution. In practice, there are many visual communication systems in which faces occupy the scenes, such as video calls. Specific visual attention model designed for face images will be of great value in these circumstances. In this paper, we conduct research on visual attention analysis and modelling on human faces. To facilitate this research, we collect 120 face images and perform eye-tracking experiments with these images. Eye-movement data shows that detailed visual attention allocation exists on faces. Using face detection and facial landmark localization, we find that some facial features are highly effective for visual attention prediction. The performance of many visual attention models can be improved by incorporating those facial features.
Modeling Blocking Visual Sensitivity Profile

Blocking artifact is the most prevailing degradation caused by block-based DCT coding techniques under low bit-rate conditions. To alleviate blockings perceptually, it is desirable to measure the visibility of blocking artifacts. In this paper, we propose an efficient method of estimating the visual sensitivity of blocking artifacts in block-based DCT coding. The differences on block boundaries are measured and transformed into block discontinuity map. We consider the effects of luminance adaptation and texture masking on the blockings and integrate them using nonlinear operator to form an overall masking map. This masking map is then incorporated with the discontinuity map to generate the blocking visual sensitivity map (BVSM). This map can be used to guide perceptual quality assessment, codec parameter optimization, post-processing, etc. We demonstrate the validity of the BVSM through its application in image quality assessment