nils-herrmann/clean_scirepeval huggingface dataset:
Visual saliency detection via image complexity feature

In this paper we propose a novel bottom-up visual saliency detection model by analysis of image complexity. Compared with existing works, we emphasize the important impact of image complexity on saliency detection. Inspired by the free energy theory, a hybrid parametric and non-parametric model is used to estimate the complexity of a visual signal. Taking the image complexity as a new feature, this paper constructs a heuristic framework to systematically combine two different types of saliency detection models, separately using local and global features, in order to predict human fixation points more accurately. In contrast to classical and modern models, our algorithm has achieved noticeably superior results. And furthermore, it is worthy to stress that the proposed saliency detection method can also help to facilitate the performance of image quality metrics on popular image databases.
Reduced-reference quality metric for screen content image

With the prevalence of digital products like cellphone, tablet and personal computer, the screen content image (SCI) consisting of text, graphic, and natural scene picture becomes a significant media in various communication scenarios. Consequently, we proposed a reduced-reference quality metric dedicated for SCI. The main contribution includes 2 aspects: 1) we innovatively proposed a layer-based segmentation method to divide SCI into text layer and pictorial layer; 2) we designed respective quality metrics dedicated for text and pictorial layers with a novel pooling strategy considering human visual saliency for SCI. Furthermore, exhaustive experimental results indicate that the proposed metric is highly comparative compared with state-of-the-art full-reference quality metrics.
Blind image quality assessment for noise

Image denoising has been fanatically researched for a very long time in that it is a commonplace yet important subject. The task to testify the performance of different image de-noising methods always resorts to PSNR in the past, until the emergence of SSIM, a landmark image quality assessment (IQA) metric. Since then, a vast majority of IQA methods were introduced in terms of various kinds of models. But unfortunately, those IQA metrics are along with more or less deficiencies such as the requirement of original images, making them far less than the ideal approaches. To address this problem, in this paper we propose an effective and blind image quality assessment for noise (dubbed BIQAN) to approximate the human visual perception to noise. The BIQAN is realized with three important portions, namely the free energy based brain principle, image gradient extraction, and texture masking. We conduct and compare the proposed BIQAN and a large amount of existing IQA metrics on three largest and most popular image quality databases (LIVE, TID2013, CSIQ). Results of experiments prove that the BIQAN has acquired very encouraging performance, outperforming those competitors stated above.
A Unified Framework for Removing Blocking Artifacts

Al-Fahoum and Reza  characterized the blocking artifacts in block based DCT (BDCT) compressed image into five types: grid noise, staircase noise, ringing artifacts, corner outliers and the corruption of edges. Most of the comprehensive deblocking algorithms lack a unified framework, and different artifacts are processed with independent ad hoc schemes. In this paper, we propose a comprehensive postprocessing method for removing all the blocking-related artifacts in the framework of overcomplete wavelet expansion (OWE). We use the wavelet transform modulus maxima extension (WTMME) and angle extracted from the wavelet coefficients of 3-level OWE to represent the image. Both the WTMME and the angle image are reconstructed accordingly using inter-/ intra-band correlation to suppress the influence of the distortions. Simulation and comparative study have demonstrated the effectiveness of the proposed algorithm in terms of both subjective and objective quality of the resultant images.
Face super-resolution using 8-connected Markov Random Fields with embedded prior

In patch based face super-resolution method, the patch size is usually very small, and neighbor patchespsila relationship via overlapped regions is only to keep smoothness of reconstructed high-resolution image, so the prior is not always strong enough to regularize super-resolution when observed low-resolution image lose facial structure information. We propose to use Gaussian Mixture Model(GMM) to learn facial prior embedded between un-overlapped regions of neighbor patches. This approach, which has never been used to regularize face super-resolution before, usually works as a potential function in 8-connected Markov Random Fields (MRFs) with belief propagation. In the proposed algorithm, we assign high probability to the neighbor candidate patches that express correct facial structure, and others not. Experiments demonstrate that our method is superior in preserving smoothness and recovers facial structure and local details when low-resolution image lost the details of facial structure.
Lossless Predictive Coding for Images With Bayesian Treatment

Adaptive predictor has long been used for lossless predictive coding of images. Most of existing lossless predictive coding techniques mainly focus on suitability of prediction model for training set with the underlying assumption of local consistency, which may not hold well on object boundaries and cause large predictive error. In this paper, we propose a novel approach based on the assumption that local consistency and patch redundancy exist simultaneously in natural images. We derive a family of linear models and design a new algorithm to automatically select one suitable model for prediction. From the Bayesian perspective, the model with maximum posterior probability is considered as the best. Two types of model evidence are included in our algorithm. One is traditional training evidence, which represents the models' suitability for current pixel under the assumption of local consistency. The other is target evidence, which is proposed to express the preference for different models from the perspective of patch redundancy. It is shown that the fusion of training evidence and target evidence jointly exploits the benefits of local consistency and patch redundancy. As a result, our proposed predictor is more suitable for natural images with textures and object boundaries. Comprehensive experiments demonstrate that the proposed predictor achieves higher efficiency compared with the state-of-the-art lossless predictors.
Application of scalable visual sensitivity profile in image and video coding

Computational visual attention models have facilitated various aspects of the evolution in visual communication systems. In this paper, we apply the proposed computational model for scalable visual sensitivity profile (SVSP) to image/video processing. SVSP generates a hierarchy of saliency maps that simulate the bottom-up and top-down attention of the human visual system (HVS). It can detect the most significant visual information in a picture and spatially prioritize the information to augment the visual quality, such as in noise-shaping and region-of-interest (ROI) coding of JPEG2000. Furthermore, ROI scalable video coding (SVC) also benefits from the proposed SVSP due to its hierarchical nature, as multiple perceptually prioritized ROIs can be extracted by simply segmenting the SVSP in multiple thresholds in a scalable manner. It can be applied to automatically evaluate the quality for ROI scalability, and thus has the potential to reduce the workload of subjective tests for ROI SVC algorithms in the current MPEG activities. Extensive experiments are conducted to show the performances that justify the effectiveness of the proposed SVSP model.
Live demonstration: Screen piracy protection using saturation laser attack and TPVM

This live demonstration presents a new screen piracy protection technology by using “saturation laser attack” and temporal psychovisual modulation(TPVM). “Saturation laser attack” is using lasers to emit beams of colored light into the camera, so that the camera is saturated and instantly blind. TPVM is a new information display technology using the interplay of signal processing, optoelectronics and psychophysics and provides a solution for information security. Using the TPVM technique, we are able to show the disguise information on the screen while authorized viewers can truly see the secret information. An automatic target recognition algorithm is used to locate the position of the target mobile phone, then the device sends out lasers to the target phone to defeat unauthorized recording screens. By combining laser attack and TPVM, the whole system can prevent disclosure of confidential and personal information through unauthorized recording screens effectively.
Adaptive Sequential Prediction of Multidimensional Signals With Applications to Lossless Image Coding

We investigate the problem of designing adaptive sequential linear predictors for the class of piecewise autoregressive multidimensional signals, and adopt an approach of minimum description length (MDL) to determine the order of the predictor and the support on which the predictor operates. The design objective is to strike a balance between the bias and variance of the prediction errors in the MDL criterion. The predictor design problem is particularly interesting and challenging for multidimensional signals (e.g., images and videos) because of the increased degree of freedom in choosing the predictor support. Our main result is a new technique of sequentializing a multidimensional signal into a sequence of nested contexts of increasing order to facilitate the MDL search for the order and the support shape of the predictor, and the sequentialization is made adaptive on a sample by sample basis. The proposed MDL-based adaptive predictor is applied to lossless image coding, and its performance is empirically established to be the best among all the results that have been published till present.
Ultra high definition video saliency database

Visual attention plays an important role in image and video processing. Nowadays, high definition (HD) techniques have been widely used. And ultra high definition (UHD) is becoming more and more popular. However, existing researches in visual attention mainly focus on relatively low resolution videos or images. There is very limited studies in visual attention of UHD videos. In this paper, we built a Ultra High Definition (4k) Video Saliency Database. Using this database, we explored the characteristics of visual attention related to ultra high definition videos. A concept of aggregation maps (AGM) for videos is put forward to better analyse the characteristics of visual attention of videos. Through the experiment, we find that there exist fairly strong correlations between the video resolution and visual attention behaviors. We also find that people tend to focus on the center of videos of relatively low resolution. The database will be make publicly available at gvsp.sjtu.edu.cn soon.
A hierarchical saliency detection approach for bokeh images

Bokeh is a popular photograph technique that aesthetically highlights the image subject by properly blurring the background contents. While a large number of bokeh images can be found in our albums, the impacts of bokeh on visual saliency has not been studied yet. Our study shows that traditional saliency models do not perform well on bokeh images in that the foreground/background cannot be efficiently differentiated. Therefore, in this paper we propose a hierarchical saliency model for bokeh images through combining local sharpness measure and foreground saliency detection. More specifically, first we use local sharpness feature as a clue to locate the foreground objects in bokeh image so as to get the first level saliency map. Then we compute the second level saliency map from the blurry background using a robust saliency model. In the third step, we can generate the final saliency map by unequally weighted pooling. In order to evaluate the models' performance quantitatively, we build up a bokeh image saliency database. We test the proposed model against traditional models on the bokeh image database. The results indicate that the proposed model systematically outperforms all traditional saliency models.
Perceptual Quality Assessment of Omnidirectional Images

Omnidirectional images and videos can provide immersive experience of real-world scenes in Virtual Reality (VR) environment. We present a perceptual omnidirectional image quality assessment (IQA) study in this paper since it is extremely important to provide a good quality of experience under the VR environment. We first establish an omnidirectional IQA (OIQA) database, which includes 16 source images and 320 distorted images degraded by 4 commonly encountered distortion types, namely JPEG compression, JPEG2000 compression, Gaussian blur and Gaussian noise. Then a subjective quality evaluation study is conducted on the OIQA database in the VR environment. Considering that humans can only see a part of the scene at one movement in the VR environment, visual attention becomes extremely important. Thus we also track head and eye movement data during the quality rating experiments. The original and distorted omnidirectional images, subjective quality ratings, and the head and eye movement data together constitute the OIQA database. State-of-the-art full-reference (FR) IQA measures are tested on the OIQA database, and some new observations different from traditional IQA are made. The OIQA database will be released to facilitate further research.
‘Window of visibility’ inspired security lighting system

This paper introduces a new information display system that can prevent the on-screen contents from being taken by cameras or camcorders without sacrificing visual qualities. The system is designed based on the idea of temporal psychovisual modulation (TPVM) with guidance from the concept of ‘window of visibility (WoV)’. TPVM utilizes the redundancy (in terms of spatiotemporal resolutions and etc.) of modern display devices over those perception limits of the human visual system (HVS). The WoV theory asserts that the perceivable spatiotemporal frequency of the HVS is different from that of those cameras. Therefore, using a properly measured or estimated WoV we can determine how to select or design the display devices and contents to achieve the goal of security protection, i.e. preventing on-screen contents from being recorded by imaging devices, while not affecting the pictorial quality. We have built a prototype of WoV inspired TPVM security display system.
A new image quality metric based on MIx-Scale transform

In the study of image quality assessment (IQA), multi-scale methods are often used because they can supply more flexibility than single-scale ones by incorporating the variations of viewing conditions. However, some recent work demonstrated that single-scale methods based on scale transformation also performed well in terms of the correlation between the quality predictions and the subjective scores. Both of the single-scale based IQA metrics conform to a two-step model: image transform using proper scale coefficient and the SSIM metric, but the influence of different components in SSIM (luminance, contrast and structural similarity) on the chosen scale coefficient is not taken into account. In our re-search, it was found that the scale transform coefficients for different components should be different. This paper accordingly proposes a new MIx-Scale (MIS) based IQA algorithm. Experimental results on six publicly available databases (LIVE, TID2008, CSIQ, Toyama, IVC and LIVE Multiply Distortion) confirm that the proposed IQA metric can often enhance predictive accuracy for image quality assessment comparing to traditional scale-based methods.
An efficient color image quality metric with local-tuned-global model

This paper investigates the problem of full-reference (FR) image quality assessment (IQA). In general, the ideal IQA metric should be effective and efficient, yet most of existing FR IQA methods cannot reach these two targets simultaneously. Under the supposition that the human visual perception to image quality depends on salient local distortion and global quality degradation, we introduce a novel effective and efficient local-tuned-global (LTG) model induced IQA metric. Extensive experiments are conducted on five publicly available subject-rated color image quality databases, including LIVE, TID2008, CSIQ, IVC and TID2013, to evaluate and compare our algorithm with classical and state-of-the-art FR IQA approaches. The proposed LTG is shown to work fast and outperform those competing methods.
Visual saliency model based on minimum description length

In this paper, a novel patch-wise visual saliency model based on Minimum Length Description (MDL) principle is presented. Visual saliency is measured as the unpredicted information of image patch through an order-adaptive predictor under MDL principle. Specifically, each image patch is estimated with a linear combination of several neighboring patches. The number and location of candidate patches are automatically tuned to local contexts based on MDL. Then the entropy of prediction residuals of center patch, which represents the surprise to the visual system, is used to measure the saliency. Furthermore, a structural redundancy operator is also involved to improve the saliency detection performance. Experimental results demonstrate that the predictor under MDL principle along with the structural redundancy operator can improve the accuracy of human fixations prediction. We show that the proposed model outperforms the mainstream algorithms in predicting human fixations.
Structural similarity weighting for image quality assessment

Recently, there has been a trend of investigating weighting/pooling strategies in the research of image quality assessment (IQA). The saliency maps, information content maps and other weighting strategies were reportedly to be able to amend performance of IQA metrics to a sizable margin. In this work, we will show that local structural similarity is itself an effective yet simple weighting scheme leading to substantial performance improvement of IQA. More specifically, we propose a Structural similarity Weighted SSIM (SW-SSIM) metric by locally weighting the SSIM map with local structural similarities computed using SSIM itself. Experimental results on LIVE database confirm the performance of SW-SSIM as compared to some major weighting/pooling type of IQA methods, such as MS-SSIM, WSSIM and IW-SSIM. We would like to emphasize that our SW-SSIM is merely a straightforward realization of a more general framework of locally weighting IQA metric using itself as similarity measures.
Portable information security display system via Spatial PsychoVisual Modulation

With the rapid development of visual media, people prefer to pay more attention to privacy protection in public situations. Currently, most existing researches on information security such as cryptography and steganography mainly concern transmission and yet little has been done to keep the information displayed on screens from reaching eyes of the bystanders. At the same time, the reporter just stands in front of the screen during traditional meetings. Limited time, screen area and report forms, which inevitably leads to limited information. As a result, we design a portable screen for assisting the reporter to present any information in a new form. In some public occasions, the reporter can show private content with important information to authorized audience while the others can not see that. In this paper, we propose a new Spatial PsychoVisual Modulation (SPVM) based solution to the privacy problem. This system uses two synchronized projectors with linear polarization filters and polarization glasses, and a camera with linear polarization filter the metallic screen. It can guarantee the system shows private information synchronously. We have implemented the system and experimental results demonstrate the effectiveness and robustness of the proposed information security display system.
A dual-model approach to blind quality assessment of noisy images

Physiological and psychological evidences exist that the human visual system (HVS) has different behavioral patterns under low and high noise/artifact levels. We propose in this paper a dual-model approach to blind or no-reference (NR) image quality assessment (IQA) of noisy images through differentiating near-threshold and suprathreshold noise conditions. The underlying assumption for the proposed dual-model method is that for images with low level near-threshold noise, HVS tries to gauge the strength of the noise, so the image quality can be well approximated via measuring strength of the noise. And for images with their contents overwhelmed by high level suprathreshold noise, the HVS tries to recover meaningful structure from the noisy pixels using past experiences and prior knowledge encoded into an internal generative model of the brain. So image quality is closely related to the agreement between the noisy observation and the internal generative model explainable part of the image. More specifically, under the near-threshold noise condition, a noise level estimation algorithm based on natural image statistics is used, while under suprathreshold condition, an active inference model based on the free energy principle is adopted. The near-and suprathreshold models can be seamlessly integrated through a transformation between both estimates. The proposed dual-model algorithm has been tested on additive Gaussian noise contaminated images. Experimental results and comparative studies suggest that although being a no-reference approach, the proposed algorithm has prediction accuracy comparable to some of the best full-reference (FR) IQA methods.
No-reference image quality assessment for photographic images of consumer device

In this paper we study common, camera-specific kinds of distortions and propose a no-reference image quality assessment algorithm for photographic images produced by consumer devices. Those real consumer-type images, being different from simulated-distortion images, are with realistic artifacts and quality ranges. We find that the state-of-the-art no-reference image quality assessment approaches do not perform well on those photographic images, and propose an approach that achieves high prediction performance on a dataset of consumer-centric images. The proposed method, with no need for the original image, is able to reveal camera-specific problems and differentiate consumer cameras.
Image error-concealment via Block-based Bilateral Filtering

We propose to use Block-based Bilateral Filter (BBF), which extends the classical Bilateral Filter (BF) through operating in block-wise manner, to conceal missing image blocks in the application of compressed image transmission over wireless channels. We show that the problem of error-concealment using BBF can be considered as a superset of image denoising using BF. The BBF has the ability to capture the block-level similarity that well matches the need of error-concealment for block based image compression. Simulation results suggest significant visual and PSNR improvements (up to 11 dB) over various classic and state-of-the-art error-concealment algorithms.