nils-herrmann/clean_scirepeval huggingface dataset:
Error concealment using block-based scale-invariant features

In this paper, we propose a scale-invariant block-based bilateral filter (SI-BBF) for spatial error concealment. The SI-BBF extends block-based bilateral filter (BBF)and takes advantage of both non-corrupted data and scale-invariant space data in restoring missing pixels, while operates in a block wise manner. In order to reduce the computational complexity, we introduce a novel weight adaptive selection scheme in our algorithm. Extensive experiments show favorable results of the proposed algorithm against several competitors.
FISBLIM: A FIve-Step BLInd Metric for quality assessment of multiply distorted images

The last decade has seen a surge of interest in the research of image quality assessment (IQA). Many successful quality metrics, such as structural similarity index (SSIM) are reportedly to achieve very high accuracy for various kinds of image distortions. However, in practice, multiple image distortions tend to occur together and this leads difficulty to previous works of IQA including SSIM and variations. This problem is even more difficult for no-reference or blind quality assessment. To answer this challenge, this paper proposes a new FIve-Step BLInd Metric (FISBLIM) for quality assessment of multiply distorted images. The algorithm is built upon several common image processing blocks to simulate the image perceiving process of the human visual system (HVS). The FISBLIM method is not training based and the performance is robust and not database-dependent. Experimental results on the newly released LIVE multiply distorted image quality database demonstrate the effectiveness of FISBLIM as compared with mainstream full-reference and no-reference image quality metrics.
Temporal Psychovisual Modulation: A New Paradigm of Information Display [Exploratory DSP]

Information display technologies have played an indispensible role in the advancement of human civilization, from rock carving of prehistoric tribesmen to pervasive use of electronic displays in modern societies. The ever-presence, great variety and continuously increasing sophistication of optoelectronic displays profoundly shape the ways we learn, work, communicate, and how entertain and even how we socially behave. This article presents a new paradigm of information display, called temporal psychoviusal modulation (TPVM), which is conceived as an ingenious interplay of signal processing, optoelectronics, and psychophysics. The TPVM display paradigm, differs fundamentally in design principle, user experience, and cost effectiveness, from the head-mounted display (HMD) technology and all others while offering richer functionalities.
LGPS: Phase Based Image Quality Assessment Metric

Phase map of the images captures the most fundamental cognitive features and thus is widely used in various digital image processing tasks. In this paper, we propose the Log Gabor Phase Similarity (LGPS), a novel full reference image quality assessment metrics based on measuring of similarities between phases in log Gabor transform domain. Phase can capture any changes in image details regardless of the fluctuation in contrast, and the similarity between phase maps provides a measure of the perceptual quality of images. An image is firstly decomposed by a filter bank consisting of a pair of log Gabor filters. The phase maps are then computed from the responses of each filter pair. We have developed a window-based similarity metric to evaluate the resemblance between phase maps so as to measure the quality of the image. Experimental results and comparative studies suggest that LGPS can be used to predict the perceived quality of images with different distortions.
DLP based anti-piracy display system

Camcorder piracy has great impact on the movie industry. Although there are many methods to prevent recording in theatre, no recognized technology satisfies the need of defeating camcorder piracy as well as having no effect on the audience. This paper presents a new projector display technique to defeat camcorder piracy in the theatre using a new paradigm of information display technology, called temporal psychovisual modulation (TPVM). TPVM exploits the difference in image formation mechanisms of human eyes and imaging sensors. The images formed in human vision is continuous integration of the light field while discrete sampling is used in digital video acquisition which has "blackout" period in each sampling cycle. Based on this difference, we can decompose a movie into a set of display frames and broadcast them out at high speed so that the audience can not notice any disturbance, while the video frames captured by camcorder will contain highly objectionable artifacts. The proposed prototype system built on the platform of DLP® LightCrafter 4500™ serves as a proof-of-concept of anti-piracy system.
Uncrowded window inspired information security display

With the booming of visual media, people pay more and more attention to privacy protection in public environments. Most existing research on information security such as cryptography and steganography is mainly concerned about transmission and yet little has been done to prevent the information displayed on screens from reaching eyes of the bystanders. To deal with the problem, for the application of text-reading, we proposed an eye tracking based solution using the newly revealed concept of uncrowded window from vision research. The theory of uncrowded window suggests that human vision can only effectively recognize objects inside a small window. Object features outside the window may still be detectable but the feature detection results cannot be efficiently combined properly and therefore those objects will not be recognizable. We use eye-tracker to locate fixation points of the authorized reader in real time, and only the area inside the uncrowded window displays the private information we want to protect. A number of dummy windows with fake messages are displayed around the real uncrowded window as diversions. And without the precise knowledge about the fixations of the authorized reader, the chance for bystanders to capture the private message from those surrounding area and the dummy windows is very low. Meanwhile, since the authorized reader can only read within the uncrowded window, detrimental impact of those dummy windows is almost negligible. The proposed prototype system was written in C++ with SDKs of Direct3D, Tobii Gaze SDK, CEGUI, MuPDF, OpenCV and etc. Extended demonstration of the system will be provided to show that the proposed method is an effective solution to the problem of information security and display.
Blind quality assessment of compressed images via pseudo structural similarity

Block-based compression causes severe pseudo structures. We find that the pseudo structures of images compressed by different levels show some degree of similarity. So we propose to evaluate the quality of compressed images via the similarity between pseudo structures of two images. To obtain a “reference” image, we introduce the most distorted image (MDI), which is derived from the distorted image and suffers from the highest degree of compression. The proposed pseudo structural similarity (PSS) model calculates the similarity between pseudo structures of the distorted image and MDI. Pseudo structures of the distorted image become similar to the MDI's under the condition of severe compression. Via comparative tests, the proposed PSS model, on one hand, is shown to be comparable to state-of-the-art competitors, and on the other hand, it is not only good at assessing natural scene images but also performs the best in the hotly-researched screen content image (SCI) database. It deserves to mention that PSS is able to boost the performance of mainstream general-purpose no-reference (NR) quality measures.
Blindly evaluating stereoscopic image quality with free-energy principle

Three-dimensional (3D) imaging technology has been growingly prevalent in today's world. But objective quality assessment of 3D images is a challenging task. In this paper, we propose a blind metric to predict the perceptual quality of stereopairs within the concept of free energy. On the basis of a psychological measure, the free energy is a principle telling where supervises more and attracts human attention. We believe that the “surprise” can account for the binocular rivalry and thus be used to predict the quality of stereopairs. We first evaluate the quality of the monoscopic image, then introduce the computation process of binocular rivalry's results for deciding the relative importance of the left and right views, and finally infer the overall quality score. Our algorithm is tested on the symmetric LIVE3D-I and asymmetric LIVE3D-II databases. Experimental results confirm that the proposed blind 3D IQA technique, without distortion identification, is able to faithfully predict the visual quality of stereopairs.
Using Structural degradation and Parallax for reduced-reference quality assessment of 3D images

Owing to the thriving market of stereoscopic image based applications, efficient and effective 3D image quality assessment (IQA) techniques become colossally required these days. Consequently, we introduce a new reduced-reference (RR) stereoscopic image quality metric to meet this demand, through measuring Structural degradation and Saliency based Parallax compensation Model (SSPM). Experimental results on the LIVE 3D Image Quality Database, including both symmetrically and asymmetrically distorted stereoscopic images in different categories and quality levels, are provided to justify the effectiveness of the proposed SSPM model as compared to some existing progressive and popular stereoscopic IQA approaches. Meanwhile, it deserves broad attentions that only four number pairs, extracted from original image, are required as the key feature to be sent to the receiver terminal, thus making this procedure also efficient.
Dual-Transform Based Noise Estimation

Noise estimation is the important premise for image denoising. Recently, a noise estimation method based on the scale invariance assumption of high order statistics of natural images' filter response has been proposed and achieved performance superior to classic algorithms. However, in our research, we notice that the scale invariance assumption may not hold well for images with highly directional edges or with large smooth areas. To rectify the problem and further improve noise estimation accuracy, in this paper, we propose a dual-transform based noise estimation algorithm, in which a cascade of wavelet transform and undirectional DCT transform is used. It will be shown that the dual-transform coefficients conforms the scale-invariance assumption well and therefore leads to more accurate noise estimation results. Moreover, computation complexity is considerably reduced due to the down scaling nature of the dual-transform. Extensive experiments and comparative study show the reliability and superiority the proposed methods.
Subjective and objective quality assessment for color changed images

Color change is an important factor in image quality assessment (IQA). Changing color manually is a hard work and improper process may destroy the perceptual quality. This problem has been largely ignored in traditional IQA which mainly focused on evaluating blur, noise and compression loss, etc. One bottleneck in this field is the lack of databases. Existing databases have very few distortion types related with color changes. In this paper, we construct a new database specially designed for color changed IQA problem. We then test a majority of state-of-the-art IQA methods on this database and prove that existing algorithms are indeed not suitable for color changed distortion. Therefore, we build a novel matrix considering features in color theory and statistics. The new model can effectively evaluate the color changed image quality.
Hybrid No-Reference Quality Metric for Singly and Multiply Distorted Images

In a typical image communication system, the visual signal presented to the end users may undergo the steps of acquisition, compression and transmission which cause the artifacts of blurring, quantization and noise. However, the researches of image quality assessment (IQA) with multiple distortion types are very limited. In this paper, we first introduce a new multiply distorted image database (MDID2013), which is composed of 324 images that are simultaneously corrupted by blurring, JPEG compression and noise injection. We then propose a new six-step blind metric (SISBLIM) for quality assessment of both singly and multiply distorted images. Inspired by the early human visual model and recently revealed free energy based brain theory, our method works to systematically combine the single quality prediction of each emerging distortion type and joint effects of different distortion sources. Comparative studies of the proposed SISBLIM with popular full-reference IQA approaches and start-of-the-art no-reference IQA metrics are conducted on five singly distorted image databases (LIVE, TID2008, CSIQ, IVC, Toyama) and two newly released multiply distorted image databases (LIVEMD, MDID2013). Experimental results confirm the effectiveness of our blind technique. MATLAB codes of the proposed SISBLIM algorithm and MDID2013 database will be available online at http://gvsp.sjtu.edu.cn/.
An Augmented-Reality night vision enhancement application for see-through glasses

Nyctalopia, millions of people all over the world suffering from which, brings much trouble to the patients. The available night vision systems are poor in user experience but cost much. In this paper we design and implement an Augmented-Reality night vision enhancement application for see-through glasses. According to our model proposed for the night blindness, fast and efficient algorithms are used for both night vision enhancement and display calibration. The processed images are much brighter and aligned to the real world. The results in different daily life scenes are presented, indicating huge convenience the application will bring to the nyctalopia patients.
Blind Quality Assessment of Tone-Mapped Images Via Analysis of Information, Naturalness, and Structure

High dynamic range (HDR) imaging techniques have been working constantly, actively, and validly in the fault detection and disease diagnosis in the astronomical and medical fields, and currently they have also gained much more attention from digital image processing and computer vision communities. While HDR imaging devices are starting to have friendly prices, HDR display devices are still out of reach of typical consumers. Due to the limited availability of HDR display devices, in most cases tone mapping operators (TMOs) are used to convert HDR images to standard low dynamic range (LDR) images for visualization. But existing TMOs cannot work effectively for all kinds of HDR images, with their performance largely depending on brightness, contrast, and structure properties of a scene. To accurately measure and compare the performance of distinct TMOs, in this paper develop an effective and efficient no-reference objective quality metric which can automatically assess LDR images created by different TMOs without access to the original HDR images. Our model is shown to be statistically superior to recent full- and no-reference quality measures on the existing tone-mapped image database and a new relevant database built in this work.
Lightweight smart glass system with audio aid for visually impaired people

In this paper, we present a new smart glass system for people who are blind or have low vision. Individuals with visual disability are difficult to communicate effectively with their environment. The blind or visually impaired rely largely on their other senses such as hearing, touch, and smell in order to understand their surroundings. It is pretty hard for them to go out alone, not to mention finding toilets, subway stations, restaurants and so on. Giving blind people the great accessibility to their environment is the objective of the smart glass system. This smart glass system can see the world for them and give voice instructions and hints through wireless bone conduction headphones. This smart glass system will help visually impaired people gain increased independence and freedom in city life. And we have implemented an application of public signs recognition on the smart glass system. This application can detect and recognize the public signs in cities, and give corresponding voice hints to the blindness. The proposed prototype system is based on Intel Edison, and the application in the system is written in C++ with libraries of OpenCV. Extended demonstration of the system will be presented to show how the system helps the blindness “see the world”.
Automatic Contrast Enhancement Technology With Saliency Preservation

In this paper, we investigate the problem of image contrast enhancement. Most existing relevant technologies often suffer from the drawback of excessive enhancement, thereby introducing noise/artifacts and changing visual attention regions. One frequently used solution is manual parameter tuning, which is, however, impractical for most applications since it is labor intensive and time consuming. In this research, we find that saliency preservation can help produce appropriately enhanced images, i.e., improved contrast without annoying artifacts. We therefore design an automatic contrast enhancement technology with a complete histogram modification framework and an automatic parameter selector. This framework combines the original image, its histogram equalized product, and its visually pleasing version created by a sigmoid transfer function that was developed in our recent work. Then, a visual quality judging criterion is developed based on the concept of saliency preservation, which assists the automatic parameters selection, and finally properly enhanced image can be generated accordingly. We test the proposed scheme on Kodak and Video Quality Experts Group databases, and compare with the classical histogram equalization technique and its variations as well as state-of-the-art contrast enhancement approaches. The experimental results demonstrate that our technique has superior saliency preservation ability and outstanding enhancement effect.
Influence of compression artifacts on visual attention

Visual attention is an important function of the human visual system (HVS). In the long term research of visual attention, various computational models have been proposed with encouraging results. However, most of those work were conducted on images with ideal visual quality. In practice, outputs of most visual communication systems contain different levels of artifacts, e.g. noise, blurring, blockiness and etc. Therefore, it is interesting to investigate the impacts of artifacts on visual attention. In this paper, we question into the problem of how the widely encountered JPEG compression artifacts affect visual attention. We designed eye-tracking experiments on images with different levels of compression and viewing time and quantitatively compared the recorded eye movement data. We found that compression level does have impacts on visual attention, and yet this influence can be negligible for low levels of compression. For high levels of compression, the visual artifacts alter visual attention in a systematic way. Dependence of the influence on viewing duration was also analyzed and it was observed that too short or too long viewing time reduces the impact of compression artifacts on visual attention.
Quality assessment for dual-view display system

Spatial psychovisual modulation (SPVM) is a new information display technology, which aims to generate multiple visual percepts for different viewers on a single display simultaneously. After the proposal of SPVM, lots of efforts have been made and several applications (i.e., dual-view display system) have been implemented based on this technology. The dual-view display (DVD) system is considered as an effective digital image hiding system based on SPVM theory, but little work has been dedicated to the perceptual quality assessment of DVD system. Up to now, there is no clear and standard method to evaluate the performance of the dual-view display system. It is important for the viewers to see a clear and non-aliasing image when they are front of the screen. Therefore, in this paper, we will build a DVD database and carry out a subjective experiment to evaluate the performance of the DVD system, and then we investigate and analyze the performance of prevailing no-reference (NR) image quality metrics on the particular DVD system. We have a sufficient belief that this paper can supply the guideline for the performance on the DVD system and serve as a good testing bed for future research of SPVM technology.
Full-reference image quality assessment via region-based analysis

Image quality assessment (IQA) plays an important role in many image processing systems. Based on the fact that different image components have different visual impact, we propose a new region-based method using a three-region image model and Back Propagation (BP) neural network. Experimental results show that our algorithm is obviously better than the peer image quality assessment method, using the Laboratory for Image and Video Engineering (LIVE) database as a test-bed.
IPAD: Intensity potential for adaptive de-quantization

Display devices at bit-depth of 10 or higher have been mature but the mainstream media source is still at bit-depth as low as 8. To accommodate the gap, the most economic solution is to render source at low bit-depth for high bit-depth display, which is essentially the procedure of de-quantization. Traditional methods, like zero-padding or bit replication, introduce annoying false contour artifacts. To better estimate the least-significant bits, later works use filtering or interpolation approaches, which exploit only limited neighbor information, can not thoroughly remove the false contours. In this paper, we propose a novel intensity potential field to model the complicated relationships among pixels. Then, an adaptive de-quantization algorithm is proposed to convert low bit-depth images to high bit-depth ones. To the best of our knowledge, this is the first attempt to apply potential field for natural images. The proposed potential field preserves local consistency and models the complicated contexts very well. Extensive experiments on natural image datasets validate the efficiency of the proposed intensity potential field. Significant improvements have been achieved over the state-of-the-art methods on both PSNR and SSIM.
Efficient Image Deblocking Based on Postfiltering in Shifted Windows

We propose a simple yet effective deblocking method for JPEG compressed image through postfiltering in shifted windows (PSW) of image blocks. The MSE is compared between the original image block and the image blocks in shifted windows, so as to decide whether these altered blocks are used in the smoothing procedure. Our research indicates that there exists strong correlation between the optimal mean squared error threshold and the image quality factor Q, which is selected in the encoding end and can be computed from the quantization table embedded in the JPEG file. Also we use the standard deviation of each original block to adjust the threshold locally so as to avoid the over-smoothing of image details. With various image and bit-rate conditions, the processed image exhibits both great visual effect improvement and significant peak signal-to-noise ratio gain with fairly low computational complexity. Extensive experiments and comparison with other deblocking methods are conducted to justify the effectiveness of the proposed PSW method in both objective and subjective measures.