nils-herrmann/clean_scirepeval huggingface dataset:
Spatio temporal characteristics of the human color perception for digital quality assessment

Understanding the contrast sensitivity function (CSF) of the Human Visual System (HVS) has been in the focus of human psychophysics for more than 3 decades. This effort, despite certain successes, is far from closure. In this paper we present a study on the spatio-temporal characteristics of the HVS. We propose a new method of integration of both spatial and temporal effects in the CSF curve that is to be used in color image quality evaluation. The obtained results are very encouraging and show that the new CSF allows some improvements of the classical quality evaluation scheme.
Using the HVS binocular properties for the construction of a comprehensive stereoscopic matching model

3D perception consists in receiving two dissimilar retinal images on both eyes. These retinal images undergo pretreatments before arriving at the visual cortex to be merged. In this paper we propose an approach which redraws this pretreatment process. A model for computing the binocular energy is proposed. This model is based on the simple cells and the complex cells properties, which are the source of binocular fusion in the visual cortex. To refine the obtained matching, a set of stereoscopic constraints are integrated into the model, to filter the information as it is made at the lateral geniculate body (LGB). This modeling also allows us to detect the regions affected by the binocular rivalry phenomenon. The obtained results can be exploited by the stereoscopic vision applications, such as the 3D reconstruction, the stereoscopic coding, stereoscopic quality assessment and other applications.
Heuristic inspired search method for fast wedgelet pattern decision in 3D-HEVC

The 3D High Efficiency Video Coding (3D-HEVC) is the latest 3D extension of the HEVC video coding standard. It supports multi-view videos plus depth (MVD), which is a sophisticated format for enhanced 3D content. Depth modeling modes (DMM) are adopted in the 3D-HEVC for better sharp edge encoding. However, employing DMM causes important computational complexity. In this paper, we propose a new method for fast wedgelet pattern decision in 3D-HEVC. In order to avoid unnecessary evaluation of the DMMs, a new early termination based on the smoothness of the current block is adopted. Besides, based on the shuffled frog leaping algorithm, a new heuristic method to search the optimal wedgelet pattern for DMM1 Depth mode is introduced. The proposed algorithm outperforms the HTM16.0 implemented scheme in terms of encoding time with similar coding efficiency.
Still image coding using a wavelet-like transform

In this paper, a new image coding scheme based on a wavelet-like transform derived from orthogonal polynomial basis is presented. From a set of bivariate orthogonal polynomial functions, we first obtain the 2D non-separable wavelet functions to propose a wavelet-like transform coding. The motivation behind using orthogonal polynomials is that they exhibit some properties related to the human visual system (HVS) . After applying the proposed transformation, the transform coefficients are threshold coded using quantization and bit allocation as in JPEG baseline system. The performance of the proposed transform coding is reported. The proposed coding scheme is also compared with other transform coding methods such as JPEG, JPEG 2000 and JPEG-XR/HDPHOTO.
Enhancing the Intra-prediction in JPEG-XR by Using Edge Information

The new image coding standard, JPEG-XR, uses an inter-block prediction mode to remove redundancy in the quantized transform coefficients. However, it is arguable whether the prediction modes in JPEG-XR are efficient enough for accurate prediction. This paper proposes a new inter-block prediction method which adapts to local image feature, i.e. edges. Since edges represent distinct image structures, they correspond to the local directional features within blocks. Taking advantage of this property, we propose a two steps inter-block prediction method. First, edges orientations are obtained. Then, these orientations are used to direct the inter- block prediction. The proposed prediction method is compared with that in JPEG-XR. The proposed method can reduce the encoding time and improve the PSNR.
A robust content-based JPWL transmission over a realistic MIMO channel under perceptual constraints

This paper proposes a global approach of JPWL (ISO/IEC 15444-11) image transmission over a realistic wireless channel able to ensure the best Quality of Service (QoS). In order to exploit the channel diversity, we consider a Closed-Loop MIMO-OFDM scheme with different precoder designs. In particular, the high flexibility of QoS precoder allows taking into account the scalability of JPWL jointly with the instantaneous MIMO channel status. This increases the visual quality of received images. The monitoring of the quality is made by a reduced-reference metric (QIP) based on object's saliency and interest point, both linked to human perception. It is performed in association with a robust JPWL decoder to determine the optimal decoding configuration in terms of PSNR. The proposed scheme provides very good results and its performance is shown through a realistic wireless channel.
On the selection of residual formula for HDR video coding

High Dynamic Range (HDR) videos are commonly compressed using existing Low Dynamic Range (LDR) coding standards. This necessarily implies a Tone Mapping (TM) stage which aims at reducing the input video range while preserving the most significant scene details. In most cases, a residual sequence is encoded and used at the decoder side to recover the HDR content. The challenge is to efficiently encode both LDR sequences (TM and residual) without compromising the quality of the reconstructed video. Furthermore, the choice of the residual sequence can have an important impact on the HDR video coder performances. In this work, we aim to study and analyse different approaches for residual determination. The performance is discussed in terms of entropy of the residual and video coding. Objective assessments show that differential residual presents much higher efficiency than ratio residuals in terms of bandwidth saving and HDR reconstruction quality.
Perceptual quality assessment for color image inpainting

A novel objective measure for assessing the quality of image in-painting is proposed. In contrast to standard image quality metrics, the proposed one takes into account some constraints and characteristics related to the specific goals of inpainting techniques. The idea is to combine spatial low-level features and perceptual criteria in the design of the objective Image Inpainting Quality Metric (IIQM). The used characteristics are the visual coherence of the recovered regions and the visual saliency describing the visual importance of an area. Experimental results demonstrate the good performance of the proposed IIQM and its well adaptation to the evaluation of image inpainting results.
Multi-resolution patch and window-based priority for digital image inpainting problem

Recently, digital image inpainting has attracted strong research interest because of its extensive applications in real life. The terminology “inpainting” refers to automatic restoration of image defects such as scratches or blotches as well as removal of unwanted objects as, for instance, subtitles, logos, etc, such that it is undetectable by viewers without the reference to the original image. Many works on this subject have been published in recent years. This paper introduces a novel unsupervised image completion framework using a modified exemplar-based method in conjunction with a pyramidal representation of an image. A top-down iterative completion is performed gradually with multi-resolution patches and a window-based priority. The proposed approach is verified on different natural images. Also, a comparison with some existing methods coming from literature is carried out and the results show improvement in favor of our approach.
Optimizing the disparity map by the integration of HVS binocular properties for efficient coding of stereoscopic images

Stereoscopic imaging technologies are seen as the next generation of visual presentation, improving the quality of experience of the viewer. It uses two different sequences acquired from two regular cameras or from a regular camera with an additional specific depth camera. This means that the size of data is at least doubled. Thus, the coding process becomes very crucial. In this framework, we propose a stereoscopic coder based on visual properties. The matching of two images is computed by a binocular energy model based on the simple and complex cells functions allowing the fusion of both retinal images in the visual cortex. Mathematical functions were used to reproduce the behavior of these cells particularly complex wavelet transform (CWT) and bandelet transform. Our coder output is a disparity map, a residual image and the reference image. The innovative part of this work lies in a matching technique based on the binocular energy. The results are presented through comparative curves with one of the most known coder in literature.
Spatio-temporal modeling of visual attention for stereoscopic 3D video

Modeling visual attention is an important stage for the optimization of image processing systems nowadays. Several models have been already developed for 2D static and dynamic content, but only few attempts can be found for stereoscopic 3D content. In this work we propose a saliency model for stereoscopic 3D video. This model is based the fusion of three maps i.e. spatial, temporal and depth. It relies on interest point features known for being close to human visual attention. Moreover, since 3D perception is mostly based on monocular cues, depth information is obtained using a monocular model predicting the depth position of objects. Several fusion strategies have been experimented in order to determine the best match for our model. Finally, our approach has been validated using state-of-the-art metrics in comparison to attention maps obtained by eye-tracking experiments, and showed good performance.
Towards light-compensated saliency prediction for omnidirectional images

Omnidirectional or 360-degree images is becoming very popular in many applications and several challenges are raised because of the nature and the representation of the data. The saliency prediction for such a content opens the door to many problems linked to the geometric distortions, lighting variation, … In this paper, we propose a saliency model taking advantage of the large literature of 2D saliency and offering three major adjustments related to the nature of 360-degree images : 1) illumination normalization to account for the variability of lighting over the scene, 2) distortion compensation to handle the conversion problem from the sphere to the equi-rectangular representation, and 3) equator bias to incorporate the perceptual property according to which the human gaze is biased towards the equator line. The obtained results showed an improvement of the performance of the 2D saliency when using the above adjustments for omnidirectional images.
A fast color quantization using a matrix of local pallets

We present a robust color quantization method based on the matrix of local pallets. The method proposed extracts a set of one-dimensional colors resulting from image partitioning. Image windowing depends upon the image variance which gives information on color dispersion. The color sets are then used to form the rows of the local pallet matrix that will be used as a smaller image. The selection of the principal pallet used to quantize the color image is accomplished on the local pallet matrix by computing the histogram, dividing the color interval adoptively and choosing the most frequent color with respect to interval total frequency. The experimental results give a good visual appearance and show that the method is very fast.
Feature-Based Color Correction of Multiview Video for Coding and Rendering Enhancement

Multiview video (MVV) consists of capturing the same scene with multiple cameras from different viewpoints. Therefore, substantial illumination and color inconsistencies can be observed between different views. These color mismatches can significantly reduce compression efficiency and rendering quality. In this paper, we propose a preprocessing method for correcting these color discrepancies in MVV. To consider the occlusion problem, our method is based on an improvement of histogram matching (HM) algorithm using only common regions across views. These regions are defined by an invariant feature detector (scale invariant feature transform), followed by random sample consensus algorithm to increase the matching robustness. In addition, to maintain temporal correlation, HM algorithm is applied on a temporal sliding window, allowing to cope with time-varying acquiring system, camera moving capture, and real-time broadcasting. Moreover, unlike always choosing the center view as the reference by default, we propose an automatic selection algorithm based on both views statistics and quality. The experimental results show that the proposed method increases coding efficiency with gains of up to 1.1 and 2.2 dB for the luminance and chrominance components, respectively. Furthermore, once the correction is performed, the color of real and rendered views is harmonized and looks very consistent as a whole.
Stereoscopic image quality metric based on local entropy and binocular just noticeable difference

Developing a metric that can reliably predict the perceptual 3D quality as perceived by the end user, is a challenging issue and a necessary tool for the success of 3D multimedia applications. The various attempts at predicting 3D quality of experience as the combination of 2D quality of the left and right images have shown their limitations, and particularly for the case of asymmetric distortions. In this paper we propose a full reference quality assessment metric for stereoscopic images based on the perceptual binocular characteristics. The proposed metric handles effectively the asymmetric distortions of stereoscopic images, by incorporating human visual system (HVS) characteristics. Our approach was motivated by the fact that in case of asymmetric quality, 3D perception mechanisms supports the view providing the most important and contrasted information. To achieve that, weighting factors are defined for the quality of each view according to the local information content. Add to that, to take into account the sensitivity of the HVS, quality score of each region are modulated based on the Binocular Just Noticeable Difference (BJND). Experimental results show that the proposed metric correlates better with human perception than the state-of-the-art metrics.
A reduced-reference metric based on the interest points in color images

In the last decade, an important research effort has been dedicated to quality assessment from subjective and objective points of view. The focus was mainly on Full Reference (FR) metrics because of the ability to compare to an original. Only few works were oriented to Reduced Reference (RR) or No Reference (NR) metrics, very useful for applications where the original image is not available such as transmission or monitoring. In this work, we propose a RR metric based on two concepts, the interest points of the image and the objects saliency on color images. This metric needs a very low amount of data (lower than 8 bytes) to be able to compute the quality scores. The results show a high correlation between the metric scores and the human judgement and a better quality range than well-known metrics like PSNR or SSIM. Finally, interest points have shown that they can predict the quality of compressed color images.
A Stereo Image Coding Using Wavelet Based Quadtree and Binocular Rivalry

In this work, a new stereo image coding technique is proposed. The encoder takes into account the case of relatively different images, by reproducing the phenomenon of binocular rivalry. This approach integrates residual image coding with the disparity map. The latter is computed in the wavelet transform domain. The motivation behind using this transform lies in its similarity with some proprieties of the human visual system (HVS); Particularly, the decomposition in perceptual channels. Therefore, the usage of wavelet transform allows better preservation of visual quality. In order to estimate the disparity map, we used quadtree segmentation in each wavelet frequency band. This segmentation has the advantage of minimizing the entropy, and will allow to match the reference (left) image and the target (right) image. Dyadic squares in the subbands generate the residual image coded by using an arithmetic coder. The results, of the proposed method, are evaluated in comparison to state of the art methods, using the PSNR criteria.
Perceptually Driven Nonuniform Asymmetric Coding of Stereoscopic 3D Video

Asymmetric stereoscopic video coding has already proven its effectiveness in reducing the bandwidth required for stereoscopic 3D delivery without degrading the visual quality. This approach, in which the left and right views are encoded with different levels of quality, relies on the perceptual theory of binocular suppression. However, to ensure comfortable 3D viewing, the just-noticeable level of asymmetry, i.e., the maximum quality gap between views, has to be carefully defined. Both subjectively and empirically fixed thresholds of asymmetry demonstrated either the maladjustment to content or dependency to the experimental design. This paper describes a new nonuniform asymmetric stereoscopic video coding method adaptively adjusting the level of asymmetry for each region of the image based on its perceptual significance. The proposed method uses a fully automated model that dynamically determines the best bounds of asymmetry for which the 3D viewing experience will not be altered. This is achieved by exploiting several human-visual-system-inspired models, namely, the binocular just-noticeable difference, and the visual saliency map and depth information. The simulation results show that the proposed method results in bit rate saving of up to 26% and provides better 3D visual quality compared with state-of-the-art asymmetric coding methods.
Perceptually-adaptive quantization for stereoscopic video coding

In this paper, we present a novel perceptually-based optimization for the improvement of stereoscopic video coding efficiency. The main idea of this proposed scheme is to adaptively adjust the quantization parameter by taking into account the Human Visual System perceptual characteristics. For this, a saliency map is generated from both views and then segmented into salient and non-salient regions. To make the proposed scheme effective, and inspired from the binocular suppression theory, the asymmetry is ensured by altering the saliency map and not the view. As a result, the proposed perceptual coding scheme effectively reduces the bit-budget without affecting the perceptual quality based on an optimization approach with asymmetric video coding taking into account the saliency map of each view. Experimental results on HEVC-MV show that the proposed algorithm can achieve over 20% bit-rate saving while preserving the perceived image quality.
On the Efficiency of Image Metrics for Evaluating the Visual Quality of 3D Models

3D meshes are deployed in a wide range of application processes (e.g., transmission, compression, simplification, watermarking and so on) which inevitably introduce geometric distortions that may alter the visual quality of the rendered data. Hence, efficient model-based perceptual metrics, operating on the geometry of the meshes being compared, have been recently introduced to control and predict these visual artifacts. However, since the 3D models are ultimately visualized on 2D screens, it seems legitimate to use images of the models (i.e., snapshots from different viewpoints) to evaluate their visual fidelity. In this work we investigate the use of image metrics to assess the visual quality of 3D models. For this goal, we conduct a wide-ranging study involving several 2D metrics, rendering algorithms, lighting conditions and pooling algorithms, as well as several mean opinion score databases. The collected data allow (1) to determine the best set of parameters to use for this image-based quality assessment approach and (2) to compare this approach to the best performing model-based metrics and determine for which use-case they are respectively adapted. We conclude by exploring several applications that illustrate the benefits of image-based quality assessment.
Blind inpainting forgery detection

The increasingly use of digital images in our daily life and the availability of powerful software for processing and editing images, open new challenges regarding illegal or unauthorized image manipulation. Thus, it becomes essential to authenticate digital copies, validate their content, and detect possible forgeries. In this paper, we focus on detection of a specific type of digital forgery, inpainting, where an object is removed using pixels coming from the same image. The problem of inpainting detection is investigated and an efficient and reliable detection method is proposed. The performance of the proposed method is demonstrated on several inpainted images using different inpainting techniques.
Camera motion influence on dynamic saliency central bias

Saliency models have been extensively studied for static images and the focus is now on moving images. There is a central bias in both cases that is emphasized in the dynamic case. One aspect in this latter is the camera motion that influences the scene interpretation. The movie director exploits this motion to make the observer focus on the targeted object which is often in the center of the scene. This aspect is not taken into account in current saliency dynamic models. In this paper, we study the camera motion influence on the gaze distribution in order to include it in a new saliency model. Observers' gazes are recorded with an eye tracker, camera motions (e.g. tracking, zoom…) are calculated thanks to a polynomial projection of the motion field and the motion influence is statistically tested on the recorded gazes.
Perceptually Adaptive Lagrangian Multiplier for HEVC Guided Rate-Distortion Optimization

Recent video coding standards typically use the Rate-distortion optimization (RDO) method, which is essential to appropriately perform mode decisions during encoding process. The newest standard high efficiency video coding (HEVC) introduces complex encoding structures and strong dependency between coding units. Particularly, the Lagrangian multiplier is a primary factor in RDO procedure, which directly affects the rate-distortion (R-D) performance and is defined for an entire video frame. This paper proposes a novel approach for perceptually guiding the RDO process in HEVC. The reference encoder does not consider effectively the perceptual characteristics of the input video and further, the visual sensitivity of each coding tree unit (CTU) in a frame. Inspired by the mechanisms of the human visual system, the proposed solution is a CTU-level adjustment of Lagrangian value based on a set of complementary perceptual features. The proposed scheme concerns important visual information of a CTU and its temporal dependency with adjacent blocks. Feature extraction is implemented in the frequency domain using efficient spatio-temporal analysis. In our experiments, we opted a perceptual mean squared error (MSE) metric and structural similarity (SSIM) index. According to perceptual MSE metric, the BD-rate savings using the Bjontegaard delta measurements, were fairly convincing over the state-of-the-art HEVC software HM16.12; 4.41% and 6.14% for random access (RA) and low delay (LD) encoding settings, respectively. Using SSIM, the BD-Rate achieved 6.95% and 9.86% for RA and LD settings, respectively. The proposed method further demonstrates a superior R-D performance over a compared approach adopting a similar scheme.