nils-herrmann/clean_scirepeval huggingface dataset:
Image retargeting using a bandelet-based similarity measure

Media content retargeting aims to adapt images/ videos to displays of large or small sizes. In this work, we propose a bandelet-based image retargeting algorithm for summarizing image data into smaller sizes. First, we define a multi-scale bandelet-based perceptual similarity measure which measures the geometric and perceptual similarities between two images at different bandelet scales. Two images are said to be geometrically similar if they have approximately the same geometric flow and quadtree structure. After determining the geometric similarity, a perceptual similarity measure based on the properties of the human visual system is defined to assess the perceptual difference between the original image and the retargeted one. Then, the problem of image retargeting is considered as a geometric optimization problem based on the bandelet-based geometric and perceptual similarity measures. That is, for an image S we search for a retargeted image T that contains as much as possible of geometric and perceptual information from S and, consequently, preserves visual coherence. The proposed retargeting algorithm outperforms the state-of-the-art methods in terms of the visual quality of the retargeted image.
Using monocular depth cues for modeling stereoscopic 3D saliency

Saliency is one of the most important features in human visual perception. It is widely used nowadays for perceptually optimizing image processing algorithms. Several models have been proposed for 2D images and only few attempts can be observed for 3D ones. In this paper, we propose a stereoscopic 3D saliency model relying on 2D saliency features jointly with depth obtained from monocular cues. On the one hand, the use of 2D saliency features is justified psychophysically by the similarity observed between 2D and 3D attention maps. On the other hand, 3D perception is significantly based on monocular cues. The validation of our model using state-of-the-art procedures including Kullback-Leibler divergence (KLD), area under the curve (AUC) and correlation coefficient (CC) in comparison with attention maps showed very good performance.
Asymmetric coding of stereoscopic 3D based on perceptual significance

Asymmetric stereoscopic coding is a very promising technique to decrease the bandwidth required for stereoscopic 3D delivery. However, one large obstacle is linked to the limit of asymmetric coding or the just noticeable threshold of asymmetry, so that 3D viewing experience is not altered. By way of subjective experiments, recent works have attempted to identify this asymmetry threshold. However, fixed threshold, highly dependent on the experiment design, do not allow to adapt to quality and content variation of the image. In this paper, we propose a new non-uniform asymmetric stereoscopic coding adjusting in a dynamic manner the level of asymmetry for each image region to ensure unaltered binocular perception. This is achieved by exploiting several HVS-inspired models; specifically we used the Binocular Just Noticeable Difference (BJND) combined with visual saliency map and depth information to quantify precisely the asymmetry threshold. Simulation results show that the proposed method results in up to 44% of bitrate saving and provides better 3D visual quality compared to state-of-the-art asymmetric coding methods.
A stereoscopic quality metric based on binocular perception

This paper presents a new metric for the quality evaluation of stereoscopic images. The proposed metric is inspired from the simple and complex cells response, in which the matching of the left and right retinal images is performed, in the visual cortex. In order to estimate the binocular energy, we used the bandelet transform which is computed on complex wavelet transform. By using the complex wavelet transform, we were able to model the behavior of the simple and complex cells. The proposed model allows the estimation of the degradation of the binocular energy according to the impairments of the right and the left images. Subjective tests have been performed to validate the results obtained by our metric. The results shows a high correlation between the predicted score and the human judgement.
Stereoscopic video coding based on the H.264/AVC standard

H.264/AVC is newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. Stereo video technology is an active research area in current video signal processing filed. More and more researchers are engaged in this field. In this paper, stereoscopic video coding is studied, and three schemes of using H.264/AVC are discussed. An improved scheme is proposed after the three stereo video coding were analyzed and compared. The proposed scheme can encode stereoscopic video sequences effectively.
A block artifact distortion measure for no reference video quality evaluation

In This paper, we propose a new perceptually significant video quality metric to estimate the effect of block coding for standards H.264 AVC and MPEG2. Our method operates in the spatial domain and doesn't require a high complexity of computation. We compare it with Suthaharan's and MSU's techniques by using “LIVE” database. Results indicate that the proposed method outperforms Suthaharan's technique. They also indicate that our method is more effective than MSU's technique for the H.264 AVC and MPEG2 standard with the Spearman Rank Order Correlation Coefficient.
Influence of video resolution, viewing device and audio quality on perceived multimedia quality for steaming applications

Objective and subjective quality assessment of images and videos have been an active research topic in the recent years. Multimedia technologies require new quality metrics and methodologies taking into account the fundamental differences in the human visual perception and the typical distortions of both video and audio modalities. Because of the increase of multimedia content platforms (Streaming, IPTV, OTT, etc) to delivering video content over the open Internet to a variety of devices; from TVs, to tablets, and smart phones, the Quality of Experience (QoE) may change. In this work we propose to evaluate the mutual interaction between audio and video and their influence on the perceived quality in case of streaming video applications. From one side, we carried out subjective experiments for assessing audio-only, video-only and audiovisual quality to create an audiovisual database that contains a wide range of degradations. From another side, a statistical analysis has been performed to investigate the influence of video resolution, viewing device and audio quality on perceived audiovisual quality. The Results show that audio quality plays a crucial role in the judgment of perceived quality especially when its quality is poor.
Colour image super-resolution using geometric grouplets

In this work, a technique for generating a super-resolution (SR) image from a single multi-valued low-resolution (LR) input image is proposed. This problem is approached from the perspective of image geometry-oriented interpolation. First, the geometry of the LR image is obtained by computing the grouplet transform. These grouplet bases are used to define a grouplet-based structure tensor to capture the geometry and directional features of the LR colour image. Then, the SR image is synthesised by an adaptive directional interpolation using the extracted geometric information to preserve the sharpness of edges and textures. This is accomplished by the minimisation of a functional, which is defined on the extracted geometric parameters of the LR image and oriented by the geometric flow defined by the grouplet transform. The proposed SR algorithm outperforms the state-of-the-art methods in terms of visual quality of the interpolated image.
A New approach of Rate-Quantization modeling for Intra and Inter frames in H.264 rate control

Video encoding rate control has been the research focus in the recent years. The existing rate control algorithms use Rate-Distortion (R-D) or Rate-Quantization (R-Q) models. These latter assume that the enhancement of the bit allocation process, the quantization parameter determination and the buffer management are essentially based on the improvement of complexity measures estimation. Inaccurate estimation leads to wrong quantization parameters and affects significantly the global performance. Therefore, several improved frame complexity measures are proposed in literature. The efficiency of such measures is however limited by the linear prediction model which remains still inaccurate to encode complexity between two neighbour frames. In this paper, we propose a new approach of Rate-Quantization modeling for both Intra and Inter frame without any complexity measure estimation. This approach results from extensive experiments and proposes two Rate-Quantization models. The first one (M1) aims at determining an optimal initial quantization parameter for Intra frames based on sequence target bit-rate and frame rate. The second model (M2) determines the quantization parameter of Inter coding unit (Frame or Macroblock) according to the statistics of the previous coded ones. This model substitutes both linear and quadratic models used in H.264 rate controller. The simulations have been carried out using both JM10.2 and JM15.0 reference softwares. Compared to JM10.2, M1 alone, improves the PSNR up to 1.93dB, M2 achieves a closer output bit-rate and similar quality while the combined model (M1+M2) minimizes the computational complexity. (M1+M2) outperforms both JM10.2 and JM15.0 in terms of PSNR.
Robust foveal wavelet-based object tracking

In this work, a foveal wavelet-based Mean Shift Tracking Algorithm is presented. The foveal wavelets introduced by Mallat [16] are known by their high capability to precisely characterize the holder regularity of singularities. Therefore, by using the foveal wavelet transform, image features are accurately identified and are well discriminated from noise. These wavelets are used to extract the texture features of the target object. The extracted features are then used to construct a joint color-foveal textures histogram to represent the target object. Once the joint histogram is obtained, it is applied to the mean shift framework in order to track a target object in a video sequence. The experimental results showed that the proposed approach overcomes the traditional mean shift tracking technique as well as other existing tracking algorithms.
Adaptive quantization based on saliency map at frame level of H.264/AVC rate control scheme

In the frame-level rate control algorithm of H.264/AVC baseline profile, the Quantization Parameter (QP) is determined according to a target bit-rate estimated by the bit allocation process and a frame complexity measure. The target bits calculation process assumes similar characteristics for all frames and for a single frame on the whole set; the Mean Average Difference (MAD) is used as complexity measure. As a result, inappropriate QP values are selected leading to inefficient bit allocation. To overcome this limitation, we propose a new frame level QP adjustment. The proposed algorithm exploits a saliency map to allocate more bits to more salient frames. The saliency map is produced simulating the human visual attention by a bottom-up (signal-dependent) approach. Experimental simulations conducted at low bit-rates show that the proposed approach reduces bit-rates up to 26% without visual quality loss when compared to H.264/AVC JM15.0 reference software.
Blind image quality assessment in the complex frequency domain

In this paper, we propose a no-reference (NR) image quality assessment (IQA) metric that operates in the complex frequency domain. A set of features are developed to model the natural scene statistics without depending on any specific visual distortion. The proposed approach relies on a statistical analysis of the transformed image, involving the importance of the phase and magnitude provided by the underlying complex coefficients. We further investigate the correlation between the different image spatial-frequency resolutions, i.e., representations under different scales and orientations in order to extract the directional features and energy distributions of an image. The validation of the NR metric is performed on a variety of challenging IQA databases and the obtained results show good correlation with subjective scores. Besides, the obtained performance is highly competitive compared to the top-performing NR IQA metrics.
No-reference perceptual blur metric for stereoscopic images

In this paper, we propose a no-reference perceptual blur metric for 3D stereoscopic images. The proposed approach relies on computing perceptual local blurriness map for each image of the stereo pair. To take into account the disparity/depth masking effect, we modulate the obtained perceptual score at each position of the blurriness maps according to its location in the scene. Under the assumption that, in case of asymmetric stereoscopic image quality, 3D perception mechanisms place more emphasis on the view providing the most important and contrasted information, the two derived local blurriness maps are combined using weighting factors based on local information content. Thanks to the inclusion of those psychophysical findings, the proposed metric handles efficiently symmetric as well as asymmetric distortions. Experimental results show that the proposed metric correlates better with human perception than state-of-the-art metrics.
Visual coherence metric for evaluation of color image restoration

Digital image inpainting technique could provide substantial support for future restoration of images. However, much less effort has been devoted to the development of image inpainting evaluation. In this paper, we proposed a novel objective metric for assessing the quality of color image inpainting which takes into account some constraints and characteristics related to the specific objectives of inpainting approaches. The used characteristics are the visual coherence of the recovered regions and the visual saliency describing the visual importance of the area. A series of psychophysical experiments have been conducted to evaluate the performance of the proposed image quality index. The results show that the proposed image quality metric is well adapted for image quality inpainting evaluation.
A grouplet-based reduced reference image quality assessment

The past decades have witnessed the tremendous growth of digital image processing techniques for visual information representation and communication. Particularly, computational representation of perceived image quality has become a fundamental problem in computer vision and image processing. It is well known that the commonly used Peak Signal-to-Noise Ratio (PSNR), although analysis friendly, falls far short of this need. In this work, we propose a reduced reference (RR) perceptual image quality measure (IQM) based on the grouplet transform. Given a reference image and its “distorted” version, we first compute the grouplet transform in order to extract the information of textures and directions of both images. Then, contrast sensitivity function (CSF) filtering is performed to obtain same visual sensitivity information within both images. Thereafter, based on the properties of the human visual system (HVS), rational sensitivity thresholding is performed to obtain the sensitivity coefficients of both images. Finally, RR image quality assessment (IQA) is performed by comparing the sensitivity coefficients of both images.
Spatial-temporal Video Quality Metric based on an estimation of QoE

In this work a new Reduced Reference (RR) Video Quality Metric (VQM) is proposed. The method takes advantage of the Human Visual System (HVS) sensitivity to sharp changes in the video. The proposed method has a spatial-temporal approach and because of that it is named as STAQ (Spatial-Temporal Assessment of Quality). In the first step of STAQ we take a temporal approach and find the matching regions in consecutive frames. In the next step, a spatial approach is taken in the way of calculating the quality of the matching regions in the temporal approach. In the last step, the quality of the video is calculated based on the parameters gathered in the spatial and temporal domain and using the motion activity density of the video as a controlling factor. An important improvement lies in taking into account the Quality of Experience (QoE) represented as the motion activity density of the reference video. The results show a great improvement in the case of H.264 and MPEG-2 compressed and IP distorted videos even when compared to state of the art Full Reference (FR) metrics.
CYCLOP: A stereo color image quality assessment metric

In this work, a reduced reference (RR) perceptual quality metric for color stereoscopic images is presented. Given a reference stereo pair of images and their “distorted” version, we first compute the disparity map of both the reference and the distorted stereoscopic images. To this end, we define a method for color image disparity estimation based on the structure tensors properties and eigenvalues/eigenvectors analysis. Then, we compute the cyclopean images of both the reference and the distorted pairs. Thereafter, we apply a multispectral wavelet decomposition to the two cyclopean color images in order to describe the different channels in the human visual system (HVS). Then, contrast sensitivity function (CSF) filtering is performed to obtain the same visual sensitivity information within the original and the distorted cyclopean images. Thereafter, based on the properties of the human visual system (HVS), rational sensitivity thresholding is performed to obtain the sensitivity coefficients of the cyclopean images. Finally, RR stereo color image quality assessment (SCIQA) is performed by comparing the sensitivity coefficients of the cyclopean images and studying the coherence between the disparity maps of the reference and the distorted pairs. Experiments performed on color stereoscopic images indicate that the objective scores obtained by the proposed metric agree well with the subjective assessment scores.
A macroblock-based perceptually adaptive bit allocation for H264 rate control

Statistical methodologies are the main tools used in video compression and this lead to a kind of stagnation in terms of performance. This means that solutions for increasing the visual performance of video compression have to come from other fields like the perception. One can notice that coding errors in highly textured areas are relatively less perceptible than errors in untextured ones because of the masking effect. The existing H.264/AVC bit allocation scheme does not take into account this phenomenon. To handle this problem, we propose an adaptive bit allocation based on spatial and temporal perceptual features. This is performed by determining a spatiotemporal importance factor that is used to adjust the number of allocated bits. The proposed spatial feature consists on classifying regions into three categories: flat, textured and edged regions. The proposed temporal feature assumes that the Human Visual System (HVS) is more sensitive for moving regions than static ones. So, a low amount of bits will be assigned to static textured regions, and large one will be assigned to moving regions, which are spatially edged or flat. Experimental results show that the proposed bit allocation at macroblock (MB) level, compared with H.264/AVC reference software, improves the average peak signal-to-noise ratio (PSNR) (up to +1.10dB) and preserves details in the most perceptually prominent regions for low bitrates.
New H.264 intra-rate estimation and inter-rate control driven by improved MAD-based Contrast Sensitivity

This paper aims to improve H.264 bit-rate control. The proposed algorithm is based on a new and efficient Rate-Quantization (R-Q) model for the intra frame. For the inter frame, we propose to replace the current use of MAD by a new MAD-based human Contrast Sensitivity (MAD-CS) which is a more accurate complexity measure. R-Q model for the intra frame results from extensive experiments. The optimal initial quantization parameter QP is based on both target bit-rate and complexity of I-frame. The I-frame target bit-rate is derived from the global target bit-rate by using a new non linear model. MAD-CS includes the contrast sensitivity of the human visual system and weights the absolute differences by the probability of their occurrence. Extensive simulation results show that the use of MAD-CS and the proposed R-Q model achieves better rate control for intra frames, reduces the bit-rates when compared to the H.264 rate control adopted in JM reference software, minimizes the peak to signal ratio variations among encoded pictures and increases significantly as well subjective visual quality (measured by psycho visual experiments) as objective one.
Quality estimation based on interest points through hierarchical saliency maps

Quality of Experience (QoE) is a widely used notion nowadays because the end-user has been re-integrated in the quality loop. Subjective experiments are tedious and time consuming but to date they are the main way to have the human judgment. An important effort has been put on the development of metric estimating the QoE. So, in this paper, we propose a new image quality metric based on two concepts: the interest points and the objects saliency on color images. This metric is constructed by taking advantage of the variability of the interest points impact function of the image impairment and also the hierarchical attention of a human observer. This combination helps in giving more importance to the variability of the most salient regions and reducing the influence of regions having less visual importance. The results show a high correlation between the metric scores and the human judgment and a better quality range than well-known metrics.
Stereoscopic image quality assessment based on the binocular properties of the human visual system

One of the most challenging issues in stereoscopic image quality assessment (IQA) is how to effectively model the binocular behaviors of the human visual system (HVS). The latter has a great impact on the perceptual stereoscopic 3D (S3D) quality. This paper presents a stereoscopic IQA metric based on the properties of the HVS. Instead of measuring the quality of the left and the right views separately, the proposed method predicts the quality of a cyclopean image to ensure that the overall S3D quality is as close as possible to the binocular vision. The cyclopean image is synthesized based on the local entropy of each view with the aim to simulate the phenomena of the binocular rivalry/suppression. A 2D IQA metric is employed to assess the quality of both the cyclopean image and the disparity map. Additionally, the quality of the cyclopean image is modulated according to the visual importance of each pixel defined by the just noticeable difference (JND). Finally, the 3D quality score is derived by combining the quality estimates of the cyclopean image and disparity map. Experimental results show that the proposed method outperforms many other state-of-the-art SIQA methods in terms of prediction accuracy and computational efficiency.
Subjective and Objective Assesment of Visual Image Quality Metrics and Still Image Codecs

Objective quality assessment of lossy image compression codecs have become an important part of the recent call of the JPEG committee for advanced image coding. We evaluated JPEG with Huffman and arithmetic coding option, a visual and PSNR optimal JPEG2000 version, H.264/AVC and the recently proposed HDPhoto format by Microsoft. For objective evaluation, we use a color version of the M-SSIM metric and the high-dynamic range version of VDP. The results obtained from these tests are compared to subjective testing obtained from an ordering test run by 15 observers in two sessions. Subjective results are compiled to Mean Opinion Score (MOS) and passed through a Kurtosis test to verify their validity and to reject outliers.