nils-herrmann/clean_scirepeval huggingface dataset:
Video quality enhancement via QP adaptation based on perceptual coding maps

This paper introduces a method for adapting block quantisation parameter values in HEVC video compression based on perceptual coding maps. These maps are computed per block taking into account masking effects. Masking levels are calculated using spatial, temporal and foveation features that are extracted from the video and are stored in a perceptual coding map. The produced map drives a QP adaptation process that aims to redistribute coding bits in the frame so that the perceived quality is improved, especially at those bitrates where coding artifacts become visible (mid to high QP values). The subjective performance evaluation that was conducted showed that the proposed method can offer a measurable improvement in perceived quality relative to a constant QP approach, with Bj⊘ntegaard mean opinion scores (MOS) gains reaching almost 9% for the test sequences used. The paper additionally highlights the need for further work in order to increase gains in perceived quality and optimise parameter selection.
The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives

Three-dimensional video content has attracted much attention in both the cinema and television industries, because 3D is considered to be the next key feature that can significantly enhance the visual experience of viewers. However, one of the major challenges is the difficulty in providing high quality images that are comfortable to view and that also meet signal transmission requirements over a limited bandwidth for display on television screens. The different processing steps that are necessary in a 3D-TV delivery chain can all introduce artifacts that may create problems in terms of human visual perception. In this paper, we highlight the importance of considering 3D visual attention when addressing 3D human factors issues. We provide a review of the field of 3D visual attention, discuss the challenges in both the understanding and modeling of 3D visual attention, and provide guidance to researchers in this field. Finally, we identify perceptual issues generated during the various steps in a typical 3D-TV broadcasting delivery chain, review them and explain how consideration of 3D visual attention modeling can help improve the overall 3D viewing experience.
Quantifying the relation between perceived interest and visual salience during free viewing using trellis based optimization

Eye movements are principally driven by two factors: bottom up salience and top-down perceived interest that is based on our semantic interpretation of the scene. The two effects interact strongly while free viewing natural scenes. However, the exact extent to which each of these factors control overt eye movements, and also their variations over time, though an important aspect, is not well understood. By performing a trellis based optimization on existing state-of-the-art Saccadic Models, using the ground truth data obtained from a carefully controlled eye-tracking experiment, we are able to quantify the extent to which top-down and bottom-up factors interact during the course of free viewing natural images. In line with the claims of earlier qualitative studies, we are able to measure that, gaze, though initially bottom-up driven, is soon countermanded by top-down perceived interest, in the time range of 200 ms-2 secs. At a much later duration however, the bottom-up effects reappear again, albeit to an extent lesser than that of initial levels. It is therefore extremely paramount, to give enough importance to the top-down effects of attention in addition to bottom-up visual salience, during the analysis of overt eye-movement data.
Diagnostic quality assessment of medical images: Challenges and trends

With medical imaging technologies growth, the question of their assessment on the impact and benefit on patient care is rising. Development and design of those medical imaging technologies should take into account the concept of image quality as it might impact the ability of practicians while they are using image information. Towards that goal, one should consider several human factors involved in image analysis and interpretation, e.g. image perception issues, decision process, image analysis pipeline (detection, localization, characterization…). While many efforts have been dedicated to objectively assess the value of imaging system in terms of ideal decision process, new trends have recently emerged to deal with human observer perfomances. This task effort is huge considering the variability of imaging acquisition methods and the possible pathologies. This paper proposes a survey of some key issues and results associated to this effort. We first outline the wide range of medical images with their own specific features. Next, we review the main methodologies to evaluate diagnostic quality of medical images from subjective assessment including ROC analysis, and diagnostic criteria quality analysis, to objective assessment including metrics based on the HVS, and model observers. At last, we present another evaluation method: eye-tracking studies to gain basic understanding of the visual search and decision-making process.
Impact of visual angle on attention deployment and robustness of visual saliency models in videos: From SD to UHD

The emergence of UHD video format induces larger screens and involves a wider stimulated visual angle. Therefore, its effect on visual attention can be questioned since it can impact quality assessment, metrics but also the whole chain of video processing and creation. Moreover, changes in visual attention from different viewing conditions challenge visual attention models. In this paper, we present a comparative study of visual attention and viewing behavior on three video datasets in SD, HD and UHD conditions. Then, we propose and assess an improvement for video visual attention models by applying a stimulated visual angle dependent center model.
The influence of relative disparity and planar motion velocity on visual discomfort of stereoscopic videos

The vergence-accommodation conflict, excessive screen disparity, binocular distortions and the motion component in stereoscopic videos are considered as main factors that may induce visual discomfort. In our previous study which was based on the experts-only experiment, we also found that the large relative disparity between the foreground and background and the fast planar motion were more likely to induce visual discomfort. In this study, we conducted the same subjective experiment but on non-expert observers. The subjective experiment results coincided with our previous findings. The two objective visual discomfort models developed in our previous study have been evaluated and showed high correlation with subjective data. Finally, we found that the observers could be classified into different clusters according to their visual discomfort sensitivity to the velocity or the relative disparity. For some observers, the velocity is the predominant factor that may induce visual discomfort; some consider that the relative disparity is the key factor, and some are sensitive to both the velocity and relative disparity.
Learning visual saliency for stereoscopic images

Currently, there are various saliency detection models proposed for saliency prediction in 2D images/video in the previous decades. With the rapid development of stereoscopic display techniques, stereoscopic saliency detection is much desired for the emerging stereoscopic applications. Compared with 2D saliency detection, the depth factor has to be considered in stereoscopic saliency detection. Inspired by the wide applications of machine learning techniques in 2D saliency detection, we propose to use the machine learning technique for stereoscopic saliency detection in this paper. The contrast features from color, luminance and texture in 2D images are adopted in the proposed framework. For the depth factor, we consider both the depth contrast and depth degree in the proposed learned model. Additionally, the center-bias factor is also used as an input feature for learning the model. Experimental results on a recent large-scale eye tracking database show the better performance of the proposed model over other existing ones.
Video quality assessment: From 2D to 3D — Challenges and future trends

Three-dimensional (3D) video is gaining a strong momentum both in the cinema and broadcasting industries as it is seen as a technology that will extensively enhance the user's visual experience. One of the major concerns for the wide adoption of such technology is the ability to provide sufficient visual quality, especially if 3D video is to be transmitted over a limited bandwidth for home viewing (i.e. 3DTV). Means to measure perceptual video quality in an accurate and practical way is therefore of highest importance for content providers, service providers, and display manufacturers. This paper discusses recent advances in video quality assessment and the challenges foreseen for 3D video. Both subjective and objective aspects are examined. An outline of ongoing efforts in standards-related bodies is also provided.
Characterization and selection of light field content for perceptual assessment

Light field technology may have a positive impact on several multimedia applications thanks to novel ways to explore the captured scenes, such as changing the parallax (horizontally and vertically) and refocusing the content. These innovative use cases require new considerations that affect the whole processing chain, from content acquisition to visualization, as well as the methodologies for quality evaluation. In particular, capturing and selecting the appropriate content is crucial for a successful development and evaluation of audiovisual technologies. Thus, this paper presents a framework addressing the reconsideration of the space of attributes for an adequate characterization of light field data. Firstly, an exhaustive characterization of light field content is described, based on various particular features, including depth and refocusing properties to traditional spatial, temporal, and color indicators. Then, based on this characterization, specific techniques are proposed for an effective selection of light field content for perceptual quality assessment.
Study on visual discomfort induced by stimulus movement at fixed depth on stereoscopic displays using shutter glasses

Stereoscopic motion images are able to provide observers with realistic and immersive viewing experience. However, observers often experience visual discomfort during the viewing process. In this paper, we investigated the relationship between visual discomfort and the planar motion at different depth levels. The Paired Comparison method was used in the subjective experiments to allow for a precise measurement. The experimental results indicated that the relative angular disparity between foreground object and background played a more important role in determining the visual discomfort than the vergence-accommodation conflict. Furthermore, the results showed that with the increase of planar motion velocity, viewers might experience more visual discomfort. To quantify the effects of relative angular disparity and velocity on visual discomfort, two visual discomfort models were constructed. The preferred model was chosen based on the performance as well as the algorithmic complexity. This model can be used as an index for other related researches.
Video quality perception in telesurgery

Telesurgery enables an expert surgeon to assist a remote surgeon during a surgical intervention, which benefits patient care in resource-poor settings. In reality, videos of surgical procedures are compressed and transmitted over large distances in real time and, therefore, are subject to a wide variety of distortions. These distortions degrade the quality of videos and potentially affect the performance of the surgeons. Very little work has been carried out on human perception of video quality in the context of telesurgery. In this paper, we investigate the impact of video compression on the perceived quality of surgical videos. We designed and performed a psychophysical experiment where surgeons rated the quality of surgical videos distorted with two different compression schemes at various compression ratios. Experimental results demonstrate that the impact of video content and compression strategy on the perceived quality is statistically significant.
Role of HEVC coding artifacts on gaze prediction in interactive video streaming systems

Sensitivity to spatial details drops across the visual periphery, and hence video streaming systems that gracefully degrades quality away from the viewpoint of the observer, provides an optimum viewing experience with potentially large bitrate savings. As reaction latency is an important performance parameter of such systems, good prediction of future gaze locations at the transmission end is very important. A major research question here is: whether a gaze prediction model designed using a pristine undistorted video, is also able to predict the gaze pattern of users when they watch a distorted/ adaptively distorted version of the same video. With several improvements to existing gaze prediction schemes, in combination with a controlled subjective experiment, we confirm not only that HEVC coding distortions have no significant impact on the predictability of gaze patterns, but also that gaze prediction errors can be restricted to 1.5 degrees of viewing angle for a round trip delay of up to 200ms.
Visual attention modeling for stereoscopic video

In this paper, we propose a computational model of visual attention for stereoscopic video. Low-level visual features including color, luminance, texture and depth are used to calculate feature contrast for spatial saliency of stereoscopic video frames. Besides, the proposed model adopts motion features to compute the temporal saliency. Here, we extract the relative planar and depth motion for temporal saliency calculation. The final saliency map is computed by fusing the spatial and temporal saliency together. Experimental results show the promising performance of the proposed method in saliency prediction for stereoscopic video.
Evaluating complex scales through subjective ranking

In this paper, a subjective assessment methodology based on ranking, reordering and image categorization is proposed. It is designed to facilitate the evaluation of complex features such as monocular depth cues in natural images. These features can be particularly challenging to explain to test participants, and the proposed methodology is designed to provide to the test participants more insights and examples about the scale under evaluation. The performances of the method are compared to the absolute category rating methodology, and the advantages of each part of the test design are studied to verify the added value of the different steps of the evaluation procedure. Results show that the method is promising and can improve stability between subjective test participant results.
A Study of Content based Watermarking using an Advanced HVS Model

A key point of image watermarking schemes is to select the image pixels or transformed coefficients where an embedded watermark would reach the optimal invisibility versus robustness trade-off. Human visual system (HVS) based JND masks are thus useful to optimally adapt the watermark strength right below the visibility threshold. Most of the perceptual masks created so far for watermarking purpose are based on the empirical assumption that edges and/or textures allow the best watermark visual masking and thus ensures good invisibility properties. Nevertheless, although invisibility control in smooth areas is challenging, it could be interesting to extend the watermark in such areas since the robustness performances might increase. The goal of this paper is to use an advanced HVS model to manage invisibility in order to evaluate the robustness of watermarks embedded in smooth areas only, edges/textured areas only or in both areas. This study is further to the one presented in (F. Autrusseau and P. Le Callet, 2007). The robustness was tested against 89 attacked images (using Stirmark), results showed that including smooth areas in the JND mask increases the robustness while preserving good invisibility properties.
Experimenting texture similarity metric STSIM for intra prediction mode selection and block partitioning in HEVC

Textures can often be found in large areas of images and videos. They have different spectral and statistical properties as compared with normal (structural) components. Encoding them with ordinary video coders requires higher bit rate and usually results are unsatisfying in perceived quality. Recently, different perceptual tools have been developed to estimate the perceived quality of textures taking into account models of human visual system. In this paper, we investigate and discuss the practical usability of one of these tools, namely STSIM, as a distortion function for selecting the intra-prediction mode and block partitioning of texture images in HEVC. We experiment few practical implementations to examine its performance compared with default metrics used by HEVC. Experimental results showed that the perceived quality of the decoded textures has been significantly improved specially for stochastic types of textures.
Evaluating Depth Perception of 3D Stereoscopic Videos

3D video quality of experience (QoE) is a multidimensional problem; many factors contribute to the global rating like image quality, depth perception and visual discomfort. Due to this multidimensionality, it is proposed in this paper, that as a complement to assessing the quality degradation due to coding or transmission, the appropriateness of the non-distorted signal should be addressed. One important factor here is the depth information provided by the source sequences. From an application-perspective, the depth-characteristics of source content are of relevance for pre-validating whether the content is suitable for 3D video services. In addition, assessing the interplay between binocular and monocular depth features and depth perception are relevant topics for 3D video perception research. To achieve the evaluation of the suitability of 3D content, this paper describes both a subjective experiment and a new objective indicator to evaluate depth as one of the added values of 3D video.
Performance and robustness of HDR objective quality metrics in the context of recent compression scenarios

The new High Dynamic Range (HDR) technologies intend to increase the immersiveness and quality of experience of the observers thanks to much larger luminance and contrast characteristics. This increase of the amount of information requires the development of new sophisticated compression algorithms. Moreover, well-known objective quality metrics designed for Low Dynamic Range (LDR) systems may not be directly used for assessing HDR videos. To address this issue, several different ways have been proposed. However, their performance and robustness in the context of the most recent HDR compression software solutions need to be assessed and validated on various types of content. In this paper, four objective metrics are benchmarked from data obtained in a new subjective test conducted on various HDR video sequences.
CNN-based transform index prediction in multiple transforms framework to assist entropy coding

Recent work in video compression has shown that using multiple 2D transforms instead of a single transform in order to de-correlate residuals provides better compression efficiency. These transforms are tested competitively inside a video encoder and the optimal transform is selected based on the Rate Distortion Optimization (RDO) cost. However, one needs to encode a syntax to indicate the chosen transform per residual block to the decoder for successful reconstruction of the pixels. Conventionally, the transform index is binarized using fixed length coding and a CABAC context model is attached to it. In this work, we provide a novel method that utilizes Convolutional Neural Network to predict the chosen transform index from the quantized coefficient block. The prediction probabilities are used to binarize the index by employing a variable length coding instead of a fixed length coding. Results show that by employing this modified transform index coding scheme inside HEVC, one can achieve up to 0.59% BD-rate gain.
Using overlapping subjective datasets to assess the performance of objective quality metrics on Scalable Video Coding and error concealment

In this paper, four subjective video datasets are presented. The considered application is Scalable Video Coding used as an error-concealment mechanism. The presented datasets explore the relations between encoding parameters and perceived quality, under different network-impairment patterns and involve error-concealment on the decoder's side, to simulate a complete distribution channel. The datasets share a part of common configurations which enables, in the first part of the paper, to compare the outcomes from several Single Stimulus experiments and draw interesting correspondances between different types of distortion. In the second part of the paper, we analyse the performance of three common objective quality metrics on each step of the distribution channel, to identify the possible directions to be followed in order to improve their accuracy in predicting the perceived quality.
On the perceptual similarity of realistic looking tone mapped High Dynamic Range images

High Dynamic Range (HDR) images are usually displayed on conventional Low Dynamic Range (LDR) displays because of the limited availability of HDR displays. For the conversion of the large dynamic luminance range into the eight bit quantized values, parameterized Tone Mapping Operators (TMO) are applied. Human observers are able to optimize the parameters in order to get the highest Quality of Experience by judging the displayed LDR images on a realism scale. In the study presented in this paper, two TMOs with three parameters each were evaluated by observers in a subjective experiment. Although the chosen parameter settings vary largely, the chosen images appear to have the same QoE for the observers. In order to assess this similarity objectively, three commonly used image quality measurement algorithms were applied. Their agreement with the preference of the observers was analyzed and it was found that the Visual Difference Predictor (VDP) outperforms the Structural Similarity Index and the Root Mean Square Error. A threshold value for VDP is derived that indicates when two LDR images appear to have the same Quality of Experience.
An eye tracking database for stereoscopic video

We present a large-scale eye tracking database for stereo-scopic video. A set of participants were involved in this eye tracking experiment. The human fixation maps were created as the ground truth for stereoscopic video from the gaze data from participants. To the best of our knowledge, this is the first large-scale eye tracking database of visual attention modeling for stereoscopic video. The details of the processing operations and properties of the database are described in this paper.
How visual discomfort affects 3DTV viewers' emotional arousal

We present the results of a research on the effects of image distor-tions known to induce visual discomfort (namely, the introduction of Gaussian blur on one visual field) on the emotional reactions of 3DTV viewers. Emotional reactions were explored by means of self-reported as well as psychophysiological methods. The re-sults showed that, despite visual discomfort not having an effect on self-reported emotions, the level of electrodermal activity (a physiological correlate of emotional arousal) was clearly affected, suggesting the existence of seemingly complex effects of visual discomfort over viewers' emotional arousal.