nils-herrmann/clean_scirepeval huggingface dataset:
Exploring the effects of 3D visual discomfort on viewers' emotions

Much of the research on stereoscopic 3D (S3D) QoE has dealt with the effects of image features over user's visual discomfort. Nevertheless, the relationships between visual discomfort and other high-level perceptual dimensions such as the viewers' emotional reactions have not been yet explored. Since emotions play a key role on media reception processes, and especially in entertainment contents consumption, this question deserves to be thoroughly addressed to improve the design of image processing techniques. This paper raises and investigates the possible relationship between image distortions and its impact on the emotion context of S3D. We implemented an experimental design in which participants watched a series of 3D contents with different levels of visual distortions supposed to induce visual discomfort, while self-reported and psychophysiological measures of emotions were recorded. Results showed that physiological correlates of emotions were affected by visual discomfort conditions, and that these measures were more sensitive than traditional self-reported measures.
An edge-based structural distortion indicator for the quality assessment of 3D synthesized views

3D-TV applications require the generation of novel viewpoints through Depth-Image-Based-Rendering methods. These synthesized views need to be assessed by a reliable quality metric. Most of the proposed metrics are inspired from 2D commonly used quality metrics. Yet, the latter were originally designed to address 2D compression distortions which are different from the distortions related to DIBR processes. We propose an edge-based method that indicates the level of structural degradation in the synthesized image. The first results are encouraging since the correlation to subjective scores is higher than other tested metrics.
Cross-lab study on preference of experience in 3DTV: Influence from display technology and test environment

Quality of Experience (QoE) in 3DTV may be influenced by many factors, such as the viewing conditions, the characteristics of the panel of viewers, etc. In this study, the impact of two factors, namely the test environment and the display technology are analyzed. Particularly, two paired comparison experiments have been carried out to assess the overall preference of experience (PoE) for a set of 3D video sequences. The first experiment, performed at University of Nantes, allows comparing the results obtained in a standard laboratory versus living-room-like environment. No significance difference has been found. The second experiment, performed at Brno University, allows comparing the results obtained using stereoscopic display with shutter glasses versus polarized glasses. Significance difference could be identified depending on the video content. Since the same test material has been used in both locations, inter-laboratory correlation is also analyzed.
Adaptive contrast adjustment for postprocessing of tone mapped high dynamic range images

Tone mapping operators (TMOs) employed to visualize high dynamic range (HDR) content on conventional low dynamic range (LDR) devices suffer from two major drawbacks. First, none of them can faithfully reproduce all the contrast present in HDR images. Second, most of them require one or more parameters which are mostly content specific and their optimal values can be set only via subjective testing. To address these issues, this paper proposes that `quality driven' adaptive contrast enhancement is a practical solution. This is achieved by enhancing the contrast adaptively based on the loss of contrast between the HDR and tone mapped image. Experimental results confirm that the proposed adaptive solution always improves upon the contrast achieved from whatever given TMO parameter settings in the tested images. So it helps to achieve the results of a more optimal TMO parameter setting without the human input.
UHD image reconstruction by estimating interpolation error

With the emerging ultra high definition (UHD) TV technology, the reconstruction of UHD images from transmitted HD images on the receiver's side is of great interests to save bandwidth and be compatible to existing HDTV systems. In this paper, a UHD image reconstruction algorithm is proposed for the UHDTV broadcasting system. A transmitted HD image at the receiver side is first up sampled to UHD resolution, then the error map between the interpolated UHD image and the original UHD image is estimated. The final reconstructed UHD image is the sum of the interpolated UHD image and the estimated UHD error map. There are two steps to estimate the content of the UHD error map: 1) key point detection and prediction estimate the location of the pixels with large error and create a spline tube between two adjacent key points; 2) spline-tube interpolation estimate the error value along the spline tube. Our simulation results on six images show that the proposed reconstruction method performs better than conventional up sampling without estimating the error map.
Region-of-Interest intra prediction for H.264/AVC error resilience

Packets in a video bitstream contain data with different levels of importance that yield unequal amounts of quality distortion when lost. In order to avoid sharp quality degradation due to packet loss, we propose in this paper an error resilience method that is applied to the region of interest (RoI) of the picture. This method protects the RoI while not yielding significant overhead. We perform an eye tracking test to determine the RoIs of a video sequence and we assess the performance of the proposed model in error-prone environments by means of a subjective quality test. Loss simulation results show that stopping the temporal error propagation in the RoIs of the pictures helps preserving an acceptable visual quality in the presence of packet loss.
Analysis and improvement of a paired comparison method in the application of 3DTV subjective experiment

Paired comparison is a frequently used method in psychophysical studies. However, with the increase of the number of the stimuli, the number of comparisons increases exponentially. Square design is one of the balanced sub-set paired comparison methods which could reduce the number of comparisons while producing comparably precise results under some assumptions. However, when there are observation errors from observers' attentiveness, the square design would produce large estimation errors. Thus, an improved square design which is robust to observation errors is proposed. Using a Monte Carlo simulation, the proposed method is evaluated and shows improvement in efficiency. The original design is applied in a visual discomfort subjective test of 3DTV. In addition, both of the two designs are studied by utilizing our previous full comparison data. The test results showed that the proposed improved square design is more robust to observation errors. Another important finding is that the influence of the occurrence of some other stimuli on voting is significant. Whether the proposed method could reduce the prediction errors induced by it is still under study.
Improved coefficient coding for adaptive transforms in HEVC

Adaptive transform learning schemes have been extensively studied in the literature with a goal to achieve better compression efficiency compared to extensively used Discrete Cosine Transforms (DCT) inside a video codec. These transforms are learned offline on a large training set and are tested either in competition with or in place of the core transforms i.e. DCT. In our previous work, we proposed an alternative approach where a set of non-separable content-adaptive transforms are learned on-the-fly on a sequence and then tested on the same sequence in competition with the core transforms. In this paper, we propose to further improve the previously proposed learning scheme by improving the coding of transformed coefficients in context to the online learning of transforms. The first proposed method improves the convergence of the learning scheme by re-ordering the transformed coefficients at each iteration. The second proposed method improves the compression efficiency by modifying the coding of last significant coefficient position in context to adaptive transforms. The results shows that by combining the above proposed methods, one can achieve 1.2% gain on top of the previously proposed scheme.
Visual attention as a dimension of QoE: Subtitles in UHD videos

With the ever-growing availability of multimedia content produced, broadcast and consumed worldwide, subtitling is becoming an essential service to quickly share understandable content. Simultaneously, the increased resolution of the ultra high definition (UHD) standard comes with wider screens and new viewing conditions. Services as the display of subtitles thus require adaptation to better fit the new induced viewing visual angle. This paper aims at evaluating quality of experience of subtitled movies in UHD to propose guidelines for the appearance of subtitles. From an eye-tracking experiment conducted on 68 observers and 30 video sequences, viewing behavior and visual saliency are analyzed with and without subtitles and for different subtitle styles. Various metrics based on eye-tracking data, such as the Reading Index for Dynamic Texts (RIDT), are computed to objectively measure the ease of reading and subtitle disturbance. The results mainly show that doubling the visual angle of subtitles from HD to UHD guarantees subtitle readability without compromising the enjoyment of the video content.
Modern displays: Why we see different colors, and what it means?

One of the basic tenets of conventional applied colorimetry is that the whole population of color normal observers can be represented by a single “standard” observer with reasonable accuracy. The 1964 CIE standard colorimetric observer has indeed served us well in all industrial color imaging applications, until recently. With the proliferation of modern wide-gamut displays with narrow-band primaries, color scientists and engineers face a new challenge. Various recent studies, including those by the current authors, have shown that the color perception on such displays varies significantly among color normal observers. Conventional colorimetry has no means to predict this variation. In this paper, we explore this problem by summarizing the results from an ongoing study, and explain the practical significance of this issue in the context of display applications.
Image quality assessment for free viewpoint video based on mid-level contours feature

Free view point video (FVV), which offers immersive experience to users with multiple views, is one of the new trends in advanced visual media. These new viewpoints are traditionally synthesized via depth image-based rendering(DIBR) and geometric distortions are therefore observed. Mid-level contours descriptors are capable of evaluating such edges incoherence among the synthesized images which common image quality metrics fail to capture. In this paper, we use the concept of ‘Sketch Token’, that is a mid-level contours descriptor, and introduce a novel metric for DIBR-synthesized image quality assessment by measuring how classes of contours change after synthesis. Experiments are conducted on the IRCCyN/IVC DIBR image database and the results show that the proposed metric achieves a correlation of 88.77% which is comparable to state-of-the-art metrics like MW-PSNR and MP-PSNR.
Subjective quality assessment of MPEG-4 Scalable Video Coding in a mobile scenario

Scalable Video Coding provides several levels of video encapsulated in a single video stream. In a transmission scenario such as broadcasting, this structure is quite advantageous as it can be used to address heterogeneous decoding targets with variable needs and requirements. However, this adaptability comes at a slight cost in coding efficiency when compared to single-layer coding. Based on subjective experiments, this cost is evaluated in this paper by comparing the new MPEG-4 Scalable Video Coding (SVC) standard with the now-established MPEG-4 AVC standard. Two scenarios are analyzed in the context of mobile transmission applications. The first scenario uses the same bitrate for SVC and AVC, leading to a slightly lower PSNR for SVC. The second scenario uses the same PSNR for SVC and AVC, leading to a slightly lower bitrate for AVC. The results of the subjective tests illustrate several interesting aspects of the relation between the performance of the two standards. First, we observe that the offset between AVC and SVC is not severe, though statistically significant in terms of user Mean Opinion Score (MOS) in such a context. Second, while adding another layer to SVC always leads to a performance loss, the impact of this loss decreases when the number of layers increases.
Quality Assessment of Sharpened Images: Challenges, Methodology, and Objective Metrics

Most of the effort in image quality assessment (QA) has been so far dedicated to the degradation of the image. However, there are also many algorithms in the image processing chain that can enhance the quality of an input image. These include procedures for contrast enhancement, deblurring, sharpening, up-sampling, denoising, transfer function compensation, and so on. In this paper, possible strategies for the QA of sharpened images are investigated. This task is not trivial, because the sharpening techniques can increase the perceived quality, as well as introduce artifacts leading to the quality drop (over-sharpening). Here, the framework specifically adapted for the QA of sharpened images and objective metrics comparison in this context is introduced. However, the framework can be adopted in other QA areas as well. The problem of selecting the correct procedure for subjective evaluation was addressed and a subjective test on blurred, sharpened, and over-sharpened images was performed in order to demonstrate the use of the framework. The obtained ground-truth data were used for testing the suitability of the state-of-the-art objective quality metrics for the assessment of sharpened images. The comparison was performed by novel procedure using rank order correlation analyses, which is found more appropriate for the task than standard methods. Furthermore, seven possible augmentations of the no-reference S3 metric adapted for sharpened images are proposed. The performance of the metric is significantly improved and also superior over the rest of the tested quality criteria with respect to the subjective data.
Study of Video Quality Assessment for Telesurgery

Telemedicine provides a transformative practice for access to and delivery of timely and high-quality healthcare in resource-poor settings. In a typical scenario of telesurgery, surgical tasks are performed with one surgeon situated at the patient’s side and one expert surgeon from a remote site. In order to make telesurgery practice realistic and secure, reliable transmission of medical videos over large distances is essential. However, telesurgery videos that are communicated remotely in real time are vulnerable to distortions in signals due to data compression and transmission. Depending on the system and its applications, visual content received by the surgeons differs in perceived quality, which may incur implications for the performance of telesurgery tasks. To rigorously study the assessment of the quality of telesurgery videos, we performed both qualitative and quantitative research, consisting of semi-structured interviews and video quality scoring with human subjects. Statistical analyses are conducted and results show that compression artifacts and transmission errors significantly affect the perceived quality; and the effects tend to depend on the specific surgical procedure, visual content, frame rate, and the degree of distortion. The findings of the study are readily applicable to improving telesurgery systems.
Toward the Assessment of Quality of Experience for Asymmetric Encoding in Immersive Media

The assessment of Quality of Experience (QoE) for stereoscopic 3-D video is a challenging task, especially in 3-D video compression and transmission applications. The focus of this contribution is the development of a QoE assessment framework, in line with the latest standardization progress in the field of QoE assessment, for understanding the visual effect of asymmetric and symmetric encoding for immersive media. Asymmetric stereoscopic video coding exploits the binocular suppression of the human vision system by representing one of the two views with a lower quality. This processing, while of limited effects on image quality, may influence the overall QoE. Many studies show that the QoE of immersive media such as 3-DTV can be thought as the combination of the perceived visual quality, the perceived depth quality, the visual fatigue, and visual discomfort. In this paper, we aim at: 1) exploiting the concept of preference of experience and protocols recently standardized for characterizing QoE; 2) conducting a case study using these standardized protocols to investigate the factors involving visual discomfort in stereoscopic video sequences with a focus on binocular rivalry; and 3) presenting the results of subjective experiments performed, by using the perceptual quality and preference of experience assessment protocols, for evaluating the impact of symmetrical, asymmetrical, and alternate coding schemes.
Image quality assessment based on pyramid decomposition and mean squared error

Widely used image quality metric peak signal-to-noise ratio, PSNR, doesn't correlate very well with human judgment. In this paper, we show that using multi-scale image decomposition before calculating mean squared errors at higher pyramid's scales, we get multi-scale MSPSNR metric that is very well correlated to subjective assessment and very efficient for computation. Proposed metric is tested using LIVE3D database which contains stereo images with common distortion types.
Optimizing feature pooling and prediction models of VQA algorithms

In this paper, we propose a strategy to optimize feature pooling and prediction models of video quality assessment (VQA) algorithms with a much smaller number of parameters than methods based on machine learning, such as neural networks. Based on optimization, the proposed mapping strategy is composed of a global linear model for pooling extracted features, a simple linear model for local alignment in which local factors depend on source videos, and a non-linear model for quality calibration. Also, a reduced-reference VQA algorithm is proposed to predict the local factors from the source video. In the IRCCyN/IVC video database of content influence and the LIVE mobile video database, the performance of VQA algorithms is improved significantly by local alignment. The proposed mapping strategy with prediction of local factors outperforms one no-reference VQA metric and is comparable to one full-reference VQA metric. Thus predicting the local factors in local alignment based on video content will be a promising new approach for VQA.
Using disparity for quality assessment of stereoscopic images

3DTV has been widely studied these last years from a technical point of view but the related perceived quality evaluations do not follow this hype. This paper firstly reviews quality assessment issues for 3DTV. Compared to 2D quality measure, depth information adds several new problems considering quality assessment. Nevertheless, efforts made for 2D content quality estimation can be used for an extension to 3D. In a second part, this paper proposes an adaptation of 2D metrics to 3D in the context of coding artifacts and stereoscopic images. Distortion on disparity is introduced to improve conventional 2D metrics. Performances have been evaluated using subjective tests.
Influence of depth rendering on the quality of experience for an autostereoscopic display

Autostereoscopic displays simplify the presentation of 3D content because they do not require any glasses and they allow the perception of motion parallax. While the perception of depth is certainly an added value, the technical rendering process of the current display technology also introduces artifacts. For the viewer, the tradeoff may be expressed in terms of quality of experience. However, quality of experience assessment related to 3D is still an open issue. Towards this goal, several original subjective test methods are proposed and compared that are meant for assessing the quality of experience. A split-screen setup simultaneously displays a 2D and a 3D presentation. The observers vote according to their preference in terms of quality of experience. In four experiments, the influence of the depth rendering process is evaluated. The results indicate that the degradation by the depth rendering process may easily dominate the added value of depth in a content specific manner.
Reliability of 2D quality assessment methods for synthesized views evaluation in stereoscopic viewing conditions

This paper investigates the reliability of objective quality metrics commonly used for the quality assessment of 2D media, in the context of 3D Video. In the absence of any dedicated tool for the evaluation of synthesized views quality, we often rely on available 2D metrics for direct evaluation of 3D media quality, or with some adaptation to the 3D case. However, recent studies showed that the use of DIBR, depending on its in-painting strategy, can lead to downsides whose range in terms of quality, has never been experienced with 2D media. This paper questions the reliability of the objective quality metrics normally used for the quality assessment, when assessing stereopairs. Seven DIBR algorithms are used to generate novel viewpoints. A series of commonly used quality metrics then assess their quality. The results of our experiments showed that the metrics are not sufficient to faithfully predict human judgment. Moreover, we compared our results with an experiment run in monoscopic viewing condition and the differences are for the less unexpected since the preferred artifacts in monoscopic condition are the most rejected in stereoscopic condition. This paper proposes some explanations.
Standardized toolchain and model development for video quality assessment — The mission of the Joint Effort Group in VQEG

Since 1997, the Video Quality Experts Group (VQEG) has been active in the field of subjective and objective video quality assessment. The group has validated competitive quality metrics throughout several projects. Each of these projects requires mandatory actions such as creating a testplan and obtaining databases consisting of degraded video sequences with corresponding subjective quality ratings. Recently, VQEG started a new open initiative, the Joint Effort Group (JEG), for encouraging joint collaboration on all mandatory actions needed to validate video quality metrics. Within the JEG, effort is made to advance the field of both subjective and objective video quality measurement by providing proper software tools and subjective databases to the community. One of the subprojects of the JEG is the joint development of a hybrid H.264/AVC objective quality metric. In this paper, we introduce the JEG and provide an overview of the different ongoing activities within this newly started group.
Artistic Stereo Imaging by Edge Preserving Smoothing

Stereo imaging is an important area of image and video processing, with exploding progress in the last decades. An open issue in this field is the understanding of the conditions under which the straightforward application of a given image processing operator to both the left and right image of a stereo pair preserves the stereoscopic perception. In this paper, we explore this problem with application to artistic imaging and we prove that, unlike other methods, artistic operators based on edge preserving smoothing have this desirable property. We also present a novel multiresolution artistic operator, purposely designed for stereo images, which enhances the perception of three-dimensionality by means of a depth driven local scale control.