nils-herrmann/clean_scirepeval huggingface dataset:
Iterative CT Reconstruction Using Shearlet-Based Regularization

Total variation (TV) methods have been proposed to improve the image quality in count-reduced images, by reducing the variation between neighboring pixels. Although very easy to implement and fast to compute, TV-based methods may lead to a loss of texture information when applied to images with complex textures, such as high-resolution abdominal CT images. Here, we investigate the use of another regularization approach in the context of medical images based on multiresolution transformations. One such transformation is the shearlet transform, which is optimally sparse for images that are C2 except for discontinuities along C2 curves, and has better directional sensitivity than most other, related, wavelet transform approaches. We propose to solve the convex problem using the split-Bregman (augmented Lagrangian) approach. One of the primary advantages of the split-Bregman approach, is that the shearlet transform can easily be incorporated into the sparse-view CT reconstruction. The required sparsity prior is the l1 norm of the shearlet coefficients. Results are shown for this method in comparison to the same framework with TV as the regularization term on simulated data. The noise-resolution performance is investigated at different contrast levels. At equal image noise, TV-based regularization outperforms shearlet-based regularization. However, when image texture is analyzed on measured mouse data, shearlets outperform TV, which suffers from staircasing effects. Our results show that there are benefits in using shearlets in CT imaging: texture is reconstructed more accurately compared to when TV is used, without biasing the image towards a piecewise constant image model. However, due to the larger support of the basis functions, our results suggest that uncareful usage of shearlets may lead to wavy artifacts, which can be equally unwanted as staircasing effects.
Compressed sensing in MRI with a Markov random field prior for spatial clustering of subband coefficients

Recent work in compressed sensing of magnetic resonance images (CS-MRI) concentrates on encoding structured sparsity in acquisition or in the reconstruction stages. Subband coefficients of typical images obey a certain structure, which can be viewed in terms of fixed groups (like wavelet trees) or statistically (certain configurations are more likely than others). Approaches using wavelet tree-sparsity have already demonstrated excellent performance in MRI. However, the use of statistical models for spatial clustering of the subband coefficients has not been studied well in CS-MRI yet, although the potentials of such an approach have been indicated. In this paper, we design a practical reconstruction algorithm as a variant of the proximal splitting methods, making use of a Markov Random Field prior model for spatial clustering of subband coefficients. The results for different undersampling patterns demonstrate an improved reconstruction performance compared to both standard CS-MRI methods and methods based on wavelet tree sparsity.
Spatiogram features to characterize pearls in paintings

Objective characterization of jewels in paintings, especially pearls, has been a long lasting challenge for art historians. The way an artist painted pearls reflects his ability to observing nature and his knowledge of contemporary optical theory. Moreover, the painterly execution may also be considered as an individual characteristic useful in distinguishing hands. In this work, we propose a set of image analysis techniques to analyze and measure spatial characteristics of the digital images of pearls, all relying on the so called spatiogram image representation. Our experimental results demonstrate good correlation between the new metrics and the visually observed image features, and also capture the degree of realism of the visual appearance in the painting. In that sense, these results set the basis in creating a practical tool for art historical attribution and give strong motivation for further investigations in this direction.
Perceptual depth indicator for S-3D content based on binocular and monocular cues

This article describes a general depth indicator for stereoscopic 3D video sequences. This indicator targets the characterization of the depth in 3D video sequences based on both monocular and binocular depth cues. Evaluating monocular depth cues is no easy task. Due to this high complexity, only a subset of the monocular depth cues are currently considered since it is believed that they play a major impact in depth perception. The proposed algorithm will consider the following different depth cues: binocular depth, linear perspective, blur from defocus, motion parallax and texture gradient. It will be detailed how all these all metrics have been designed. The second main contribution is the definition of a specific application scope of each metrics. This is motivated by the need to take into account the reliability of each individual metrics during the pooling. Therefore, it will be considered in the paper to identify the cases where each individual metric may fail and integrate these aspects in the general combination of the depth cues.
Comparing upscaling algorithms from HD to Ultra HD by evaluating preference of experience

As the next generation of TV, Ultra High Definition Television (UHDTV) is attracting more and more people's attention as it provides a new viewing experience. Considering content delivery, due to the lack of Ultra HD resources, a direct question for the industry is that whether the state-of-the-art upscaling algorithms can be utilized to upscale the current HD or Full HD resources to UHD, gaining benefit from the higher resolution but without losing the high quality viewing experience. To investigate this, in this study, we upscaled 720p and 1080p sequences to UHD resolution by different upscaling algorithms. Paired Comparison methodology was used in the subjective experiment to evaluate their performances. The results showed that for the case of fast motion content, viewers' preference on different upscaled video sequences is not significantly different. In general conditions, the low complexity upscaling algorithms (e.g., lanczos-3) performed better than the high complexity algorithms (e.g., Robust Super Resolution algorithm). A novel upscaling algorithm adapted to UHD is recommended to be developed based on the mechanisms of human visual system.
Quality assessment for synthesized view based on variable-length context tree

In free viewpoint television (FTV) application scenario, views that synthesized with depth image-based rendering (DIBR) techniques mainly contain special artifacts like geometric distortions. These artifacts may affect the structure of images/videos by changing the global contour characteristics and thus are annoying for human observers. Context tree based contour coding scheme can be a good tool to measure such structure loss since the more geometric distortion there is in the synthesized view the lager the gap between the encoding cost of the reference and synthesized views. In this paper, we investigate whether such overall encoding cost can be related to perceptual annoyance reflecting in quality score and propose a variable-length context tree based image quality assessment (CT-IQA) scheme. This scheme quantify (1) the overall structure dissimilarity and (2) dissimilarities in various contour characteristics between the reference and synthesized views. The proposed metric is robust to global shifting artifact that is over penalized by traditional metrics. According to the experimental results on the IRCCyn/IVC DIBR image database, the performance of the proposed CT-IQA is promising.
A Subjective Evaluation of 3D Iptv Broadcasting Implementations Considering Coding and Transmission Degradation

This paper describes the results of a subjective test to assess current technology used for 3DTV broadcasting. As a first aspect, the performance of the currently deployed coding schemes was compared to state of the art algorithms. Our results show that down sampling and packing 3D stereoscopic videos according to the so called Side-By-Side format gives the highest perceived quality for a given bit rate. The second aspect of the study was to investigate how common 2D error concealment algorithms perform in case of 3D, and how their 3D-related performance compares with the 2D case. The results provide information on whether binocular suppression or binocular rivalries play the most important role for 3D video quality under transmission error. The results indicate that binocular rivalries and related visual discomfort are the dominant factors. Another aspect of the paper is a comparison of the test results with results from different labs to evaluate the repeatability of a subjective experiment in the 3D case, and to compare the employed test methodologies. Here, the study shows the variation between observers when they are rating visual discomfort and illustrates the difficulty to evaluate this new dimension.
A subjective and objective evaluation of a realistic 3D IPTV transmission chain

In 3D transmissions, often a large perceptual quality gain can be achieved by slightly increasing the bitrate. However, at a certain bitrate, a saturation effect is noted and further increasing the bitrate does not lead to significant improvements of Quality of Experience (QoE). This bitrate will be called quality saturation bitrate. The purpose of this paper is to investigate a subjective and objective method to determine the quality saturation bitrate. An evaluation is presented which uses a wide spread of content types and a realistic transmission chain that includes a hardware encoder and commercial Set-Top-Boxes. A subjective assessment for various bitrates is performed using the SAMVIQ methodology and the results are also compared to objective measurements with VQM and VQUAD.
Computational Model of Stereoscopic 3D Visual Saliency

Many computational models of visual attention performing well in predicting salient areas of 2D images have been proposed in the literature. The emerging applications of stereoscopic 3D display bring an additional depth of information affecting the human viewing behavior, and require extensions of the efforts made in 2D visual modeling. In this paper, we propose a new computational model of visual attention for stereoscopic 3D still images. Apart from detecting salient areas based on 2D visual features, the proposed model takes depth as an additional visual dimension. The measure of depth saliency is derived from the eye movement data obtained from an eye-tracking experiment using synthetic stimuli. Two different ways of integrating depth information in the modeling of 3D visual attention are then proposed and examined. For the performance evaluation of 3D visual attention models, we have created an eye-tracking database, which contains stereoscopic images of natural content and is publicly available, along with this paper. The proposed model gives a good performance, compared to that of state-of-the-art 2D models on 2D images. The results also suggest that a better performance is obtained when depth information is taken into account through the creation of a depth saliency map, rather than when it is integrated by a weighting method.
Subjective assessment methodology for preference of experience in 3DTV

The measurement of the Quality of Experience (QoE) in 3DTV recently became an important research topic as it relates to the development of the 3D industry. Pair comparison is a reliable method as it is easier for the observers to provide their preference on a pair rather than give an absolute scale value to a stimulus. The QoE measured by pair comparison is thus called “Preference of Experience (PoE)”. In this paper, we introduce some efficient designs for pair comparison which can reduce the number of comparisons. The constraints of the presentation order of the stimuli in pair comparison test are listed. Finally, some analysis methods for pair comparison data are provided accompanied with some examples from the studies of the measurement of PoE.
Virtual interactions: Can EEG Help Make the Difference with Real Interaction?

Science and technology progress fast, but mouse and keyboard are still used to control multimedia devices. One of the limiting factors of gesture based HCIs adoption is the detection of the user's intention to interact. This study tries to make a step in that direction with use of consumer EEG sensor headset. EEG headset records in real-time data that can help to identify intention of the user based on his emotional state. For each subject EEG responses for different stimuli are recorded. Acquiring these data allows to determine the potential of EEG based intention detection. The findings are promising and with proper implementation should allow to building a new type of HCI devices.
Visual Attention and Applications in Multimedia Technologies

Making technological advances in the field of human-machine interactions requires that the capabilities and limitations of the human perceptual system be taken into account. The focus of this report is an important mechanism of perception, visually selective attention, which is becoming more and more important for multimedia applications. We introduce the concept of visual attention and describe its underlying mechanisms. In particular, we introduce the concepts of overt and covert visual attention, and of bottom-up and top-down processing. Challenges related to modeling visual attention and their validation using ad hoc ground truth are also discussed. Examples of the usage of visual attention models in image and video processing are presented. We emphasize multimedia delivery, retargeting and quality assessment of image and video, medical imaging, and the field of stereoscopic 3-D image applications.
Effect of content features on short-term video quality in the visual periphery

The area outside our central field of vision, also referred to as the visual periphery, captures most information in a visual scene, although much less sensitive than the central Fovea. Vision studies in the past have stated that there is reduced sensitivity of texture, color, motion and flicker (temporal harmonic) perception in this area, that bears an interesting application in the domain of quality perception. In this work, we particularly analyze the perceived subjective quality of videos containing H.264/AVC transmission impairments, incident at various degrees of retinal eccentricities of observers. We relate the perceived drop in quality, to five basic types of features that are important from a perceptive standpoint: texture, color, flicker, motion trajectory distortions and also the semantic importance of the underlying regions. We are able to observe that the perceived drop in quality across the visual periphery, is closely related to the Cortical Magnification fall-off characteristics of the V1 cortical region. Additionally, we see that while object importance and low frequency spatial distortions are important indicators of quality in the central foveal region, temporal flicker and color distortions are the most important determinants of quality in the periphery. We therefore conclude that, although users are more forgiving of distortions they viewed peripherally, they are nevertheless not totally blind towards it: the effects of flicker and color distortions being particularly important.
Modeling the perceptual distortion of dynamic textures and its application in HEVC

In the quest of perceptually optimized video coding, coding textures is representing a challenging case. While a large body of research was put into the perception of static textures, dynamic textures are still not sufficiently explored. In this paper, we focus on short term consistent patches, known as dynamic textures, with a very limited spatial and temporal extent. We estimated the visual distortion due to HEVC compression via subjective testing. The estimated distortion profile per texture was used to optimize the HEVC coding process. Experimental results showed that this technique can offer a high bitrate saving for the same subjective quality.
The impact of complexity in the rate-distortion optimization: A visualization tool

Recently, the new video coding standard, High Efficiency Video Coding (HEVC), was released. A 50% bitrate reduction at the same visual quality relative to the previous standard was achieved. These gains are partly related to tools that increase complexity such as coding decisions taken on smaller image areas or further improvements in motion compensated coding. This paper addresses the complexity issue, i.e. the execution time, and its impact when it is combined with the bitrate and distortion optimization of the video coding. Given a set of encoder configurations that restrict the encoding such that the complexity varies, an analysis including a visualization tool is proposed to help the user to select the best configuration for a specific amount of rate, distortion and complexity. A possible targeted applications are also introduced.
Real Time Dynamic Image Re-targeting Based on a Dynamic Visual Attention Model

With the constantly growing number of multimedia devices, images can now be viewed on a broad range of display of tremendously varying size. However, perceiving all the details of a multimedia content is difficult on smaller mobile devices. To solve this problem, saliency based image re-targeting algorithms have been proposed in order to generate more perceptually efficient thumbnails. In this article, we propose a new real-time visual attention based image re-targeting method which differs from previous works in that it generates dynamic (i.e. time changing) thumbnails. This unique behavior is made possible by the use of a dynamic computational model of visual attention. By exploiting the singular characteristics of this model, we are able to dynamically re-target large and complex images in order to efficiently script their discovery on large and small displays. We achieved such re-targeting by integrating simulated attentional focuses over a fixed time window into a temporal heat-map. By segmenting this heat-map at each time step, we can generate the region of interest that will be displayed as dynamic preview. In order to validate that every salient location of the image has been displayed, we compare the regions shown in the preview with mean observers eye fixations obtained during a free viewing experiment.
A new H.264/AVC error resilience model based on Regions of Interest

Video transmission over the Internet can sometimes be subject to packet loss which reduces the end-user's Quality of Experience (QoE). Solutions aiming at improving the robustness of a video bitstream can be used to subdue this problem. In this paper, we propose a new Region of Interest-based error resilience model to protect the most important part of the picture from distortions. We conduct eye tracking tests in order to collect the Region of Interest (RoI) data. Then, we apply in the encoder an intra-prediction restriction algorithm to the macroblocks belonging to the RoI. Results show that while no significant overhead is noted, the perceived quality of the video's RoI, measured by means of a perceptual video quality metric, increases in the presence of packet loss compared to the traditional encoding approach.
Saliency detection for stereoscopic images

Saliency detection techniques have been widely used in various 2D multimedia processing applications. Currently, the emerging applications of stereoscopic display require new saliency detection models for stereoscopic images. Different from saliency detection for 2D images, depth features have to be taken into account in saliency detection for stereoscopic images. In this paper, we propose a new stereoscopic saliency detection framework based on the feature contrast of color, intensity, texture, and depth. Four types of features including color, luminance, texture, and depth are extracted from DC-T coefficients to represent the energy for image patches. A Gaussian model of the spatial distance between image patches is adopted for the consideration of local and global contrast calculation. A new fusion method is designed to combine the feature maps for computing the final saliency map for stereoscopic images. Experimental results on a recent eye tracking database show the superior performance of the proposed method over other existing ones in saliency estimation for 3D images.
A foveated short term distortion model for perceptually optimized dynamic textures compression in HEVC

Due to their rapid change over time, dynamic textures represent challenging contents for the video compression standards. While several studies proposed different coding techniques, such as synthesis based coding, they mostly lack the perceptual studies on the perceived distortions. In this paper, a framework of perceptually optimized dynamic texture compression is presented. It is based on a psychophysically driven distortion model, which is utilized inside the rate distortion loop of HEVC. The model is tested for three compression levels, and showed a significant rate saving for both training and validation sequences set.
Local texture synthesis: A static texture coding algorithm fully compatible with HEVC

Textures are one of the main characteristics of the visual scene. Perceptually, their details are less important than their semantic meaning. This property has been exploited in many texture coding (content based video coding) approaches by removing parts of the textures in the encoder and synthesizing them at the decoder side. Such an approach would necessarily need modification of the coding process and violating the standard. This paper introduces a novel algorithm for texture coding called Local Texture Synthesis (LTS), in which texture synthesis is employed in a full compatibility with HEVC standard. This implies that a basic HEVC decoder can be used to reconstruct the signal. LTS defines the necessary conditions to synthesize a patch and produces different synthesis of it. It tries then coding each of them, and finally chooses the one that minimizes the coding cost. A prototype of this algorithm, based on Markov Random Fields, is given in this paper. This prototype provides up to 10% bitrate saving (using the same quantization parameter) while maintaining an equivalent visual quality.
Data Analysis in Multimedia Quality Assessment: Revisiting the Statistical Tests

Assessment of multimedia quality relies heavily on subjective assessment, and is typically done by human subjects in the form of preferences or continuous ratings. Such data are crucial for analysis of different multimedia-processing algorithms as well as validation of objective (computational) methods for the said purpose. To that end, statistical testing provides a theoretical framework toward drawing meaningful inferences, and making well-grounded conclusions and recommendations. While parametric tests (such as $t$  test, ANOVA, and error estimates like confidence intervals) are popular and widely used in the community, there appears to be a certain degree of confusion in the application of such tests. Specifically, the assumptions of normality and homogeneity of variance are often not well understood, leading to incorrect application and/or interpretation of the statistical test results. Therefore, the main goal of this paper is to present new guidelines toward proper use of statistical tests and, hence, fix some of the issues in multimedia quality assessment. The said guidelines are derived based on theoretical analysis of sampling distribution of test statistics, and consider practical aspects of data analysis in the said domain. Experimental results on both simulated and real data are presented to support the arguments made. Software that implements the said recommendations is also made publicly available, in order to help researchers and practitioners perform correct statistical comparison of models.