nils-herrmann/clean_scirepeval huggingface dataset:
A Perceptually Inspired Variational Framework for Color Enhancement

Basic phenomenology of human color vision has been widely taken as an inspiration to devise explicit color correction algorithms. The behavior of these models in terms of significative image features (such as, e.g., contrast and dispersion) can be difficult to characterize. To cope with this, we propose to use a variational formulation of color contrast enhancement that is inspired by the basic phenomenology of color perception. In particular, we devise a set of basic requirements to be fulfilled by an energy to be considered as 'perceptually inspired', showing that there is an explicit class of functionals satisfying all of them. We single out three explicit functionals that we consider of basic interest, showing similarities and differences with existing models. The minima of such functionals is computed using a gradient descent approach. We also present a general methodology to reduce the computational cost of the algorithms under analysis from O(N2) to O(N logN), being N the number of pixels of the input image.
Angular-Based Preprocessing for Image Denoising

There is not a large research on how to use color information for improving results in image denoising. Currently, most of the methods modify the color space from standard red green blue (sRGB) to an opponent-like one as better results are obtained, but out of this conversion, color is mostly ignored in the image denoising pipelines. In this letter, we propose a color decomposition to preprocess an image before applying a typical denoising. Our decomposition consists in obtaining a set of images in the spherical coordinate system, each of them with the origin of the spherical transformation in a different color value. These color values, that we call color centers, are defined so as to be far away from the dominant colors of the image. Once in the spherical coordinate system, we perform a mild denoising operation with some state-of-the-art method in the angular components. Then, we convert these images back to sRGB, and we merge them depending on the distance between the color of each pixel and the color centers. Finally, we denoise the preprocessed image with the same state-of-the-art method used in our preprocessing. Experiments show that our method outperforms the results of directly applying the denoising method on the input image for different state-of-the-art denoising methods.
Video inpainting of occluding and occluded objects

We present a basic technique to fill-in missing parts of a video sequence taken from a static camera. Two important cases are considered. The first case is concerned with the removal of non-stationary objects that occlude stationary background. We use a priority based spatio-temporal synthesis scheme for inpainting the stationary background. The second and more difficult case involves filling-in moving objects when they are partially occluded. For this, we propose a priority scheme to first inpaint the occluded moving objects and then fill-in the remaining area with stationary background using the method proposed for the first case. We use as input an optical-flow based mask, which tells if an undamaged pixel is moving or is stationary. The moving object is inpainted by copying patches from undamaged frames, and this copying is independent of the background of the moving object in either frame. This work has applications in a variety of different areas, including video special effects and restoration and enhancement of damaged videos. The examples shown in the paper illustrate these ideas.
Perceptual Color Correction Through Variational Techniques

In this paper, we present a discussion about perceptual-based color correction of digital images in the framework of variational techniques. We propose a novel image functional whose minimization produces a perceptually inspired color enhanced version of the original. The variational formulation permits a more flexible local control of contrast adjustment and attachment to data. We show that a numerical implementation of the gradient descent technique applied to this energy functional coincides with the equation of automatic color enhancement (ACE), a particular perceptual-based model of color enhancement. Moreover, we prove that a numerical approximation of the Euler-Lagrange equation reduces the computational complexity of ACE from O(N2) to O(NlogN), where N is the total number of pixels in the image
Gamut Mapping in Cinematography Through Perceptually-Based Contrast Modification

Gamut mapping transforms the colors of an input image to the colors of a target device so as to exploit the full potential of the rendering device in terms of color rendition. In this paper we present spatial gamut mapping algorithms that rely on a perceptually-based variational framework. Our algorithms adapt a well-known image energy functional whose minimization leads to image enhancement and contrast modification. We show how by varying the importance of the contrast term in the image functional we are able to perform gamut reduction and gamut extension. We propose an iterative scheme that allows our algorithms to successfully map the colors from the gamut of the original image to a given destination gamut while keeping the perceived colors close to the original image. Both subjective and objective evaluations validate the promising results achieved via our proposed algorithms.
Local denoising based on curvature smoothing can visually outperform non-local methods on photographs with actual noise

We propose a fast, local denoising method where the Euclidean curvature of the noisy image is approximated in a regularizing manner and a clean image is reconstructed from this smoothed curvature. User preference tests show that when denoising real photographs with actual noise our method produces results with the same visual quality as the more sophisticated, nonlocal algorithms Non-local Means and BM3D, but at a fraction of their computational cost. These tests also highlight the limitations of objective image quality metrics like PSNR and SSIM, which correlate poorly with user preference.
Gamut Extension for Cinema

Emerging display technologies are able to produce images with a much wider color gamut than those of conventional distribution gamuts for cinema and TV, creating an opportunity for the development of gamut extension algorithms (GEAs) that exploit the full color potential of these new systems. In this paper, we present a novel GEA, implemented as a PDE-based optimization procedure related to visual perception models, that performs gamut extension (GE) by taking into account the analysis of distortions in hue, chroma, and saturation. User studies performed using a digital cinema projector under cinematic (low ambient light, large screen) conditions show that the proposed algorithm outperforms the state of the art, producing gamut extended images that are perceptually more faithful to the wide-gamut ground truth, as well as free of color artifacts and hue shifts. We also show how currently available image quality metrics, when applied to the GE problem, provide results that do not correlate with users’ choices.
Automatic, Viewing-Condition Dependent Contrast Grading based on Perceptual Models

Cameras automatically apply non-linear transformations to the sensor data, allowing for perceptually-uniform quantization suited to standard dynamic range displays in dim conditions. In the cinema industry, data is recorded in raw (linear) format and non-linearly corrected in post-production by a skilled technician who optimizes image appearance for cinema (dark) conditions. We propose a method that automatically performs this non-linear transformation taking into account the intended viewing conditions. It is based on visual perception models and produces results that look natural, without any spatio-temporal artifacts. User preference tests show that our method outperforms state of the art approaches. The technique is fast and could be implemented on camera hardware. It can be used for on-set monitoring on regular displays, as a substitute for gamma-correction, and as a way of providing the colorist with content that is both natural looking and has a crisp, clear image.
Filling-in by joint interpolation of vector fields and gray levels

A variational approach for filling-in regions of missing data in digital images is introduced. The approach is based on joint interpolation of the image gray levels and gradient/isophotes directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data. This interpolation is computed by solving the variational problem via its gradient descent flow, which leads to a set of coupled second order partial differential equations, one for the gray-levels and one for the gradient orientations. The process underlying this approach can be considered as an interpretation of the Gestaltist's principle of good continuation. No limitations are imposed on the topology of the holes, and all regions of missing data can be simultaneously processed, even if they are surrounded by completely different structures. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity. Examples of these applications are given. We conclude the paper with a number of theoretical results on the proposed variational approach and its corresponding gradient descent flow.
Efficient vector quantisation of LPC parameters for noisy channels

Line spectral pairs (LSP) are, because of their advantageous properties, commonly used to represent the spectral information in a linear predictive (LP) speech coder. However, the quantised spectrum is relatively sensitive to channel errors. This paper describes a robust quantiser for LSPs transmitted over noisy waveform channels, based on an optimisation of the source coder over the modulation scheme. Modulation organized vector quantisation, MOR-VQ, jointly optimises a codebook and its index assignment, governed by the defined modulation symbol constellation. The system closely follows the optimum performance theoretically available (OPTA). Moreover, the MOR-VQ scheme allows for efficient codebook search strategies.<<ETX>>
Objective metrics for quality of experience in stereoscopic images

Stereoscopic Quality of Experience (QoE) is the result of a complex combination of different influencing factors. Previously we had investigated the effect of factors such as scene content, camera baseline, screen size and viewing position on stereoscopic QoE using subjective tests. In this paper, we propose two objective metrics for predicting stereoscopic QoE using bottom-up and top-down approaches, respectively. Specifically, the bottom-up metric is based on characterizing the significant factors of QoE directly, which are scene content, camera baseline, screen size and crosstalk level. While the top-down metric interprets QoE from its perceptual attributes, including crosstalk perception and perceived depth. These perceptual attributes are modeled by their individual relationship with the significant factors and then combined linearly to build the top-down metric. Both proposed metrics have been validated against our own database and a publicly available database, showing a high correlation (over 86%) with the subjective scores of stereoscopic QoE.
Spatial noise shaping using convex optimization for perceptual image coding

In this paper we propose a new convex optimization framework for precise spatial noise shaping. The effectiveness of this new technique is demonstrated in the application of perceptual coding of images. A modified JPEG 2000 codec is implemented using the proposed new framework and compared with existing perceptual coding algorithms. Results of subjective tests show that the new framework can provide a significant improvement in bitrate savings compared to the best performing wavelet domain technique. The algorithm allows much more precise control of distortion than existing spatial domain techniques and is fully compliant with part 1 of the JPEG 2000 standard.
Improving image quality assessment with modeling visual attention

Visual attention is an important attribute of the human visual system (HVS), while it has not been explored in image quality assessment adequately. This paper investigates the capabilities of visual attention models for image quality assessment in different scenarios: two-dimensional images, stereoscopic images, and Digital Cinema setup. Three bottom-up attention models are employed to detect attention regions and find fixation points from an image and compute respective attention maps. Different approaches for integrating the visual attention models into several image quality metrics are evaluated with respect to three different image quality data sets. Experimental results demonstrate that visual attention is a positive factor that can not be ignored in improving the performance of image quality metrics in perceptual quality assessment.
Towards Analysis of Intra-flow Contention in Multi-hop Wireless Networks

Based on whether the competing active nodes are forwarding packets of the same flow, the channel access contention in multi-hop wireless networks can be classified into intra-flow contention and inter-flow contention. In this paper, we analyze the effect of intra-flow contention on multi-hop network performance. First, we demonstrate that intra-flow contention makes up a large proportion of contention in the network and may severely degrade the network performance. Then, to understand the performance improvement by eliminating intra-flow contention, we consider an ideal MAC based on 802.11 DCF where intra-flow contention is eliminated. We analyze the performance of the ideal MAC by proposing a three-dimensional Markov model. Furthermore, we present an approximate implementation of the ideal MAC based on simple modifications to 802.11 DCF. Finally, extensive simulations are performed and the results validate the proposed model and reveal the performance improvement by eliminating intra-flow contention.
Exploring alternative content in Digital Cinema

Digital Cinema (D-Cinema) business is much more than feature films. Experimentation by showing programming other than feature films has been carried out since the early days of D-Cinema. Alternative content in Digital Cinema enables the cinema to become a multi-arts venue, attracting new and existing users by offering a range of vary products. We give an overview on this alternative content. Human factors issue—Quality of Experience (QoE), which is closely associated with the adoption of alternative content in D-Cinema, is also discussed.
Subjective Crosstalk Assessment Methodology for Auto-stereoscopic Displays

Cross talk is one of the most annoying distortions in the visualization stage of stereoscopic systems. Specifically, both pattern and amount of cross talk in multi-view auto-stereoscopic displays are more complex because of viewing angle dependability, when compared to cross talk in 2-view stereoscopic displays. Regarding system cross talk there are objective measures to assess it in auto-stereoscopic displays. However, in addition to system cross talk, cross talk perceived by users is also impacted by scene content. Moreover, some cross talk is arguably beneficial in auto-stereoscopic displays. Therefore, in this paper, we further assess how cross talk is perceived by users with various scene contents and different viewing positions using auto-stereoscopic displays. In particular, the proposed subjective cross talk assessment methodology is realistic without restriction of the users viewing behavior and is not limited to the specific technique used in auto-stereoscopic displays. The test was performed on a slanted parallax barrier based auto-stereoscopic display. The subjective cross talk assessment results show their consistence to the system cross talk meanwhile more scene content and viewing position related cross talk perception information is provided. This knowledge can be used to design new cross talk perception metrics.
An HVS-based no-reference perceptual quality assessment of JPEG coded images using neural networks

In this paper, we present a novel no-reference (NR) metric to assess the quality of JPEG-coded images. The features for predicting the perceived image quality are extracted by considering the key human visual sensitivity factors such as, edge amplitude, edge length, background activity and background luminance. The extracted features with the subjective test results are used to train a multi-layer perceptron (MLP) neural network. Experimental results show that the prediction of the trained neural network is very close to the mean opinion score (MOS). The subjective test results of the proposed metric are compared with the Wang-Bovik's NR blockiness metric. Further, this metric can be extended to assess the quality of the MPLG/H.26x compressed videos.
Assessment of Stereoscopic Crosstalk Perception

Stereoscopic three-dimensional (3-D) services do not always prevail when compared with their two-dimensional (2-D) counterparts, though the former can provide more immersive experience with the help of binocular depth. Various specific 3-D artefacts might cause discomfort and severely degrade the Quality of Experience (QoE). In this paper, we analyze one of the most annoying artefacts in the visualization stage of stereoscopic imaging, namely, crosstalk, by conducting extensive subjective quality tests. A statistical analysis of the subjective scores reveals that both scene content and camera baseline have significant impacts on crosstalk perception, in addition to the crosstalk level itself. Based on the observed visual variations during changes in significant factors, three perceptual attributes of crosstalk are summarized as the sensorial results of the human visual system (HVS). These are shadow degree, separation distance, and spatial position of crosstalk. They are classified into two categories: 2-D and 3-D perceptual attributes, which can be described by a Structural SIMilarity (SSIM) map and a filtered depth map, respectively. An objective quality metric for predicting crosstalk perception is then proposed by combining the two maps. The experimental results demonstrate that the proposed metric has a high correlation (over 88%) when compared with subjective quality scores in a wide variety of situations.
Filter bank optimization for high-dimensional compression of pre-stack seismic data

A multi-dimensional variable-length subband coder for prestack seismic data was presented. A 2-D and 3-D separable near perfect reconstruction filter bank was optimized to maximize the coding gain, assuming that the correlation properties of pre-stack seismic data can be modeled by directional dependent autoregressive processes. Identical quantization and entropy coder allocation strategies were utilized to isolate the compression efficiency of the different high-dimensional filter bank methods. An example, with compression ratios ranging from 160:1 to 320:1, showed that 3-D subband coding of common shot gathers performed 50% better in terms of bit rate at a given signal-to-noise ratio compared to 2-D subband coding of common shot gathers.
Multimedia communications: research, standardisation and regulation, hand in hand

The evolution of 2nd to 3rd generation mobile communication systems is requiring a totally new approach to the development of services and applications. The applications are moving from speech to interactive multimedia and broadband services. The driving forces are a merging of technology, politics, and market demands. Success of this development is dependent on a close co-operation of manufacturers, the research community, regulators, and the standardisation bodies. The lack of a common understanding of the issues amongst them might be a problem. This paper addresses the different roles of the players and discusses the needs and possibilities for creating a fruitful environment suitable for deployment of broadband multimedia communications.
Estimating quality of experience on stereoscopic images

Stereo quality of experience (QoE) can be affected by many factors in three-dimensional (3D) media. We pay our attention to three main factors, namely scene content, camera baseline, and screen size, which were found to be the most significant influencing factors from our previous subjective tests. In order to estimate the QoE on stereoscopic presentations, this paper proposes a perceptual quality metric by characterizing these three factors using image disparity plus additional weights on screen size. The experimental results demonstrate that the proposed metric can achieve more than 87% correlation with subjective scores, which is better than the traditional two dimensional (2D) metrics without integrating 3D characteristics, with the ground truth of QoE from our subjective tests and a publicly available stereoscopic quality database.
Video quality metric based on fixation prediction and foveal imaging

This paper proposes a full-reference video quality metric based on foveated vision mechanism. Due to a non-uniform distribution of photo-receptors on the retina, the human visual system (HVS) has the highest resolution around the fixation point of eyes and dramatically decreases away from this point. Two key factors in the foveated vision are fixation point and retinal eccentricities of different visual objects. Based on an advanced video attention model in quality assessment scenarios, eye fixations are predicted from the attention map using a winner-takes-all (WTA) neural network. Four quality features describing distortions on luminance, spatial and temporal activities, as well as chrominance are derived between foveated representations of reference and distorted video frames. These quality features are then combined together by an appropriate spatiotemporal pooling scheme to build a video quality metric. Experimental results with respect to publicly available video quality databases demonstrate that the proposed quality model outperforms a previously proposed foveated video quality metric as well as state-of-the-art video quality models.
Performance evaluation of different filter banks in the JPEG-2000 baseline system

The performance of several decorrelating transforms is evaluated in the JPEG-2000 baseline system. The transforms considered are based on uniform parallel 2-channel and 8-channel filter banks and discrete cosine transform (DCT) building blocks. Through the analysis of lossy compression results, a system employing a combination of 8- channel and 2-channel filter banks is found to perform best overall. Also, the performance of several reversible transforms is evaluated in the same coding framework. Based on lossy and lossless compression results, a reversible version of the Cohen-Daubechies-Feauveau (2,2) wavelet transform was found to be most effective.
Rate-Distortion Optimized I-Slice Selection for Low Delay Video Transmission

Rate smoothing is essential for achieving lower delay when transmitting real-time video over the network. Recently, "explicit slice-based mode selection" (ESM) is proposed as a new way of achieving this goal together with its inherent quality smoothness and error resilience features. However previous studies focus on the practical aspects and do not address an optimized solution. In this paper, we propose a rate-distortion (RD) optimized solution for finding the best location and size of the intra-coded slices. The experimental results show that for a target bit rate the optimized scheme is able to offer performance close to that of mode selection on a macroblock level, over wireless channels with different packet loss rates. Moreover, the optimized ESM algorithm provides significant advantages of granular bit stream prioritization for network transmission. However, the RD based optimization is in general computationally expensive. We therefore propose a heuristic approach which incorporates both channel statistics and sequence characteristics. Results show that it yields close to optimal performance at lower complexity.