nils-herrmann/clean_scirepeval huggingface dataset:
Motion-compensated residue preprocessing in video coding based on just-noticeable-distortion profile

We present a motion-compensated residue signal preprocessing scheme in video coding scheme based on just-noticeable-distortion (JND) profile. Human eyes cannot sense any changes below the JND threshold around a pixel due to their underlying spatial/temporal masking properties. An appropriate (even imperfect) JND model can significantly help to improve the performance of video coding algorithms. From the viewpoint of signal compression, smaller variance of signal results in less objective distortion of the reconstructed signal for a given bit rate. In this paper, a new JND estimator for color video is devised in image-domain with the nonlinear additivity model for masking (NAMM) and is incorporated into a motion-compensated residue signal preprocessor for variance reduction toward coding quality enhancement. As the result, both perceptual quality and objective quality are enhanced in coded video at a given bit rate. A solution of adaptively determining the parameter for the residue preprocessor is also proposed. The devised technique can be applied to any standardized video coding scheme based on motion compensated prediction. It provides an extra design option for quality control, besides quantization, in contrast with most of the existing perceptually adaptive schemes which have so far focused on determination of proper quantization steps. As an example for demonstration, the proposed scheme has been implemented in the MPEG-2 TM5 coder, and achieved an average peak signal-to-noise (PSNR) increment of 0.505 dB over the twenty video sequences which have been tested. The perceptual quality improvement has been confirmed by the subjective viewing tests conducted.
Rate control for videophone using local perceptual cues

We present a method for extracting local visual perceptual cues and its application for rate control of videophone, in order to ensure the scarce bits to be assigned for maximum perceptual coding quality. The optimum quantization step is determined with the rate-distortion model considering the local perceptual cues in the visual signal. For extraction of the perceptual cues, luminance adaptation and texture masking are used as the stimulus-driven factors, while skin color serves as the cognition-driven factor in the current implementation. Both objective and subjective quality evaluations are given by evaluating the proposed perceptual rate control (PRC) scheme in the H.263 platform, and the evaluations show that the proposed PRC scheme achieves significant quality improvement in block-based coding for bandwidth-hungry applications.
Perceived visual quality metric based on error spread and contrast

This paper presents a new objective visual quality metric based on the measurement of error spread and isotropic local contrast of the image. It is shown that error spread is a reasonable measure for perceived visual quality while the image contrast is a content indicator for different images. The performance of the proposed method is shown to be superior compared to the 9 proponents in VQEG Phase I test dataset and another recently reported method in terms of prediction accuracy, monotonicity and consistency. In addition, the new method is very simple and does not need to tune any parameters.
Perceptually-adaptive pre-processing for motion-compensated residue in video coding

We present a perceptually-adaptive pre-processing scheme for motion-compensated residue, based on just-noticeable-distortion (JND) profile. Human eyes cannot sense any changes below the JND threshold around a pixel due to their underlying spatial/temporal masking properties. From the viewpoint of signal compression, a smaller variance of signal results in less objective distortion of the reconstructed signal for a given bit-rate. In this paper, the JND profile is incorporated into a motion-compensated residue signal preprocessor for variance reduction, aimed at coding quality enhancement. A solution of adaptively determining the parameter for the residue pre-processor is also proposed. Experimental results show that both perceptual quality and objective quality are enhanced in coded video at a given bitrate.
IN-service video quality monitoring

A video quality monitoring system for in-service monitoring of videos is described in this paper. The technologies developed include a no-reference method for blockiness measurement that exploits the characteristics of human visual system using the concept of just-noticeable-difference. This approach works even for scenario where the blockiness is not at known fixed locations. To provide a complete over-view of the integrated video quality monitoring system, methodologies for measuring picture freeze and picture loss have also been provided. The experimental results show that the proposed video quality monitoring system gives good accuracy for artefacts detections.
Spatial selectivity modulated just-noticeable-distortion profile for video

Both visual sensitivity and spatial selectivity determine the overall visibility threshold at each pixel in an image, according to the physiological and psychological evidence towards the human visual system (HVS). Visual sensitivity can be decided by an existing estimator for just-noticeable-distortion (JND). A computational model is proposed for incorporating a selectivity measure into the JND profile so that more effective noise shaping is possible in various applications. Experimental results with noise-embedded video sequences confirm that the introduction of spatial selectivity enhances the performance of the JND profile used in noise shaping.
On incorporating just-noticeable-distortion profile into motion-compensated prediction for video compression

We explore a new perceptually-adaptive video coding (PVC) scheme in the motion-compensated prediction loop for hybrid video compression, aiming at better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color images/video is first devised in image-domain based on our introduced nonlinear additivity model for masking (NAMM). Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation (ME) and residue filtering. The scheme works with any prevalent video coding standards and various ME strategies. As an example of implementation, it is applied to the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in ME efficiency, 0.6 dB in perceptual coding quality and most remarkably, 0.17 dB in the objective coding quality measure (PSNR).
Emotional facial expression transfer based on temporal restricted Boltzmann machines

Emotional facial expression transfer involves sequence-to-sequence mappings from an neutral facial expression to another emotional facial expression, which is a well-known problem in computer graphics. In the graphics community, current considered methods are typically linear (e.g., methods based on blendshape mapping) and the dynamical aspects of the facial motion itself are not taken into account. This makes it difficult to retarget the facial articulations involved in speech. In this paper, we apply a temporal restricted Boltzmann machines based model to emotional facial expression transfer. The method can encode a complex nonlinear mapping from the motion of one neutral facial expression to another emotional facial expression which captures facial geometry and dynamics of both neutral state and emotional state.
An effective perceptual weighting model for videophone coding

A perceptual weighting model is proposed for effective rate control so as to enhance the perceptual coding quality of videophone. We exploit two categories of factors affecting the perception of the human visual system, stimulus-driven factors and cognition-driven factors. In order to achieve a simple, but effective, perceptual weighting model, we use luminance adaptation and texture masking as the stimulus-driven factors, while skin color serves as the cognition-driven factor in the videophone application. Both objective and subjective quality evaluations of videophone-like sequences in the H.263 platform validate the effectiveness of our perceptual weighting model.
Blocking artifacts reduction for DCT-based image compression using neurofuzzy driven anisotropic diffusion

In this paper, we propose a new deblocking method for DCT-based compressed images using anisotropic diffusion. An adaptive anisotropic coefficient, derived from a neuro-fuzzy network, is introduced. The optimal diffusion parameters are estimated using a stochastic optimization technique, which ensures that heavy diffusion performs across the block boundaries and in flat regions while image edges are well preserved. Experimental results have shown that the proposed adaptive diffusion method can significantly suppress coding artifacts and improve the visual quality of the compressed image.
Video Quality Metrics - An Analysis for Low Bit Rate Videos

Objective video quality metrics reported in the literature have so far been focused on TV signals with large frame sizes, full TV frame rates, and high compressed bit rates. The aim of this paper is to provide an analysis of 4 objective video quality metrics tested on low bit rate videos with small frame sizes, various frame rates, and low bit rates. The video quality metrics used were the NTIA video quality metric, modified Watson's digital video quality metric, video structural similarity metric, and the perceptual video quality metric. The test videos consist of H.264 compressed videos with CIF and QCIF frame sizes, at various bit rates (24 kbps - 384 kbps) and frame rates (7.5 fps - 30 fps). Here, results and analysis derived from the comparison of the metrics are provided.
Colour perceptual video quality metric

This paper proposed an objective colour perceptual video quality metric to automatically assess the perceived quality of colour digital videos. The rationale in proposing perceptual-based metric is because peak signal-to-noise ratio (PSNR), which has been used traditionally to represent the quality of a compressed video sequence, has been found to correlate poorly with subjective quality ratings, particularly at much lower bit rates and frame rates. In this paper, computational models have been applied to emulate human visual perception based on a combination of local and global modulating factors. The proposed video quality metric has been tested on CIF and QCIF colour video sequences compressed at various bit rates and frame rates. It has been shown to give significantly better correlations to human perception than PSNR.
Combining H.264 and MPEG2 for multi-resolution video applications

The recent completion of the H.264 standard has opened new areas of research in the area of scalability. Spatial scalability for high definition sequences is particularly important, since future DVDs and broadcasts will undoubtedly be in high definition formats. In this paper, a new method of combining a low-resolution MPEG2 base layer with a high-resolution H.264 enhancement layer is explored. For a CIF sequence, the proposed scalability scheme offers bitrate savings (when compared to H.264 alone) of up to 54%, depending on the bitrate of the MPEG2 base layer.
Effect of displayed contrast on perceptual 3D video quality

There are several different factors that affect the perceived quality of 3D content. In this paper we study how the overall 3D visual perception is affected by the contrast between the objects of interest and the background in 3D-displayed videos. Our evaluations showed that the Weber contrast between objects of interest and background should be within the range of -0.5 to 1.5 to provide viewers with high quality of 3D experience.
Evaluation of chroma subsampling for high dynamic range video compression

Subsampling chroma channels of video content is a processing performed in most video distribution pipelines. In such pipelines, chroma subsampled content also undergoes video coding, using a codec, before being distributed. Presently, it is common practice to optimize compression efficiency and visual quality of content with respect to its subsampled version rather than the original one. Although this may suffice for traditional imagery, it is unclear if emerging technologies such as Wide Color Gamut and High Dynamic Range are more sensitive to quality loss due to chroma subsampling. In this article, we assess the efficiency of chroma subsampling when compressing HDR content. We compare the performance of two different downsampling filters against the full chroma sampling. Objective results show that distributing 4:4:4 is more efficient than its 4:2:0 counterpart at medium to high bit-rates. For low bit-rates, this increase in efficiency is reduced and in some cases even reversed.
Fast vignetting correction and color matching for panoramic image stitching

When images are stitched together to form a panorama there is often color mismatch between the source images due to vignetting and differences in exposure and white balance between images. In this paper a low complexity method is proposed to correct vignetting and differences in color between images, producing panoramas that look consistent across all source images. Unlike most previous methods which require complex non-linear optimization to solve for correction parameters, our method requires only linear regressions with a low number of parameters, resulting in a fast, computationally efficient method. Experimental results show the proposed method effectively removes vignetting effects and produces images that are highly visually consistent in color and brightness.
3-D Scalable Medical Image Compression With Optimized Volume of Interest Coding

We present a novel 3-D scalable compression method for medical images with optimized volume of interest (VOI) coding. The method is presented within the framework of interactive telemedicine applications, where different remote clients may access the compressed 3-D medical imaging data stored on a central server and request the transmission of different VOIs from an initial lossy to a final lossless representation. The method employs the 3-D integer wavelet transform and a modified EBCOT with 3-D contexts to create a scalable bit-stream. Optimized VOI coding is attained by an optimization technique that reorders the output bit-stream after encoding, so that those bits belonging to a VOI are decoded at the highest quality possible at any bit-rate, while allowing for the decoding of background information with peripherally increasing quality around the VOI. The bit-stream reordering procedure is based on a weighting model that incorporates the position of the VOI and the mean energy of the wavelet coefficients. The background information with peripherally increasing quality around the VOI allows for placement of the VOI into the context of the 3-D image. Performance evaluations based on real 3-D medical imaging data showed that the proposed method achieves a higher reconstruction quality, in terms of the peak signal-to-noise ratio, than that achieved by 3D-JPEG2000 with VOI coding, when using the MAXSHIFT and general scaling-based methods.
A rate adaptation approach for streaming multiview plus depth content

Three-dimensional video has recently gained great popularity as it enhances the viewing experience. Streaming of 3D video over IP networks will soon become a common means of multimedia consumption, thanks to the recent standardization efforts such as MPEG-DASH and 3D-HEVC. To this end, rate adaptation techniques are required to overcome the unreliable nature of IP networks. In this paper, we propose a rate adaptation approach for streaming multiview plus depth (MVD) video by exploring the effects of the number of views and the quality of views on the quality of experience. Specifically, we show that concurrently decreasing the distance between virtual camera positions and reducing the number of views help maintain an acceptable perceptual quality level when the network resources are scarce.
An efficient early-termination mode decision algorithm for H.264

We present a novel early-termination motion estimation scheme specifically designed for the H.264 standard. Our method takes advantage of correlation between the rate-distortion cost of the 16times16 pixel motion search and the cost of the potential best choice for the block size in order to terminate the motion search before all possible options have been checked. A pixel decimation method yields additional reduction in computational complexity. The method is suitable for supplementing various existing fast motion estimation algorithms. When supplementing the EPZS method  presently used in H.264, our method achieves total computational savings of 55% for almost the same picture quality, and a slight increase in bitrate.
Trade off between source and channel distortion for a data-partitioned video in a capacity constrained network

Video transmission is affected by two types of distortions: source distortion, which is caused by the encoder parameters; and channel distortion for which the error prone characteristics of the wireless network is responsible. In this paper, we vary the Quantization Parameter (QP) to determine the effect of source distortion. In a capacity constrained network, changing QP has contradictory effects: smaller QP decreases the source distortion but the bit rate is increased, leaving small room for employing error protection techniques. In a highly lossy environment, this might not be the optimal solution. In this paper, by changing the QP we determine the trade off between source and channel distortion that leads to the best possible picture quality performance.
Correcting Sharpness Variations in Stereo Image Pairs

When stereo images are captured under less than ideal conditions, there may be inconsistencies between the two images in brightness, contrast, blurring, etc. When stereo matching is performed between the images, these variations can greatly reduce the quality of the resulting depth map. In this paper, we propose a method for correcting sharpness variations in stereo image pairs which is performed as a preprocessing step to stereo matching. We modify the more blurred of the two images so that it will match the less blurred image in sharpness as closely as possible. The 2D discrete cosine transform (DCT) of both images is taken, and the DCT coefficients are divided into a number of frequency bands. The signal and noise energy in each band are estimated, and the coefficients are scaled so that the two images have equal signal energy in each band. Experiments show that applying the proposed correction method can greatly improve the disparity map quality when one image in a stereo pair is more blurred than the other.
An H.264-based scheme for 2D to 3D video conversion

An efficient method that converts 2D video sequences to 3D is presented. This method utilizes the motion information between consecutive frames to approximate the depth map of the scene. To estimate the depth map, the horizontal motion captured by a single camera is revised and then approximated as the displacement between the right and left frames captured by two cameras in a stereoscopic set-up case. To enhance the visual depth perception, a non-linear scaling model is then applied to the modified motion vectors. The low complexity of our approach and its compatibility with future 3D systems, allows real-time implementations at the receiver-end for no additional burden on the network. Performance evaluations show that our approach outperforms the existing H.264-based depth map estimation technique by 1.84 dB PSNR, providing more realistic depth representation of the scene. Moreover, the subjective comparison of results (obtained by viewers watching the generated stereo video sequences on a 3D display system) confirms the better performance of our method.
An Efficient Compression Scheme for Colour Filter Array Images Using Estimated Colour Differences

Most consumer digital cameras capture colour images using a single light sensor and a colour filter array (CFA). The CFA allows only one colour of light (red, green, or blue) to reach the sensor at each pixel location. This results in a mosaic image being captured, where either a red, green or blue sample is captured at each location. The two missing colours must be interpolated from the surrounding samples in a process called demosaicking. The conventional approach to performing compression in these cameras is to first perform demosaicking and then compress the resulting full colour image with standard methods. However, recent work has attempted to compress the raw data captured by the sensor, with demosaicking being performed later. One challenge with this method (compression before demosaicking) is exploiting the correlation between the red, green and blue colour planes. Existing methods have used a modified YCbCr colour space conversion which is applied to groups of RGB samples captured at adjacent pixel locations. This results in poor performance around edges, where there is much lower correlation between the RGB samples. In this paper, we propose a new method of exploiting correlation between RGB samples when compressing CFA images. We use a 2D 5times5 linear filter to estimate the green value at each location where a red or blue sample is captured. We then compress the captured green samples, and two "chrominance" planes, which are obtained by taking the difference between the captured red and blue samples and the estimated green value at the same location. This results in much smoother chrominance signals than the modified YCbCr conversion used in previous work. Simulation results show that for different images, the proposed method results in bit rate reductions between 12-25% relative to the previous modified YCbCr conversion when compressing CFA images with JPEG.
A low complexity mode decision approach for HEVC-based 3D video coding using a Bayesian method

The 3D extension of High Efficiency Video Coding (HEVC) standard (3D-HEVC) aims at improving coding efficiency by introducing new and unique approaches for utilizing correlations between the different views of a scene. Reported coding efficiency, however, comes at the expense of increased computational complexity. For real-time applications, reducing the computational complexity of 3D-HEVC is very important. In this paper, we propose an adaptive fast mode assigning method based on a Bayesian classifier that reduces 3D-HEVC's coding complexity by up to 51.95%, while maintaining the overall quality and bitrate.
A learning-based visual saliency fusion model for High Dynamic Range video (LBVS-HDR)

Saliency prediction for Standard Dynamic Range (SDR) videos has been well explored in the last decade. However, limited studies are available on High Dynamic Range (HDR) Visual Attention Models (VAMs). Considering that the characteristic of HDR content in terms of dynamic range and color gamut is quite different than those of SDR content, it is essential to identify the importance of different saliency attributes of HDR videos for designing a VAM and understand how to combine these features. To this end we propose a learning-based visual saliency fusion method for HDR content (LVBS-HDR) to combine various visual saliency features. In our approach various conspicuity maps are extracted from HDR data, and then for fusing conspicuity maps, a Random Forests algorithm is used to train a model based on the collected data from an eye-tracking experiment. Performance evaluations demonstrate the superiority of the proposed fusion method against other existing fusion methods.