nils-herrmann/clean_scirepeval huggingface dataset:
Adaptive motion vector resolution prediction in block-based video coding

In the classical block-based video coding, motion vector is derived for each coding block to remove the inter-frame redundancy. However, the motion vector resolution is usually restricted to be identical, typically 1/4-pixel resolution, regardless of the different video contents. In this paper, we propose an algorithm that can adaptively select the optimal motion vector resolution at frame level according to the characteristics of the video contents. We first derived a residual energy model, and the major factors that may impact the motion vector resolution are considered, including the texture complexity, motion scale, inter-frame noise and quantization parameter. Experimental results have shown that the proposed scheme can achieve 1.8% BD-rate gain on average without complexity increment.
Transform-domain in-loop filter with block similarity for HEVC

In-loop filtering is an important technique in modern video coding standards. In this paper, we propose a transform-domain in-loop filter to further improve the compression performance of high efficiency video coding (HEVC) standard. The proposed method estimates block transform coefficients by adaptively fusing two prediction sources according to their uncertainties respectively. The first prediction is the block transform coefficients of compressed video frames, the uncertainty of which is related to quantization parameters. The second prediction is the weighted average of transform blocks in a neighborhood, and the weights are designed according to block similarity. Its uncertainty is estimated based on the coefficient variance. To optimize the filtering performance, the parameters utilized in the proposed in-loop filter are learned from compressed videos for each quantization parameter offline, and frame level flags are utilized to switch the proposed in-loop filter according to rate-distortion cost. Extensive experimental results show that the proposed in-loop filter can further improves the compression efficiency of HEVC.
A fast background model based surveillance video coding in HEVC

High Efficiency Video Coding (HEVC) is the most up to dated video coding standard which reduces the bit rate by almost 40% with the same objective quality compared to H.264/AVC but at about 40% extra encoding complexity overhead. The main reason for HEVC complexity increase is inter prediction that accounts for 60-70% of the whole encoding time. Especially for surveillance video with huge amount of data, the reducing of coding complexity is a very pressing work. So in this paper, a novel fast method is proposed based on background model to speed up the searching procedure of surveillance video coding. This paper analyzes the proportion of different block partitions and reference frames for multiple types blocks, and puts the analysis results into practice by developing optimized mode decision and reference frame selection schemes for the HEVC encoder according to spatial-temporal corresponding CUs (coding units) or PUs (prediction units). Simulation results show that the proposed algorithm can achieve more than 41 % reduction of coding time with comparable rate distortion (RD) performance.
Optimized inter-view prediction based light field image compression with adaptive reconstruction

In this paper, we explore the structure of light field (LF) and efficiently improve the performance of the pseudo sequence based lenslet image compression, by optimized sub-view rearrangement, enhanced illumination compensation and adaptive reconstruction filtering. First, the decomposed sub-view images are rearranged into a pseudo sequence according to our optimized scan order based on sub-view correlation. Second, the generated pseudo sequence is compressed with JEM codec in which we enhance the illumination compensation by adaptively selecting reference pixels used in parameter derivation. Finally, to reduce the distortions in lenslet image decomposition and reconstruction, we propose an lenslet reconstruction method by applying adaptive filters to the reconstructed lenslet images to compensate the reconstruction errors. Each filter is derived by minimizing the distortions between the original and reconstructed pixels with same geolocation in the lenslet image. Extensive experimental results show that the proposed method achieves up to 53.7% bit rate reduction over HEVC intra coding and 34.8% over JEM intra coding in terms of BDBR.
Flexible CTU-level parallel motion estimation by CPU and GPU pipeline for HEVC

In the high efficiency video coding (HEVC) encoder, motion estimation (ME) takes up more than 50% encoding time. To reduce the complexity of the ME module in HEVC, this paper proposes a flexible coding tree unit (CTU)-level parallel ME method through CPU and GPU pipeline collaboration. Firstly a highly scalable CTU-level parallel motion search scheme on GPU is provided, in which, the parallel CTU group can be configured to be any size to adapt to the variable sequence resolution and hardware configurations. Then, the motion search range can be adaptively adjusted based on the motion intensity. Therefore, the unnecessary GPU time wasting can be further avoided for slow-moving scenes, while high performance kept for fast-moving scenes. Moreover, the ME information returned from GPU can be used by CPU for fast mode decision. Experimental results show that the proposed method achieves up to 73% complexity reduction than HM10.0 anchor using CPU only with acceptable coding performance loss, providing higher performance than the state-of-the-art scheme.
On-line energy distribution for hybrid electric vehicles using optimal power-split-ratio

This paper presents a near-optimal on-line energy control strategy based on power-split-ratio for hybrid electric vehicles. Firstly, a typical series-parallel hybrid electric vehicle model, Prius II, is developed and validated by ADVISOR. Secondly, the optimal energy distribution problem is initially solved by dynamic programming. Considering the frequent engine start-stop phases in existing energy management strategies, the objective of dynamic programming is to optimally balance the fuel consumption and the engine start-stop phases. Lastly, through analysis of the engine operating points obtained from dynamic programming results, the improved control rules are proposed by obtaining the engine optimal operating line and finding the optimal power-split-ratio in different driving situations. Moreover, A Matlab/Simulink model is built to evaluate the validity of the proposed control rules. Four different driving cycles are tested, and the results show 10%-30% performance improvement in comparison with the results in ADVISOR.
Nonlocal Adaptive In-Loop Filter via Content-Dependent Soft-Thresholding for HEVC

In-loop filters have been widely utilized in latest video coding standards to improve the video coding efficiency by reducing compression artifacts. However, existing in-loop filters only utilize image local correlations, leading to limited performance improvement. In this paper, we explore a novel adaptive in-loop filter by means of the nonlocal similar content to improve the quality of reconstructed video frames. In our proposed filter, the input video frame is first divided into different image patch groups based on their similarity, and then a soft-thresholding method is applied to the singular values of matrices composed of image patches in every group. Since compression noise is highly correlated with image content, we propose a group-wise threshold estimation method based on image statistical characteristics, coding modes and quantization parameters. To ensure the filtering efficiency, slice level control flags are utilized and determined based on the distortion changes after filtering. The proposed in-loop filter is integrated into HM7.0, and experimental results show that it can significantly improve the performance of HEVC on top of the state-of-the-art in-loop filters.
Effective Quadtree Plus Binary Tree Block Partition Decision for Future Video Coding

Block partition structure has been recognized as a crucial module in video coding scheme. Recently, a quadtree plus binary tree (QTBT) block partition structure has been proposed in the Joint Video Exploration Team (JVET) development. Compared to the quadtree structure in HEVC, QTBT can achieve better coding performance with hugely increased encoding complexity. Here, we propose an effective QTBT partition decision algorithm to achieve a good trade-off between computational complexity and coding performance. In particular, at the Coding Tree Unit level, the partition parameters of QTBT are dynamically derived to adapt to the local characteristics without transmitting any overhead. Subsequently, at the Coding Unit level, a joint-classifier decision tree structure is designed to eliminate unnecessary iterations and meanwhile control the risk of false prediction. Experimental results show that the proposed algorithm can achieve 64% encoding time reduction on average with only 1.26% increase in terms of bit rate. This greatly benefits the practical implementations of QTBT in real application scenarios.
A study on the rate distortion modeling for High Efficiency Video Coding

This paper gives a study on the rate distortion modeling for High Efficiency Video Coding (HEVC) and new rate distortion models are proposed by considering the coding framework change of HEVC. Although HEVC is also characterized by the traditional prediction/transform hybrid coding framework with more improved coding tools, e.g. large size coding unit (CU), prediction unit (PU) and transform unit (TU), two new coding modules are added as sample adaptive offset(SAO) and adaptive loop filter(ALF). SAO and ALF are added after the motion compensation reconstruction, which would reduce the distortion of the reconstructed image. With considerations on these changes, new rate distortion models are proposed. For rate modeling, this paper proposes a separate model for header bits and coefficient bits respectively. For distortion modeling, the quantization distortion and the distortion reduction induced by SAO/ALF are modeled jointly. The experimental results show that the coding rate and distortion can be estimated by the proposed model accurately. These models can be used for rate distortion cost estimation for mode decision or rate control in the future.
Perceptually optimized sparse coding for HDR images via divisive normalization

High dynamic range (HDR) imaging techniques have been widely advocated that could shape next generation of digital photography. However, the popularity of HDR contents is hindered by the lack of displaying devices for rendering HDR images which could be very expensive. To tackle this, extensive tone-mapping operators (TMOs) have been proposed in order for transforming HDR images to viewable low dynamic range (LDR), and also applied in the backward-compatibility based HDR compression. However, how to efficiently improve the compression performance based on the perceptual evaluation is seldom addressed. In this work, we first propose a quality evaluation index for measuring the quality of the LDR image with the access of pristine HDR image. Then a sparse coding framework for efficiently compressing the LDR image, which is generated from its HDR version using TMO, is presented. Finally the compression efficiency could be improved by jointly optimize the sparse coding process in terms of the proposed quality metric based on the divisive normalization mechanism. Extensive experiments have shown that the proposed scheme can improve the perceptual quality of the compressed LDR image.
Stereoscopic video quality assessment model based on spatial-temporal structural information

Most of the existing 3D video quality assessment methods estimate the quality of each view independently and then pool them into unique objective score. Besides, they seldom take the motion information of adjacent frames into consideration. In this paper, we propose an effective stereoscopic video quality assessment method which focuses on the inter-view correlation of spatial-temporal structural information extracted from adjacent frames. The metric jointly represents and evaluates two views. By selecting salient pixels to be processed and discarding the others, the processing speed is significantly improved. Experimental results on our stereoscopic video database show that the proposed algorithm correlates well with subjective scores.
Thousand to one: An image compression system via cloud search

With the advent of the `big data' era, a huge number of images are produced every day. Traditional image compression methods no longer satisfy the demand to store and transmit them. In this paper, we face this challenge and take advantage of the correlations existing between images to achieve a higher compression rate. We propose an image compression system that encodes each image by referencing its correlated images in the cloud. We first extract features from an image and retrieve its similar image from the massive images in the cloud by comparing these features. Then we preprocess the retrieved picture by applying projective transformation and illumination compensation to obtain multiple reference images of higher prediction accuracy. By taking advantage of the redundancy between the reference images and the current image, we encode the current image through prediction coding techniques. The experimental results demonstrate that the proposed method outperforms JPEG and HEVC intra coding by 61.5% and 21.3% on average, respectively. It has an average compression ratio of over a thousand to one.
Utility-Driven Adaptive Preprocessing for Screen Content Video Compression

In this work, we propose a utility-driven preprocessing technique for high-efficiency screen content video (SCV) compression based on the temporal masking effect, which was found to be a fundamental attribute that plays an important role in human visual perception of video quality, but has not been fully exploited in the context of SCV coding. Specifically, we investigate the temporal masking effect from the perspective of perceived utility, which allows us to preserve the quality of the high utility content and substitute the low utility regions with the corresponding smooth version. To distinguish the regional utilities, a specifically designed block type identification algorithm for screen content is employed to measure the local properties. Subsequently, the Gaussian filter is applied to smooth out the high-frequency components in the detected low utility regions to save consumption bits. Validations based on subjective testings show that the proposed approach is capable of achieving significant bitrate savings with little sacrifice on the final utility compared with the conventional SCV coding scheme.
Quality assessment of tone-mapped images based on sparse representation

Recently, an increasing number of tone-mapping operators (TMOs) have been proposed in order to display high dynamic nge (HDR) images on low dynamic range (LDR) devices. Developing perceptually consistent image quality assessment (QA) measures for TMO is highly desired because traditional LDR based IQA methods cannot support the cross dynamic range quality comparison. In this paper, a novel objective quality assessment method is proposed on the basis of sparse-domain representation, which has been well advocated as a powerful tool in describing natural sparse signals with the over-complete dictionary. Specifically, two indices, incorporating both local and global features extracted from sparsely represented coefficients, are introduced to simulate the human visual system (HVS) characteristics on HDR images. The local feature measures the sparse-domain similarity between the pristine HDR and tone-mapped L R images by leveraging the intrinsic structure with sparse coding. On the other hand, benefiting from the natural scene statistics (NSS), the global features are recovered from the sparse coefficients to account for the natural behaviors of tone-mapped images. Combining the local sparse-domain similarity and the global “naturalness” prior, validations on the public database show that the proposed sparse-domain model for tone-mapped images (SMTI) provides accurate predictions on the human perception of tone-mapped images.
Probabilistic Decision Based Block Partitioning for Future Video Coding.

In the latest Joint Video Exploration Team development, the quadtree plus binary tree (QTBT) block partitioning structure has been proposed for future video coding. Compared to the traditional quadtree structure of High Efficiency Video coding (HEVC) standard, QTBT provides more flexible patterns for splitting the blocks, which results in dramatically increased combinations of block partitions and high computational complexity. In view of this, a confidence interval based early termination (CIET) scheme is proposed for QTBT to identify the unnecessary partition modes in the sense of rate-distortion (RD) optimization. In particular, a RD model is established to predict the RD cost of each partition pattern without the full encoding process. Subsequently, the mode decision problem is casted into a probabilistic framework to select the final partition based on the confidence interval decision strategy. Experimental results show that the proposed CIET algorithm can speed up QTBT block partitioning structure by reducing 54.7% encoding time with only 1.12% increase in terms of bit rate. Moreover, the proposed scheme performs consistently well for the high resolution sequences, of which the video coding efficiency is crucial in real applications.
Tagged multi-hypothesis motion compensation scheme for video coding

Accuracy of prediction block (PB) plays a very important role in improving the coding performance. In this paper, we propose tagged multi-hypothesis motion compensation scheme (TMHMC) for inter frames to improve the accuracy of PB. TMHMC not only makes use of temporal correlation between frames but also the spatial correlation as motion vectors of adjacent blocks are used to derive the PB. For entropy coding process, only one motion vector and a tag indicating which adjacent block is used are coded in bit-stream. Adding TMHMC scheme as an additional mode in MPEG internet video coding (TVC) platform, the bitrate saving is up to 12% at the same objective quality compared with anchor. Average bitrate saving is close to 6% over all test sequences. In addition, we also implement the conventional multi-hypothesis motion compensation (MHMC) scheme. 3% bitrate is further saved on average by TMHMC compared with conventional MHMC.
Progressive motion vector resolution for HEVC

This paper proposes a progressive motion vector resolution (PMVR) method for High Efficiency Video Coding (HEVC). In the proposed scheme, high motion vector (MV) resolutions, e.g. 1/4 or 1/8 pixel resolution, are employed for MVs near to the motion vector predictor (MVP) and low MV resolutions are employed for MVs far from the MVP. The range of each MV resolution is indicated by a threshold parameter. And a new motion vector difference (MVD) derivation method is designed to encode MVD efficiently. Experimental results show that PMVR with 1/8 pixel motion search can achieve a BD-rate gain up to 16% with almost the same coding time with HM8.0, and for PMVR without 1/8 pixel motion search, up to 6.1% BD-rate gain can be achieved with 9% encoding time saving on average.
Adaptive Progressive Motion Vector Resolution Selection Based on Rate-Distortion Optimization.

In the state-of-the-art H.265/HEVC video coding standard, the motion vector (MV) resolution is fixed to be 1/4-pel for the entire video sequence, while the inherent video characteristics, e.g., texture complexities and motion activities, have been largely ignored. Obviously such strategy may not suffice the demand of high-accuracy motion compensation. In this paper, we propose a specially designed rate-distortion model in terms of the MV resolution by taking the video characteristics into consideration. In particular, the MV resolution selection is formulated as a rate-distortion optimization problem by analyzing the rate-distortion cost of each MV resolution candidate. To further improve the coding performance, the progressive MV resolution strategy is employed, where the optimal progressive MV resolution is determined by decision trees constructed with the rate-distortion model. In this manner, a novel adaptive progressive motion vector resolution selection scheme can be realized and the MV resolution can be adaptively adjusted based on the properties of local content. Extensive experiments and comparisons show that the proposed algorithm significantly improves the coding performance, and 1.8% BD-rate gain on average has been achieved without introducing any noticeable computational complexity.
A VLSI Architecture for a Fast Computation of the 2-D Discrete Wavelet Transform

Video encoding to yield a decoder-friendly H.264 bit stream that consumes less decoding power yet with little coding efficiency degradation is investigated in this work. The in pipeline mode enhances the computing time by deblocking filters (ADF). We first propose a power among the three stages and by incorporating parallelism at encoder performs the rate-distortion-decoder complexity optimization (RDC) to save the decoder power needed for deblocking filter operations, which is called the decoder-friendly adaptive deblocking filter (DF-ADF) mode decision. The RDC optimization framework presents a way to balance coding efficiency and the ADF decoding cost in the mode decision process. The effectiveness of the proposed DF-ADF algorithm is demonstrated by experiments with diverse video contents and bit rates.
H.264/AVC-Based Depth Map Sequence Coding Using Improved Loop-filter

Research has been widely carried out on three dimensional television(3DTV) system using Depth-Image-Based Rendering (DIBR) techniques. The depth data is a kind of important 3D data, transmitted along with 2D texture data. At present, coding of depth map sequence takes existing H.264/AVC standard as a major means. As a key method to improve the video visual quality, loop-filter technology of H.264/AVC standard exerts great impact on the compressed depth map as well as the visual effects of stereo video. This paper proposes an improved loop-filter algorithm based on H.264/AVC, which solves the severe quality degradation problem existing in loop-filter for depth map sequence coding. While improving the coding quality, the proposed algorithm can greatly reduce the storage cost and computation complexity.
Intermediate view synthesis based on edge detecting

Intermediate view synthesis is a key technique for free-viewpoint video applications. Various methods have been proposed in this filed. However, most of those methods are so complex that they can't be realized for realtime. In addition, a perfect depth image is hard to acquire for the mixtures of foreground pixels and background pixels at the object boundary. The errors in depth map would involve many holes in intermediate views when it is synthesized between the left-most view and right-most view based on depth map. How to handle the artifacts around object boundaries in intermediate view is another challenge. In this paper, we propose a simple but effective method to enhance the synthesized intermediate views based on edge detecting(SIVBE). Experiment results on typical test sequences show that the quality of intermediate view can be improved by 1.3 dB and the proposed algorithm achieves a reduction of 50% computational complexity on average.