nils-herrmann/clean_scirepeval huggingface dataset:
CONCOLOR: Constrained Non-Convex Low-Rank Model for Image Deblocking

Due to independent and coarse quantization of transform coefficients in each block, block-based transform coding usually introduces visually annoying blocking artifacts at low bitrates, which greatly prevents further bit reduction. To alleviate the conflict between bit reduction and quality preservation, deblocking as a post-processing strategy is an attractive and promising solution without changing existing codec. In this paper, in order to reduce blocking artifacts and obtain high-quality image, image deblocking is formulated as an optimization problem within maximum a posteriori framework, and a novel algorithm for image deblocking using constrained non-convex low-rank model is proposed. The lp (0 <; p <; 1) penalty function is extended on singular values of a matrix to characterize low-rank prior model rather than the nuclear norm, while the quantization constraint is explicitly transformed into the feasible solution space to constrain the non-convex low-rank optimization. Moreover, a new quantization noise model is developed, and an alternatively minimizing strategy with adaptive parameter adjustment is developed to solve the proposed optimization problem. This parameter-free advantage enables the whole algorithm more attractive and practical. Experiments demonstrate that the proposed image deblocking algorithm outperforms the current state-of-the-art methods in both the objective quality and the perceptual quality.
In-scale motion aligned temporal filtering

To handle the mismatch problems of spatial-domain motion aligned temporal filtering (MATF) in providing spatial scalability, this paper presents a novel in-scale motion aligned temporal filtering (ISMATF) for highly scalable video coding. For each lifting step of ISMATF, the predict or update is carried out at various resolutions in a scale-by-scale way and the operation at higher resolution is designed to embed the operations at lower resolutions to prevent redundancy in frame representation. The proposed ISMATF has several main features, which as a whole differentiates it from other schemes (e.g. spatial-domain MATF, in-band MATF, straightforward multi-resolution MATF) and makes it effective in video coding with spatial scalabilities. These features include 1) Multi-resolution lifting with perfect reconstruction capability; 2) non-redundancy in frame representation, which favors highly scalable video coding; 3) temporal filtering performed in the image-domain of corresponding resolution, which is different from in-band schemes and preserves high efficiency of motion compensation; 4) no mismatch between the encoder and decoder, no matter which resolution is decoded. The proposed ISMATF scheme solves the mismatch problems of SDMATF at low resolution while maintains a good performance at high resolution. It can outperform redundant multi-resolution MATF schemes by up to 1.0 dB at high resolution
PET Protection Optimization for Streaming Scalable Videos With Multiple Transmissions

This paper investigates priority encoding transmission (PET) protection for streaming scalably compressed video streams over erasure channels, for the scenarios where a small number of retransmissions are allowed. In principle, the optimal protection depends not only on the importance of each stream element, but also on the expected channel behavior. By formulating a collection of hypotheses concerning its own behavior in future transmissions, limited-retransmission PET (LR-PET) effectively constructs channel codes spanning multiple transmission slots and thus offers better protection efficiency than the original PET. As the number of transmission opportunities increases, the optimization for LR-PET becomes very challenging because the number of hypothetical retransmission paths increases exponentially. As a key contribution, this paper develops a method to derive the effective recovery-probability versus redundancy-rate characteristic for the LR-PET procedure with any number of transmission opportunities. This significantly accelerates the protection assignment procedure in the original LR-PET with only two transmissions, and also makes a quick and optimal protection assignment feasible for scenarios where more transmissions are possible. This paper also gives a concrete proof to the redundancy embedding property of the channel codes formed by LR-PET, which allows for a decoupled optimization for sequentially dependent source elements with convex utility-length characteristic. This essentially justifies the source-independent construction of the protection convex hull for LR-PET.
An adaptive hierarchical QP setting for screen content coding

Screen content refers to computer generated content like text, graphics, and animations. In such video, many regions may remain static for a long period after a sudden change. Traditional hierarchical Quantization Parameter (QP) setting may not be able to handle these regions efficiently because the encoder probably needs to refine the quality of these static regions multiple times. It will cost more bits while the quality of the static regions may reach the expected degree which the flat QP setting is able to achieve. This paper proposes using different QP settings for different regions in a picture. Region classification algorithms are developed to determine whether a flat or hierarchical QP setting is used. Experimental results demonstrate that the proposed scheme can achieve an average bitrate reduction of 3.1%, and up to 8.1% bitrate reduction for IBBB coding. The proposed method improves coding efficiency without increasing encoding complexity.
High quality image reconstruction via non-local collaborative estimation for wireless image/video softcast

For wireless scenarios where the channel condition fluctuates unpredictably, a novel image/video communication scheme, named SoftCast, was recently proposed to provide graceful quality degradation and competitive performance simultaneously. Unlike conventional approaches, SoftCast decorrelates input images by a transform and modulates the coefficients directly to a dense constellation for transmission, leaving out the conventional quantization, entropy coding and channel coding. The transmission is lossy in nature, with its noise level commensurate with the channel condition. To reconstruct images from the received noisy data, SoftCast employs a linear least-square estimator (LLSE), but it tends to produce annoying reconstruction artifacts. This paper proposes a high-quality image reconstruction algorithm for SoftCast, employing a collaborative estimator to utilize both the local correlation and non-local similarity within images. Experimental results show that the proposed method outperforms the existing SoftCast scheme, achieving remarkable improvement in the objective and subjective qualities of the reconstruction images.
Image super-resolution via dual-dictionary learning and sparse representation

Learning-based image super-resolution aims to reconstruct high-frequency (HF) details from the prior model trained by a set of high- and low-resolution image patches. In this paper, HF to be estimated is considered as a combination of two components: main high-frequency (MHF) and residual high-frequency (RHF), and we propose a novel image super-resolution method via dual-dictionary learning and sparse representation, which consists of the main dictionary learning and the residual dictionary learning, to recover MHF and RHF respectively. Extensive experimental results on test images validate that by employing the proposed two-layer progressive scheme, more image details can be recovered and much better results can be achieved than the state-of-the-art algorithms in terms of both PSNR and visual perception.
Low-Rank-Based Nonlocal Adaptive Loop Filter for High-Efficiency Video Compression

In video coding, the in-loop filtering has emerged as a key module due to its significant improvement on compression performance since H.264/Advanced Video Coding. Existing incorporated in-loop filters in video coding standards mainly take advantage of the local smoothness prior model used for images. In this paper, we propose a novel adaptive loop filter utilizing image nonlocal prior knowledge by imposing the low-rank constraint on similar image patches for compression noise reduction. In the filtering process, the reconstructed frame is first divided into image patch groups according to image patch similarity. The proposed in-loop filtering is formulated as an optimization problem with low-rank constraint for every group of image patches independently. It can be efficiently solved by soft-thresholding singular values of the matrix composed of image patches in the same group. To adapt the properties of the input sequences and bit budget, an adaptive threshold derivation model is established for every group of image patches according to the characteristics of compressed image patches, quantization parameters, and coding modes. Moreover, frame-level and largest coding unit-level control flags are signaled to further improve the adaptability from the sense of rate-distortion optimization. The performance of the proposed in-loop filter is analyzed when it collaborates with the existing in-loop filters in High Efficiency Video Coding. Extensive experimental results show that our proposed in-loop filter can further improve the performance of state-of-the-art video coding standard significantly, with up to 16% bit-rate savings.
An adaptive and low-complexity all-zero block detection for HEVC encoder

To improve the detection accuracy of SAD or SATD based threshold and save the time cost of RDO determination, we proposed an All Zero Block (AZB) detection method by adaptively searching the maximum transform coefficient amplitude in low frequency of TU after conventional SATD detection was failed. The experimental results show that our algorithm can achieve around 39% transform and quantization time-saving with only 0.1% on average RD performance reduction. The detection accuracy of larger TU size, i.e. 16×16 and 32 × 32, can reach up to about 95% on average.
Image Denoising via Bandwise Adaptive Modeling and Regularization Exploiting Nonlocal Similarity.

This paper proposes a new image denoising algorithm based on adaptive signal modeling and regularization. It improves the quality of images by regularizing each image patch using bandwise distribution modeling in transform domain. Instead of using a global model for all the patches in an image, it employs content-dependent adaptive models to address the non-stationarity of image signals and also the diversity among different transform bands. The distribution model is adaptively estimated for each patch individually. It varies from one patch location to another and also varies for different bands. In particular, we consider the estimated distribution to have non-zero expectation. To estimate the expectation and variance parameters for every band of a particular patch, we exploit the nonlocal correlation in image to collect a set of highly similar patches as the data samples to form the distribution. Irrelevant patches are excluded so that such adaptively learned model is more accurate than a global one. The image is ultimately restored via bandwise adaptive soft-thresholding, based on a Laplacian approximation of the distribution of similar-patch group transform coefficients. Experimental results demonstrate that the proposed scheme outperforms several state-of-the-art denoising methods in both the objective and the perceptual qualities.
Exploiting Image Local and Nonlocal Consistency for Mixed Gaussian-Impulse Noise Removal

Most existing image denoising algorithms can only deal with a single type of noise, which violates the fact that the noisy observed images in practice are often suffered from more than one type of noise during the process of acquisition and transmission. In this paper, we propose a new variational algorithm for mixed Gaussian-impulse noise removal by exploiting image local consistency and nonlocal consistency simultaneously. Specifically, the local consistency is measured by a hyper-Lap lace prior, enforcing the local smoothness of images, while the nonlocal consistency is measured by three-dimensional sparsity of similar blocks, enforcing the nonlocal self-similarity of natural images. Moreover, a Split-Bregman based technique is developed to solve the above optimization problem efficiently. Extensive experiments for mixed Gaussian plus impulse noise show that significant performance improvements over the current state-of-the-art schemes have been achieved, which substantiates the effectiveness of the proposed algorithm.
Barbell-Lifting Based 3-D Wavelet Coding Scheme

This paper provides an overview of the Barbell lifting coding scheme that has been adopted as common software by the MPEG ad hoc group on further exploration of wavelet video coding. The core techniques used in this scheme, such as Barbell lifting, layered motion coding, 3D entropy coding and base layer embedding, are discussed. The paper also analyzes and compares the proposed scheme with the oncoming scalable video coding (SVC) standard because the hierarchical temporal prediction technique used in SVC has a close relationship with motion compensated temporal lifting (MCTF) in wavelet coding. The commonalities and differences between these two schemes are exhibited for readers to better understand modern scalable video coding technologies. Several challenges that still exist in scalable video coding, e.g., performance of spatial scalable coding and accurate MC lifting, are also discussed. Two new techniques are presented in this paper although they are not yet integrated into the common software. Finally, experimental results demonstrate the performance of the Barbell-lifting coding scheme and compare it with SVC and another well-known 3D wavelet coding scheme, MC embedded zero block coding (MC-EZBC).
Video Compressive Sensing Reconstruction via Reweighted Residual Sparsity

The compressive sensing (CS) theory indicates that robust reconstruction of signals can be obtained from far fewer measurements than those required by the Nyquist–Shannon theorem. Thus, CS has great potential in video acquisition and processing, considering that it makes the subsequent complex data compression unnecessary. In this paper, we propose a novel algorithm for effectively reconstructing videos from CS measurements. The algorithm comprises double phases, of which the first phase exploits intra-frame correlation and provides good initial recovery for each frame, and the second phase iteratively enhances reconstruction quality by alternating interframe multihypothesis (MH) prediction and sparsity modeling of residuals in a weighted manner. The weights of residual coefficients are updated in each iteration using a statistical method based on the MH predictions. These procedures are performed in the unit of overlapped patches such that potential blocking artifacts can be effectively suppressed through averaging. In addition, we devise an effective scheme based on the split Bregman iteration algorithm to solve the formulated weighted  ${\ell }_{1}$  minimization problem. The experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art methods in both objective and subjective reconstruction quality.
Nonlocal Gradient Sparsity Regularization for Image Restoration

Total variation (TV) regularization is widely used in image restoration to exploit the local smoothness of image content. Essentially, the TV model assumes a zero-mean Laplacian distribution for the gradient at all pixels. However, real-world images are nonstationary in general, and the zero-mean assumption of pixel gradient might be invalid, especially for regions with strong edges or rich textures. This paper introduces a nonlocal (NL) extension of TV regularization, which models the sparsity of the image gradient with pixelwise content-adaptive distributions, reflecting the nonstationary nature of image statistics. Taking advantage of the NL similarity of natural images, the proposed approach estimates the image gradient statistics at a particular pixel from a group of nonlocally searched patches, which are similar to the patch located at the current pixel. The gradient data in these NL similar patches are regarded as the samples of the gradient distribution to be learned. In this way, more accurate estimation of gradient is achieved. Experimental results demonstrate that the proposed method outperforms the conventional TV and several other anchors remarkably and produces better objective and subjective image qualities.
Hybrid All Zero Soft Quantized Block Detection for HEVC

Transform and quantization account for a considerable amount of computation time in video encoding process. However, there are a large number of discrete cosine transform coefficients which are finally quantized into zeros. In essence, blocks with all zero quantized coefficients do not transmit any information, but still occupy substantial unnecessary computational resources. As such, detecting all-zero block (AZB) before transform and quantization has been recognized to be an efficient approach to speed up the encoding process. Instead of considering the hard-decision quantization (HDQ) only, in this paper, we incorporate the properties of soft-decision quantization into the AZB detection. In particular, we categorize the AZB blocks into genuine AZBs (G-AZB) and pseudo AZBs (P-AZBs) to distinguish their origins. For G-AZBs directly generated from HDQ, the sum of absolute transformed difference-based approach is adopted for early termination. Regarding the classification of P-AZBs which are generated in the sense of rate-distortion optimization, the rate-distortion models established based on transform coefficients together with the adaptive searching of the maximum transform coefficient are jointly employed for the discrimination. Experimental results show that our algorithm can achieve up to 24.16% transform and quantization time-savings with less than 0.06% RD performance loss. The total encoder time saving is about 5.18% on average with the maximum value up to 9.12%. Moreover, the detection accuracy of larger TU sizes, such as  $16\times 16$  and  $32\times 32$  can reach to 95% on average.
Inter-frame Correlation Based Quantization Parameter Offset Optimization for Screen Content Video Coding

In this paper, we propose a novel quantization parameter offset optimization scheme for screen content videos based on the high correlation among adjacent frames. Firstly, we define a measurement of inter-frame correlation for successive frames. Then, according to the defined correlation measurement, quantization parameter offset for different frames is appropriately adjusted. Experimental results demonstrate that the proposed optimization scheme brings coding performance improvement. The maximum BD-rate gain can be over 3.8% and the average performance gain is over 2% compared with the reference software.
Intra prediction using fully connected network for video coding

Traditional intra prediction methods exploit some fixed rules to generate prediction, which might not be adaptive enough to handle complicated contents. In this paper, we investigate applying deep neural network to improve the state-of-the-art intra prediction. Considering the characteristics of block-based video coding framework, we propose a fully connected network for intra prediction where all layers except non-linear ones are fully connected. In the proposed network, the inputs are multiple reference lines of the current block and the output is the prediction for the block. When compared with the traditional intra prediction method, the richer context of current block is exploited. For this reason, the proposed network is capable of providing more accurate prediction. Experimental results demonstrate the effectiveness of proposed network. When integrated into the HEVC reference software, the proposed method can achieve up to 3.3% bitrate saving and an average of 1.6% bitrate saving for 4K sequences.
Wireless image and video soft transmission via perception-inspired power distortion optimization

Recently, a scheme called SoftCast has shown great potential for wireless image/video communication in the scenarios where the channel quality may fluctuate drastically and unpredictably. The transmission is lossy in nature, with its transmission power allocated among coefficients unequally to minimize the distortion. One problem is that its performance is optimized using mean square errors (MSE) as the quality metric, which is known for not matching the perception of human eyes. Inspired by the researches in image quality assessment, this paper proposes a power allocation and optimization scheme that minimizes the perceptual distortion of reconstruction image. In particular, we establish a perception model to evaluate the perceptual importance of different transform coefficients, based on the structure similarity (SSIM) image quality metric. Experimental results show that the proposed scheme can improve the perceptual performance of the original SoftCast scheme.
Video Compression Artifact Reduction via Spatio-Temporal Multi-Hypothesis Prediction

Annoying compression artifacts exist in most of lossy coded videos at low bit rates, which are caused by coarse quantization of transform coefficients or motion compensation from distorted frames. In this paper, we propose a compression artifact reduction approach that utilizes both the spatial and the temporal correlation to form multi-hypothesis predictions from spatio-temporal similar blocks. For each transform block, three predictions with their reliabilities are estimated, respectively. The first prediction is constructed by inversely quantizing transform coefficients directly, and its reliability is determined by the variance of quantization noise. The second prediction is derived by representing each transform block with a temporal auto-regressive (TAR) model along its motion trajectory, and its corresponding reliability is estimated from local prediction errors of the TAR model. The last prediction infers the original coefficients from similar blocks in non-local regions, and its reliability is estimated based on the distribution of coefficients in these similar blocks. Finally, all the predictions are adaptively fused according to their reliabilities to restore high-quality videos. The experimental results show that the proposed method can efficiently reduce most of the compression artifacts and improve both subjective and objective quality of block transform coded videos.
In-Scale Motion Compensation for Spatially Scalable Video Coding

In existing pyramid-based spatially scalable coding schemes, such as H.264/MPEG-4 SVC (scalable video coding), video frame at a certain high-resolution layer is mainly predicted either from the same frame at the next lower resolution layer, or from the temporal neighboring frames within the same resolution layer. But these schemes fail to exploit both kinds of correlation simultaneously and therefore cannot remove the redundancies among resolution layers efficiently. This paper extends the idea of spatiotemporal subband transform and proposes a general in-scale motion compensation technique for pyramid-based spatially scalable video coding. Video frame at each high-resolution layer is partitioned into two parts in frequency. Prediction for the lowpass part is derived from the next lower resolution layer, whereas prediction for the highpass part is obtained from neighboring frames within the same resolution layer, to further utilize temporal correlation. In this way, both kinds of correlation are exploited simultaneously and the cross-resolution-layer redundancy can be highly removed. Furthermore, this paper also proposes a macroblock-based adaptive in-scale technique for hybrid spatial and SNR scalability. Experimental results show that the proposed techniques can significantly improve the spatial scalability performance of H.264/MPEG-4 SVC, especially when the bit-rate ratio of lower resolution bit stream to higher resolution bit stream is considerable.