nils-herrmann/clean_scirepeval huggingface dataset:
Power-distortion optimization for wireless image/video SoftCast by transform coefficients energy modeling with adaptive chunk division

Traditional communication systems usually suffer from the threshold effect when channel signal-to-noise ratio (CSNR) fluctuates unpredictably in wireless and mobile scenarios. The SoftCast scheme, however, provides graceful quality transition in wide CSNR range. In SoftCast, input image is decorrelated by a transform and modulated directly to a dense constellation for transmission, leaving out the conventional quantization, entropy coding and channel coding. A key point of SoftCast is that the transmission power needs to be allocated among the transform coefficients unequally, according to the energy of coefficients. Importantly, the energy diversity used to guide power allocation should be shared between the sender and the receiver for correct decoding. This paper addresses the power distortion optimization problem, introducing a new adaptive chunk division scheme to describe the energy diversity among coefficients. A concrete algorithm is developed to determine the chunk boundaries that achieve optimal transmission power usage. Experimental results show that the proposed scheme can improve the performance of the original SoftCast by 4~8dB using a smaller number of chunks.
Distributed soft video broadcast with variable block size motion estimation

In recent years, video broadcast has become a popular application, but the traditional hierarchical design requires the source to pick a bitrate and video resolution for encoding before transmission, it cannot be efficient to accommodate users with different channel quality. The proposed DCAST scheme can solve this problem. DCAST uses the ME and MC technology to generate a predicted frame which helps current frame to do coset coding. However, in the ME process, DCAST uses fixed block size. In this paper, we use the variable block size motion estimation to replace the fixed block size motion estimation, it can effectively reduce the block effect and improve the quality of the reconstructed frame generated by MC and predicted frame. The DCAST with variable block size motion estimation is 0.5dB better than DCAST with fixed block size motion estimation.
Fully Connected Network-Based Intra Prediction for Image Coding

This paper proposes a deep learning method for intra prediction. Different from traditional methods utilizing some fixed rules, we propose using a fully connected network to learn an end-to-end mapping from neighboring reconstructed pixels to the current block. In the proposed method, the network is fed by multiple reference lines. Compared with traditional single line-based methods, more contextual information of the current block is utilized. For this reason, the proposed network has the potential to generate better prediction. In addition, the proposed network has good generalization ability on different bitrate settings. The model trained from a specified bitrate setting also works well on other bitrate settings. Experimental results demonstrate the effectiveness of the proposed method. When compared with high efficiency video coding reference software HM-16.9, our network can achieve an average of 3.4% bitrate saving. In particular, the average result of 4K sequences is 4.5% bitrate saving, where the maximum one is 7.4%.
Efficient Multiple-Line-Based Intra Prediction for HEVC

Traditional intra prediction usually utilizes the nearest reference line to generate the predicted block when considering strong spatial correlation. However, this kind of single-line-based method does not always work well due to at least two issues. One is the incoherence caused by the signal noise or the texture of other objects, where this texture deviates from the inherent texture of the current block. The other reason is that the nearest reference line usually has worse reconstruction quality in block-based video coding. Due to these two issues, this paper proposes an efficient multiple-line-based intra-prediction scheme to improve coding efficiency. Besides the nearest reference line, further reference lines are also utilized. The further reference lines with a relatively higher quality can provide potentially better prediction. At the same time, the residue compensation is introduced to calibrate the prediction of boundary regions in a block when we utilize further reference lines. To speed up the encoding process, this paper designs several fast algorithms. The experimental results show that compared with HM-16.9, the proposed fast search method achieves a 2.0% bit saving on average and up to 3.7% by increasing the encoding time by 112%.
Analysis of Decorrelation Transform Gain for Uncoded Wireless Image and Video Communication.

An uncoded transmission scheme called SoftCast has recently shown great potential for wireless video transmission. Unlike conventional approaches, SoftCast processes input images only by a series of transformations and modulates the coefficients directly to a dense constellation for transmission. The transmission is uncoded and lossy in nature, with its noise level commensurate with the channel condition. This paper presents a theoretical analysis for an uncoded visual communication, focusing on developing a quantitative measurements for the efficiency of decorrelation transform in a generalized uncoded transmission framework. Our analysis reveals that the energy distribution among signal elements is critical for the efficiency of uncoded transmission. A decorrelation transform can potentially bring a significant performance gain by boosting the energy diversity in signal representation. Numerical results on Markov random process and real image and video signals are reported to evaluate the performance gain of using different transforms in uncoded transmission. The analysis presented in this paper is verified by simulated SoftCast transmissions. This provide guidelines for designing efficient uncoded video transmission schemes.
Low-Rank Decomposition-Based Restoration of Compressed Images via Adaptive Noise Estimation.

Images coded at low bit rates in real-world applications usually suffer from significant compression noise, which significantly degrades the visual quality. Traditional denoising methods are not suitable for the content-dependent compression noise, which usually assume that noise is independent and with identical distribution. In this paper, we propose a unified framework of content-adaptive estimation and reduction for compression noise via low-rank decomposition of similar image patches. We first formulate the framework of compression noise reduction based upon low-rank decomposition. Compression noises are removed by soft thresholding the singular values in singular value decomposition of every group of similar image patches. For each group of similar patches, the thresholds are adaptively determined according to compression noise levels and singular values. We analyze the relationship of image statistical characteristics in spatial and transform domains, and estimate compression noise level for every group of similar patches from statistics in both domains jointly with quantization steps. Finally, quantization constraint is applied to estimated images to avoid over-smoothing. Extensive experimental results show that the proposed method not only improves the quality of compressed images obviously for post-processing, but are also helpful for computer vision tasks as a pre-processing method.
Compressed Sensing Recovery via Collaborative Sparsity

Compressed Sensing (CS) has drawn quite an amount of attention as a joint sampling and compression approach. Its theory shows that a signal can be decoded from many fewer measurements than suggested by the Nyquist sampling theory, when the signal is sparse in some domain. So one of the most significant challenges in CS is to seek a domain where a signal can exhibit a high degree of sparsity and hence be recovered faithfully. Most of conventional CS recovery approaches, however, exploited a set of fixed bases (e.g. DCT, wavelet and gradient domain) for the entirety of a signal, which are irrespective of the nonstationarity of natural signals and cannot achieve high enough degree of sparsity, thus resulting in poor rate-distortion performance. In this paper, we propose a new framework for compressed sensing recovery via collaborative sparsity (RCoS), which enforces local two-dimensional sparsity and nonlocal three-dimensional sparsity simultaneously in an adaptive hybrid space-transform domain, thus substantially utilizing intrinsic sparsities of natural images and greatly confining the CS solution space. In addition, an efficient augmented Lagrangian based technique is developed to solve the above optimization problem. Experimental results on a wide range of natural images are presented to demonstrate the efficacy of the new CS recovery strategy.
Towards accurate visual information estimation with Entropy of Primitive

Recently, a novel concept referred to as Entropy of Primitive (EoP) has been proposed for evaluating the visual information of natural images. The idea originates from the sparse representation, which has been successfully applied in a wide variety of signal processing and analysis tasks. This is because of the high efficiency of sparse representation in dealing with rich, varied and directional information contained in the natural scene. In this paper, we further explore the EoP to bridge the sparse representation and visual perception. Sparse primitives are divided into three categories depending on their visual importance. Accordingly the visual signal is decomposed into structural and non-structural layers. It is found that the image sparse representation is highly relevant with the hierarchical visual information construction process in representing the natural scene. We evaluate the efficiency and robustness of the EoP in real applications, including surveillance video and shot boundary detection.
Reducing Blocking Artifacts in Compressed Images via Transform-Domain Non-local Coefficients Estimation

Block transform coding using discrete cosine transform is the most popular approach for image compression. However, many annoying blocking artifacts are generated due to coarse quantization on transform coefficients independently. This paper proposes an effective blocking artifacts reduction method by estimating the transform coefficients from their quantized version. In the proposed scheme, we estimate the transform coefficients based on an image statistic model and non-local similarity among blocks in transform domain. The parameters used in our proposed scheme are discussed and adaptively selected. Extensive experimental results show that our proposed method significantly reduces blocking artifacts and improves the subjective and the objective quality of block transform coded images.
Transductive Regression with Local and Global Consistency for Image Super-Resolution

In this paper, we propose a novel image super-resolution algorithm, referred to as interpolation based on transductive regression with local and global consistency (TRLGC). Our algorithm first constructs a set of local interpolation models which can predict the intensity labels of all image samples, and a loss term will be minimized to keep the predicted labels of available low-resolution (LR) samples sufficiently close to the original ones. Then, all of the losses evaluated in local neighborhoods are accumulated together to measure the global consistency on all samples. Furthermore, a graph-Laplacian based manifold regularization term is incorporated to penalize the global smoothness of intensity labels, such smoothing can alleviate the insufficient training of the local models and make them more robust. Finally, we construct a unified objective function to combine together the accumulated loss of the locally linear regression, square error of prediction bias on the available LR samples and the manifold regularization term, which could be solved with a closed-form solution as a convex optimization problem. In this way, a transductive regression algorithm with local and global consistency is developed. Experimental results on benchmark test images demonstrate that the proposed image super-resolution method achieves very competitive performance with the state-of-the-art algorithms.
Adaptive multi-dimension sparsity based coefficient estimation for compression artifact reduction

Sparsity has shown promising results in various image restoration applications. Recent advances have suggested that structured or group sparsity often leads to more powerful results in compression artifact reduction studies. In this paper, we introduce nonlocal multi-dimension sparsity in an adaptive space-transform domain, which performs multi-scale wavelet transform on DCT coefficients of similar patches. The new transform efficiently reduces image redundancies between inner block and inter block simultaneously, thus it can substantially achieve sparse representation for images. Furthermore, a band-based filter is proposed to reduce compression artifacts by shrinking transform coefficients adaptively. Because of the overlapped processing, adaptive aggregation is used to combine different estimates for each block. The proposed algorithm achieves improvement over some methods in terms of both objective and subjective qualities.
Multiple Hypotheses Bayesian Frame Rate Up-Conversion by Adaptive Fusion of Motion-Compensated Interpolations

Frame rate up-conversion (FRUC) improves the viewing experience of a video because the motion in a FRUC-constructed high frame-rate video looks more smooth and continuous. This paper proposes a multiple hypotheses Bayesian FRUC scheme for estimating the intermediate frame with maximum a posteriori probability, in which both temporal motion model and spatial image model are incorporated into the optimization criterion. The image model describes the spatial structure of neighboring pixels while the motion model describes the temporal correlation of pixels along motion trajectories. Instead of employing a single uniquely optimal motion, multiple “optimal” motion trajectories are utilized to form a group of motion hypotheses. To obtain accurate estimation for the pixels in missing intermediate frames, the motion-compensated interpolations generated by all these motion hypotheses are adaptively fused according to the reliability of each hypothesis. We revealed by numerical analysis that this reliability (i.e., the variance of interpolation errors along the hypothesized motion trajectory) can be measured by the variation of reference pixels along the motion trajectory. To obtain the multiple motion fields, a set of block-matching sizes is used and the motion fields are estimated by progressively reducing the size of matching block. Experimental results show that the proposed method can significantly improve both the objective and the subjective quality of the constructed high frame rate video.
Gradient based image/video softcast with grouped-patch collaborative reconstruction

Inspired by the recent image quality assessment (IQA) studies which indicate that the image gradient data reflects the visual information more reliably than the image pixels, gradient based transmission scheme was recently proposed to pursue better perceptual quality for wireless visual communication. This paper develops an effective method to reconstruct high quality image from the received noisy gradient data. The proposed method utilizes both local correlation and non-local similarity within the image signal to regularize the reconstruction image. Principle component analysis (PCA) is employed to learn signal-adaptive two-dimensional (2D) transform basis, and 3D transform is performed on grouped similar patches to further decorrelate the coefficients. In this way, distortions can be effectively suppressed via adaptive collaborative shrinkage on the transform coefficients. Experimental results demonstrate that the proposed method improves the reconstruction performance remarkably compared with the existing schemes.
Interpolation-Dependent Image Downsampling

Traditional methods for image downsampling commit to remove the aliasing artifacts. However, the influences on the quality of the image interpolated from the downsampled one are usually neglected. To tackle this problem, in this paper, we propose an interpolation-dependent image downsampling (IDID), where interpolation is hinged to downsampling. Given an interpolation method, the goal of IDID is to obtain a downsampled image that minimizes the sum of square errors between the input image and the one interpolated from the corresponding downsampled image. Utilizing a least squares algorithm, the solution of IDID is derived as the inverse operator of upsampling. We also devise a content-dependent IDID for the interpolation methods with varying interpolation coefficients. Numerous experimental results demonstrate the viability and efficiency of the proposed IDID.
High-quality image restoration from partial random samples in spatial domain

In this paper, a novel algorithm for high-quality image restoration is proposed. The contributions of this work are two-fold. First, a new form of minimization function for solving image inverse problems is formulated via combining local total variation model and nonlocal adaptive 3-D sparse representation model as regularizers under the regularization-based framework. Second, a new Split-Bregman based iterative algorithm is developed to solve the above optimization problem efficiently associated with proved theoretical convergence property. Experimental results on image restoration from partial random samples have shown that the proposed algorithm achieves significant performance improvements over the current state-of-the-art schemes and exhibits nice convergence property.
New image coding scheme with hierarchical representation and adaptive interpolation

In this paper, we propose a new image coding scheme which combines the advantage of hierarchical (i.e. multi-resolution) representation, adaptive interpolation and rate-distortion optimization capability of block-based coding. We divide the coding process into two layers by partitioning the image pixels into four groups with chessboard structure. In the first layer, pixels in one of these partitions are encoded and reconstructed using the state-of-the-art coding scheme. In the second layer, directional adaptive interpolation is employed to predict and code the pixels in the remaining three partitions. Joint mode decision is performed for the co-located macroblocks of these three partitions. Skip mode is introduced to save coding bits for blocks where the interpolation process works very well. Experimental results demonstrate that the proposed coding scheme achieves very promising performance.
Adaptive MCTF based on Correlation Noise Model for SNR Scalable Video Coding

This paper proposes a subband adaptive motion compensated temporal filtering (MCTF) technique for scalable video coding and introduces a revised synthesis gain model for the quantization in this adaptive MCTF scheme. In scalable video coding, hierarchical MCTF is extensively adopted to exploit the temporal correlation across video frames. In this hierarchical MCTF structure, the strength of temporal correlation varies with the level of temporal transform and varies with the various spatial frequency components in a frame. The reconstruction noises also have diverse strength at various subbands. According to the correlation and noise characteristics of various subbands, we can adjust the strength of motion compensated prediction step in MCTF to maximally take the advantage of temporal correlation but restrict the propagation of reconstruction noise. The quantization step of each subband is also adjusted according to synthesis gain determined by the MCTF structure. In this way an adaptive MCTF scheme is formed and the proposed technique improves the coding performance of scalable video coding
A dual structured-sparsity model for compressive-sensed video reconstruction

The compressive sensing theory indicates that robust reconstruction of signals can be obtained from far fewer measurements than those required by the Nyquist theorem. Thus, it has great potential in video acquisition and processing in that it can tremendously save the complex compression required by traditional video coding standards. In this paper, we consider reconstruction of compressive-sensed videos and propose a novel structured-sparsity model with a dual prediction strategy. This structured-sparsity model goes beyond simple sparsity and characterizes the intrinsic structure within the transform coefficients. Also, it exploits the sparsity of the residual between the current patch and its prediction. The prediction process is comprised of a dual strategy, which integrates the advantages of the ambient pixel domain and the measurement domain. In addition, an effective optimization method is designed for solving the formulated problem derived from the model. Experiments demonstrate that the proposed algorithm outperforms the state-of-the art methods for compressive-sensed video reconstruction in both subjective and objective quality.
Image denoising via adaptive soft-thresholding based on non-local samples

This paper proposes a new image denoising approach using adaptive signal modeling and adaptive soft-thresholding. It improves the image quality by regularizing all the patches in image based on distribution modeling in transform domain. Instead of using a global model for all patches, it employs content adaptive models to address the non-stationarity of image signals. The distribution model of each patch is estimated individually and can vary for different transform bands and for different patch locations. In particular, we allow the distribution model for each individual patch to have non-zero expectation. To estimate the expectation and variance parameters for the transform bands of a particular patch, we exploit the non-local correlation of image and collect a set of similar patches as data samples to form the distribution. Irrelevant patches are excluded so that this non-local based modeling is more accurate than global modeling. Adaptive soft-thresholding is employed since we observed that the distribution of non-local samples can be approximated by Laplacian distribution. Experimental results show that the proposed scheme outperforms the state-of-the-art denoising methods such as BM3D and CSR in both the PSNR and the perceptual quality.
Compressive gradient based scalable image SoftCast

In wireless visual communication systems, it is crucial to effectively utilize channel power and bandwidth in the pursue of optimal performance, and it is worthwhile to adapt the transmission scheme to human vision system (HVS) so as to achieve perceptually appealing results. Inspired by observations that visual quality of an image is closely related to the gradient data, this paper proposes to convey visual information by random projection measurements of image gradients in an analog framework. Since HVS is more sensitive to luminance variations of image contents, which are contained in the gradient data, the proposed scheme achieves better perceptual quality than conventional analog uncoded schemes like SoftCast. Besides, the gradient transform removes the low and medium frequency components of the image hence substantially reduces the power of the signal transmitted in the analog channel, thus evidently improves the power-distortion performance of the system. Furthermore, by applying random projection to the gradients, the number of transmitted data can be adjusted according to bandwidth conditions. Another contribution of this paper is to develop an effective optimization scheme for the compressive gradient based reconstruction problem. Experimental results validate the effectiveness of the proposed transmission and reconstruction scheme under different channel signal-to-noise ratio and bandwidth conditions.