nils-herrmann/clean_scirepeval huggingface dataset:
Reduced-Reference Image Quality Assessment With Visual Information Fidelity

Reduced-reference (RR) image quality assessment (IQA) aims to use less data about the reference image and achieve higher evaluation accuracy. Recent research on brain theory suggests that the human visual system (HVS) actively predicts the primary visual information and tries to avoid the residual uncertainty for image perception and understanding. Therefore, the perceptual quality relies to the information fidelities of the primary visual information and the residual uncertainty. In this paper, we propose a novel RR IQA index based on visual information fidelity. We advocate that distortions on the primary visual information mainly disturb image understanding, and distortions on the residual uncertainty mainly change the comfort of perception. We separately compute the quantities of the primary visual information and the residual uncertainty of an image. Then the fidelities of the two types of information are separately evaluated for quality assessment. Experimental results demonstrate that the proposed index uses few data (30 bits) and achieves high consistency with human perception.
Dense depth acquisition via one-shot stripe structured light

Depth acquisition for moving objects becomes increasingly critical for some applications such as human facial expression recognition. This paper presents a method for capturing the depth maps of moving objects that uses a one-shot black-and-white stripe pattern with the features of simplicity and easily generation. Considering the accuracy of a matching is crucial for a precise depth map but the matching of variant-width stripes is sparse and rough, the phase differences extracted by Gabor filter to achieve a pixel-wise matching with sub-pixel accuracy are used. The details of the derivation are presented to prove that this method based on the phase difference calculated by Gabor filter is valid. In addition, the periodic ambiguity of the encoded stripe is eliminated by the epipolar segment covering a given depth range at a camera-projector calibrating stage to decrease the calculation complexity. Experimental results show that our method can get a dense and accurate depth map of a moving object.
A robust image encryption scheme over wireless channels

In traditional image encryption system, decryption is extremely sensitive to packet loss. However, in wireless networks, packet loss is inevitable. Compressed sensing (CS) theory shows that sparse signal can be recovered from few incomplete measurements of it. Strong randomness of measurement matrix and irrelevance among the elements of the measurement vector imply that measurement process can be regarded as encryption process. So, this paper, based on CS theory, presents a new image encryption scheme with robustness to packet loss. In the scheme, we design a Gaussian random measurement matrix as the key to realize data encryption. Moreover, to enhance the incoherence between the plain-image and the cipher-image, we add a random disturbance term to the measurements (cipher-image) and thus improve the security level of the cipher-image. Numerical experiments show that the proposed method not only has well anti-attack ability but also is robust to packet loss, which can still decrypt plain-image even when the packet loss ratio is up to 50%.
Variable Block-Sized Signal-Dependent Transform for Video Coding

Transform, as one of the most important modules of mainstream video coding systems, seems very stable over the past several decades. However, recent developments indicate that bringing more options for transform can lead to coding efficiency benefits. In this paper, we go further to investigate how the coding efficiency can be improved over the state-of-the-art method by adapting a transform for each block. We present a variable block-sized signal-dependent transforms (SDTs) design based on the High Efficiency Video Coding (HEVC) framework. For a coding block ranged from <inline-formula> <tex-math notation="LaTeX">$4\times4$ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$32\times32$ </tex-math></inline-formula>, we collect a quantity of similar blocks from the reconstructed area and use them to derive the Karhunen–Loève transform. We avoid sending overhead bits to denote the transform by performing the same procedure at the decoder. In this way, the transform for every block is tailored according to its statistics, to be signal-dependent. To make the large block-sized SDTs feasible, we present a fast algorithm for transform derivation. Experimental results show the effectiveness of the SDTs for different block sizes, which leads to up to 23.3% bit-saving. On average, we achieve BD-rate saving of 2.2%, 2.4%, 3.3%, and 7.1% under AI-Main10, RA-Main10, RA-Main10, and LP-Main10 configurations, respectively, compared with the test model HM-12 of HEVC. The proposed scheme has also been adopted into the joint exploration test model for the exploration of potential future video coding standard.
Parallel Implementation of the Range-Doppler Radar Processing on a GPU Architecture

Graphic processing units (GPUs) is widely used to accelerate the processing speed of the radar detection procedure, including the range compression, coherent integration and constant false alarm rate. Specifically, detailed parallel design of the radar algorithm and the thread programming are shown. The experimental results show that, by engaging the parallel technology into the radar processing procedure, much high speedup ratio can be obtained. Furthermore, precise target detection can be guaranteed.
Perceptual Quality Metric With Internal Generative Mechanism

Objective image quality assessment (IQA) aims to evaluate image quality consistently with human perception. Most of the existing perceptual IQA metrics cannot accurately represent the degradations from different types of distortion, e.g., existing structural similarity metrics perform well on content-dependent distortions while not as well as peak signal-to-noise ratio (PSNR) on content-independent distortions. In this paper, we integrate the merits of the existing IQA metrics with the guide of the recently revealed internal generative mechanism (IGM). The IGM indicates that the human visual system actively predicts sensory information and tries to avoid residual uncertainty for image perception and understanding. Inspired by the IGM theory, we adopt an autoregressive prediction algorithm to decompose an input scene into two portions, the predicted portion with the predicted visual content and the disorderly portion with the residual content. Distortions on the predicted portion degrade the primary visual information, and structural similarity procedures are employed to measure its degradation; distortions on the disorderly portion mainly change the uncertain information and the PNSR is employed for it. Finally, according to the noise energy deployment on the two portions, we combine the two evaluation results to acquire the overall quality score. Experimental results on six publicly available databases demonstrate that the proposed metric is comparable with the state-of-the-art quality metrics.
Blind Quality Index for Multiply Distorted Images Using Biorder Structure Degradation and Nonlocal Statistics

In the past decade, extensive image quality metrics have been proposed. The majority of them are tailored for the images that contain a specific type of distortion. However, in practice, the images are usually degraded by different types of distortions simultaneously. This poses great challenges to the existing quality metrics. Motivated by this, this paper proposes a no-reference quality index for the multiply distorted images using the biorder structure degradation and the nonlocal statistics. The design philosophy is inspired by the fact that the human visual system (HVS) is highly sensitive to the degradations of both the spatial contrast and the spatial distribution, which are prone to be changed by the joint effects of the multiple distortions. Specifically, the multiresolution representation of the image is first built by downsampling to simulate the hierarchical property of the HVS. Then, the structure degradation is calculated to measure the spatial contrast. Considering the fact that the human visual cortex has the separate mechanisms to perceive the first- and second-order structures, dubbed biorder structures, the degradations of biorder structures are calculated to account for the spatial contrast, producing the first group of the quality-aware features. Furthermore, the nonlocal self-similarity statistics is calculated to measure the spatial distribution, producing the second group of features. Finally, all the features are fed into the random forest regression model to learn the quality model for the multiply distorted images. Extensive experimental results conducted on the three public databases demonstrate the superiority of the proposed metric to the state-of-the-art metrics. Moreover, the proposed metric is also advantageous over the existing metrics in terms of the generalization ability.
Efficient matrix inversion based on VLIW architecture

Matrix inversion is a critical part in communication,signal processing and electromagnetic system. A flexible and scalable very long instruction word(VLIW) processor with clustered architecture is proposed for matrix inversion. A global register file(RF) is used to connect all the clusters. Two nearby clusters share a local register file. The instruction sets are also designed for the VLIW processor. Experimental results show that the proposed VLIW architecture takes only 45 latency to invert a 4 × 4 matrix when running at 150 MHz. The proposed design is roughly five times faster than the DSP solution in processing speed.
Color demosaicking with sparse representations

Color demosaicking is an ill-posed inverse problem of image restoration. The performance of a color demosaicking algorithm depends on how thoroughly it can exploit domain knowledge to confine the solution space for the underlying true color image. We propose a sparsity-based ℓ1 minimization technique for color demosaicking that exploits both interband and intra-band sparse representations of natural images. In some of most challenging cases of color demosaicking, the proposed technique outperforms those published in the literature by a significant margin in both PSNR and visual quality.
Gradient sparsity for piecewise continuous optical flow estimation

This paper introduces a new sparse model for robust and reliable optical flow estimation. We show that the sparsity is directly related to gradient fields of optical flow. According to theory on sparse signal recovery, we rigorously formulate the optical flow estimation as an ℓ0 optimization problem. Considering the piecewise continuous nature of motion, the basic optical flow constraint is regularized by an ℓ1 norm. Then, with convex relaxation, the solution is obtain via the ordinary or reweighted ℓ1 optimization approach. Experimental results show the proposed method performs better than traditional methods and deals well with the discontinuities on motion boundaries.
Pattern Masking Estimation in Image With Structural Uncertainty

A model of visual masking, which reveals the visibility of stimuli in the human visual system (HVS), is useful in perceptual based image/video processing. The existing visual masking function mainly considers luminance contrast, which always overestimates the visibility threshold of the edge region and underestimates that of the texture region. Recent research on visual perception indicates that the HVS is sensitive to orderly regions that possess regular structures and insensitive to disorderly regions that possess uncertain structures. Therefore, structural uncertainty is another determining factor on visual masking. In this paper, we introduce a novel pattern masking function based on both luminance contrast and structural uncertainty. Through mimicking the internal generative mechanism of the HVS, a prediction model is firstly employed to separate out the unpredictable uncertainty from an input image. In addition, an improved local binary pattern is introduced to compute the structural uncertainty. Finally, combining luminance contrast with structural uncertainty, the pattern masking function is deduced. Experimental result demonstrates that the proposed pattern masking function outperforms the existing visual masking function. Furthermore, we extend the pattern masking function to just noticeable difference (JND) estimation and introduce a novel pixel domain JND model. Subjective viewing test confirms that the proposed JND model is more consistent with the HVS than the existing JND models.
Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach

Simultaneous sparse coding (SSC) or nonlocal image representation has shown great potential in various low-level vision tasks, leading to several state-of-the-art image restoration techniques, including BM3D and LSSC. However, it still lacks a physically plausible explanation about why SSC is a better model than conventional sparse coding for the class of natural images. Meanwhile, the problem of sparsity optimization, especially when tangled with dictionary learning, is computationally difficult to solve. In this paper, we take a low-rank approach toward SSC and provide a conceptually simple interpretation from a bilateral variance estimation perspective, namely that singular-value decomposition of similar packed patches can be viewed as pooling both local and nonlocal information for estimating signal variances. Such perspective inspires us to develop a new class of image restoration algorithms called spatially adaptive iterative singular-value thresholding (SAIST). For noise data, SAIST generalizes the celebrated BayesShrink from local to nonlocal models; for incomplete data, SAIST extends previous deterministic annealing-based solution to sparsity optimization through incorporating the idea of dictionary learning. In addition to conceptual simplicity and computational efficiency, SAIST has achieved highly competent (often better) objective performance compared to several state-of-the-art methods in image denoising and completion experiments. Our subjective quality results compare favorably with those obtained by existing techniques, especially at high noise levels and with a large amount of missing data.
HEVC Encoding Optimization Using Multicore CPUs and GPUs

Although the High Efficiency Video Coding (HEVC) standard significantly improves the coding efficiency of video compression, it is unacceptable even in offline applications to spend several hours compressing 10 s of high-definition video. In this paper, we propose using a multicore central processing unit (CPU) and an off-the-shelf graphics processing unit (GPU) with 3072 streaming processors (SPs) for HEVC fast encoding, so that the speed optimization does not result in loss of coding efficiency. There are two key technical contributions in this paper. First, we propose an algorithm that is both parallel and fast for the GPU, which can utilize 3072 SPs in parallel to estimate the motion vector (MV) of every prediction unit (PU) in every combination of the coding unit (CU) and PU partitions. Furthermore, the proposed GPU algorithm can avoid coding efficiency loss caused by the lack of a MV predictor (MVP). Second, we propose a fast algorithm for the CPU, which can fully utilize the results from the GPU to significantly reduce the number of possible CU and PU partitions without any coding efficiency loss. Our experimental results show that compared with the reference software, we can encode high-resolution video that consumes 1.9% of the CPU time and 1.0% of the GPU time, with only a 1.4% rate increase.
Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling

Sparse representation is proven to be a promising approach to image super-resolution, where the low-resolution (LR) image is usually modeled as the down-sampled version of its high-resolution (HR) counterpart after blurring. When the blurring kernel is the Dirac delta function, i.e., the LR image is directly down-sampled from its HR counterpart without blurring, the super-resolution problem becomes an image interpolation problem. In such cases, however, the conventional sparse representation models (SRM) become less effective, because the data fidelity term fails to constrain the image local structures. In natural images, fortunately, many nonlocal similar patches to a given patch could provide nonlocal constraint to the local structure. In this paper, we incorporate the image nonlocal self-similarity into SRM for image interpolation. More specifically, a nonlocal autoregressive model (NARM) is proposed and taken as the data fidelity term in SRM. We show that the NARM-induced sampling matrix is less coherent with the representation dictionary, and consequently makes SRM more effective for image interpolation. Our extensive experimental results demonstrate that the proposed NARM-based image interpolation method can effectively reconstruct the edge structures and suppress the jaggy/ringing artifacts, achieving the best image interpolation results so far in terms of PSNR as well as perceptual quality metrics such as SSIM and FSIM.
Image compression with downsampling and overlapped transform at low bit rates

This paper proposes an image coding method based on adaptive downsampling which not only uses the pixel redundancy but also considers visual redundancy. At the encoder side, codec adaptively chooses some smooth regions of the original image to downsample, and then overlapped transform with selectivity, block DCT and adaptive-shape DCT (SA-DCT) are used against the image after being downsampled. For the incomplete transformed image, OB-SPECK is adopted to code. At the decoder side, in order to reduce the computational complexity, we use the simple cubic interpolation which not only is very suitable to the downsampled regions but also enhances greatly the real time of this coding system. Experimental results shows the proposed method outperforms JPEG2000, SPECK, SPIHT, and LT+SPECK at low bit rates.
SAR image despeckling with adaptive sparse representation

SR-based denoising methods have shown promising performance in image denoising. However, Because of the degradation of the noisy image, conventional SR based denoising models may not be accurate enough for the reconstruction of a clean image. Therefore, to reduce the noise corruption, a novel adaptive sparse representation based SAR image despeckling algorithm is proposed in this paper, where the noise component is considered as the coefficient residual, which equals to the difference between the actual image coefficient and the estimated coefficient. By imposing the sparsity constraint on this residual, the noise corruption can be somehow reduced. Furthermore, both the autoregressive model and the nonlocal similarity are incorporated to characterize better the image details. The experimental results demonstrate that the proposed algorithm outperforms other algorithms both subjectively and objectively.
A simple design of M-channel FIR wavelet-like filter bank

In this paper, we propose a simple method for designing M-channel FIR wavelet-like filter banks. Unlike the conventional design of wavelet filter banks, our approach is based on near-perfect-reconstruction (NPR) filter banks, justifying the name "wavelet-like." The major contribution of this paper lies in the technique to impose regularity constraints individually into each filter of the NPR filter bank, yielding more free parameters to design wavelet-like filter banks. Moreover, our method involves no optimization procedure by imposing regularity constraints. The designed 1-regular wavelet-like filter bank possesses good properties as the original NPR filter bank. The 2-regular wavelet-like filter bank can also be obtained preserving the NPR property. Examples are given to numerically illustrate the proposed technique.
Color demosaicking with the spatial alignment property of spectral Laplacians

We present a color demosaicking method which aims to improve the demosaicking performance by thoroughly exploiting domain knowledge to confine the solution space for the underlying true color image. We use a physically-induced model to infer spatial alignment property of spectral Laplacians. Specifically, the 2D Laplacians of different channels are sparse, and it benefits spatial alignment property across different color channels. Then, we establish an optimization model, using the image formation model and the spatial alignment property of spectral Laplacians, to solve the ill-posed problem of the color demosaicking. Finally, some extensive experiments have been done and show the availability of our approach in color image demosaicking.
Exploiting Non-Local Correlation via Signal-Dependent Transform (SDT)

Over the past few decades, many studies on image and video compression have found various approaches to the exploitation of spatial and temporal local correlations. However, we believe it is imperative to find more efficient methods to progress the development of image and video compression. In this paper, we first study spatial non-local correlation, deducing that there exist strong correlations in non-local regions. However, it is rather difficult to make use of these non-local correlations while simultaneously minimizing overhead. To solve this problem, we propose the signal-dependent transform (SDT), which is derived from decoded non-local blocks that are selected by matching neighboring pixels. Since the encoder and decoder can use the same methods to derive the proposed transform, we can successfully eliminate overhead. Finally, we have implemented the proposed transform into the Key Technology Area (KTA) software to exploit both spatial and temporal non-local correlations. The experimental results show that the coding gain over KTA can be as high as 1.4 dB in intra-frame coding, and up to 1.0 dB in inter-frame coding. We believe we have effectively created an alternate method to improve image and video compression.
Lossy to lossless image compression based on reversible integer DCT

A progressive image compression scheme is investigated using reversible integer discrete cosine transform (RDCT) which is derived from the matrix factorization theory. Previous techniques based on DCT suffer from bad performance in lossy image compression compared with wavelet image codec. And lossless compression methods such as IntDCT, I2I-DCT and so on could not compare with JPEG-LS or integer discrete wavelet transform (DWT) based codec. In this paper, lossy to lossless image compression can be implemented by our proposed scheme which consists of RDCT, coefficients reorganization, bit plane encoding, and reversible integer pre- and post-filters. Simulation results show that our method is competitive against JPEG-LS and JPEG2000 in lossless compression. Moreover, our method outperforms JPEG2000 (reversible 5/3 filter) for lossy compression, and the performance is even comparable with JPEG2000 which adopted irreversible 9/7 floating-point filter (9/7F filter).