nils-herrmann/clean_scirepeval huggingface dataset:
Motion vector predictor selection for the enhancement layer in the H.264/AVC extension-spatial SVC

Scalable Video Coding (SVC) has been approved as the extension of the H.264/AVC video coding standard recently . In current spatial scalability scheme, the technique to examine both inter modes with residual prediction and without residual prediction for enhancement layers can achieve the highest possible coding efficiency, but it's typically one of the most time-consuming parts of a video encoder. In this paper, we propose a method to skip half of motion estimation processes under “inter modes without residual prediction” based on the motion vector predictor selection under “inter modes with residual prediction”. Experimental results show that our proposed scheme can achieve up to 24% time saving for motion estimation in the enhancement layer and meanwhile the coding efficiency can be preserved very well.
Sketch-Guided Texture-Based Image Inpainting

In this paper, we propose a novel framework for image inpainting, named sketch-guided texture-based image inpainting. Inspired by the well-known primal sketch model, we present a penetrating perspective into the process of image formation, where each image is seen as a variety of texture organized by some underlying structure. Based on this conceptual foundation, our approach of image inpainting integrates two unified stages: it first reconstructs the image structure with the sketch model, and then guided by the structure, it restores the missing region by patch-based texture synthesis. The major superiority of the framework over other ones consists in its capability of simultaneously recovering the structure and texture in the missing regions. Comprehensive experiments are performed to compare our method with other state-of-the-art ones; the encouraging results obtained convincingly demonstrate the effectiveness of our method.
Sub-optimal quarter-pixel inter-prediction algorithm (SQIA)

Motion estimation (ME) is an important part of modern video coding systems to exploit temporary redundancy in a video. Motion estimation is typically per-formed firstly with integer-pixel accuracy and then at sub-pixel accuracy, which includes half-pixel and quarter-pixel accuracy. When sophisticated fast integer-pixel accuracy motion-search algorithms are used to decrease the number of search points for integer-pixel motion search, quarter-pixel motion search becomes another important processing bottleneck in the encoding process. The conventional method is to search 8 half-pixel positions around the motion vector (MV) obtained from integer-pixel motion search, then do motion search in the same way on 8 quarter-pixel positions around the MV obtained from the half-pixel motion search, therefore, in total, 16 search points are needed. The proposed algorithm, named sub-optimal quarter-pixel inter-prediction algorithm (SQIA), successfully optimizes the quarter-pixel motion search part and improves the processing speed with low PSNR penalty.
Optimal dependent bit allocation for AVS intra-frame coding via successive convex approximation

We consider the optimal dependent bit allocation strategy for AVS intra-frame coding. Due to the block-based predictive coding, the rate-distortion (R-D) characteristics of neighboring blocks are dependent with each other. However, the interblock coding dependency is neglected in most of the existing bit allocation methods. Different from the conventional methods, the proposed method fully exploit the interblock coding dependency and carefully leverage it in the problem formulation. Then successive convex optimization techniques are employed to convert the original nonconvex optimization problem into a series of convex optimization problems which can be solved efficiently and optimally. Experimental results have proved the superiority of the proposed method in terms of significant R-D performance improvement.
New Digital Watermarking for Few-Color Images

In this paper, we propose a new digital watermarking algorithm named spatial unified key insertion (SUKI) for few-color images. By performing the adaptive threshold halftoning and contour shaping and modification, we can embed a binary logo into a digital image with superior JPEG compression resistance ability. The bit error rate (BER) of JPEG attack is equal to zero for all testing images with quality factor≥80. The BER is much lower than the traditional watermarking algorithms, such as spread spectrum watermarking and quantization index modulation watermarking. The payload is considerably high and there are no "salt-and-peppers" artifacts in the watermarked images. No additional color is introduced and the palette keeps unchanged after watermark embedding. The proposed algorithm can be applied to natural images with distinct boundaries by segmenting the images into different parts.
Video Error Concealment Using Spatio-Temporal Boundary Matching and Partial Differential Equation

Error concealment techniques are very important for video communication since compressed video sequences may be corrupted or lost when transmitted over error-prone networks. In this paper, we propose a novel two-stage error concealment scheme for erroneously received video sequences. In the first stage, we propose a novel spatio-temporal boundary matching algorithm (STBMA) to reconstruct the lost motion vectors (MV). A well defined cost function is introduced which exploits both spatial and temporal smoothness properties of video signals. By minimizing the cost function, the MV of each lost macroblock (MB) is recovered and the corresponding reference MB in the reference frame is obtained using this MV. In the second stage, instead of directly copying the reference MB as the final recovered pixel values, we use a novel partial differential equation (PDE) based algorithm to refine the reconstruction. We minimize, in a weighted manner, the difference between the gradient field of the reconstructed MB in current frame and that of the reference MB in the reference frame under given boundary condition. A weighting factor is used to control the regulation level according to the local blockiness degree. With this algorithm, the annoying blocking artifacts are effectively reduced while the structures of the reference MB are well preserved. Compared with the error concealment feature implemented in the H.264 reference software, our algorithm is able to achieve significantly higher PSNR as well as better visual quality.
Objective speech quality measure for cellular phone

Cellular phone network speech quality monitoring is a regular task performed by the cellular service providers. Objective speech quality measures are needed in such tasks to provide a reasonably accurate estimate of subjective quality of the network. We performed an experiment to collect real distorted data, conducted a survey to obtain subjective quality measure of the collected speech samples and studied the statistical correlation of 32 objective speech quality measures with the subjective measures. Four of the objective measures were found to be good. Synchronization was found to be important.
New results on zonal based motion estimation algorithms-advanced predictive diamond zonal search

Zonal based motion estimation algorithms have been gaining in popularity in video coding. This is due to their superior performance and reduced complexity versus other conventional algorithms. In this paper we propose a further improvement on these algorithms named the Advanced Predictive Diamond Zonal Search (APDZS). The proposed algorithm introduces the concepts of multiple initial predictor candidates and adaptive thresholding, and manages to significantly improve the reliability and performance of the estimation. Extensive simulation results demonstrate that the performance of this algorithm is equivalent in terms of PSNR, to that of the brute force full search algorithm even for the most complicated sequences, with almost negligible complexity.
Determining the Capacity Parameters in PEE-Based Reversible Image Watermarking

In the existing prediction-error expansion (PEE)-based reversible image watermarking schemes, the capacity parameters are determined in a recursive manner until the payload is just accommodated. This class of methods requires many rounds of embedding iterations, especially when the payload is high, and therefore, is computationally inefficient. Moreover, when multiple capacity parameters need to be determined, the previous methods cannot guarantee optimality in the capacity-distortion sense. In this work, a capacity-distortion optimization (CDO) framework is built to estimate the optimal capacity parameters. We prove that the CDO problem is convex for any embedding rates, permitting efficient solution. The estimated capacity parameters then serve as starting point to facilitate a local search algorithm to find the optimal capacity parameters with much less rounds of embedding iterations. Experimental results are provided to validate our findings.
Graph segmentation revisited: Detailed analysis and density learning based implementation

In this paper we give a step-by-step detailed analysis on the performance of shortest spanning tree (SST) and its revised version, recursive SST (RSST). We further propose a novel segmentation scheme based on recursive SST in the warped domain produced by density estimation. The proposed method is robust for variant natural image input and is easy to implement. Experimental results and comparisons with other methods have illustrated the effectiveness and robustness of the proposed method.
A partitioned linear minimum mean square estimator for error concealment [video decoder error concealment]

In this paper, we proposed a partitioned linear minimum mean square error estimator (P-LMMSE) for error concealment. The proposed P-LMMSE estimator adopts the multi-hypothesis motion compensation (MHMC) technique to reconstruct the corrupted block, in which the lost blocks are predicted by a linear combination of motion compensated blocks (hypotheses). In our proposed P-LMMSE estimator, the weighting coefficients are optimal in the sense that they minimize the mean square error. In addition, our proposed estimator exploits the properties of the hypotheses to improved accuracy of prediction. Each hypothesis has its own assumption and hence works well only in a particular situation, or equivalently, the statistics in different situations are not the same. Therefore, the dataset is divided into finer partitions and the weighting coefficient set in the most appropriate partition is selected to reconstruct the corrupted block.
COSMOS: Peer-to-Peer Collaborative Streaming Among Mobiles

In traditional mobile streaming networks such as 3G cellular networks, all users pull streams from a server. Such pull model leads to high streaming cost and problem in system scalability. In this paper, we propose and investigate a scalable and cost-effective protocol to distribute multimedia content to mobiles in a peer-to-peer manner. Our protocol, termed collaborative streaming among mobiles (COSMOS), makes use of multiple description coding (MDC) and data sharing to achieve high performance. In COSMOS, only a few peers pull video descriptions through a telecommunication channel. Using a free broadcast channel (such as Wi-Fi and Bluetooth), they share the descriptions to nearby neighbors in an ad-hoc manner. This way reduces greatly the telecommunication cost and cellular bandwidth requirement. As video descriptions are supplied by multiple peers, COSMOS is robust to peer failure. Since broadcasting is used to distribute video data, the protocol is highly scalable to large number of users. By taking turns to pull descriptions, we show through simulation that peers can effectively share, and hence substantially reduce, streaming cost. As peers can often obtain a number of descriptions from nearby neighbors, they enjoy lower delay as compared to a recent scheme CHUM
Multichannel Nonlocal Means Fusion for Color Image Denoising

In this paper, we propose an advanced color image denoising scheme called multichannel nonlocal means fusion (MNLF), where noise reduction is formulated as the minimization of a penalty function. An inherent feature of color images is the strong interchannel correlation, which is introduced into the penalty function as additional prior constraints to expect a better performance. The optimal solution of the minimization problem is derived, consisting of constructing and fusing multiple nonlocal means (NLM) spanning all three channels. The weights in the fusion are optimized to minimize the overall mean squared denoising error, with the help of the extended and adapted Stein's unbiased risk estimator (SURE). Simulations on representative test images under various noise levels verify the improvement brought by the multichannel NLM, compared to the traditional single-channel NLM. In the meantime, MNLF provides competitive performance both in terms of the color peak signal-to-noise ratio and in perceptual quality when compared with other state-of-the-art benchmarks.
Enhanced Image Trans-coding Using Reversible Data Hiding

The primary application of watermarking and data hiding is for authentication, or to prove the ownership of digital media. In this paper, a new trans-coding system with the help of the technique of watermarking and data hiding is proposed. Side information is extracted before the image transcoding, such as resizing. In this paper, we focus on the problem of resizing in "thin edge" region. "Thin edge" structure normally cannot preserve after resizing process and become discrete. As the "thin edge" region is difficult to analyze real time and with a high degree of accuracy, data hiding can be used. The information of the "thin edge" region is generated and embedding into the multimedia content in encoder. Experimental results shown that there is a great improvement in the visual quality by using side information.
Dual Laplacian editing for meshes

Recently, differential information as local intrinsic feature descriptors has been used for mesh editing. Given certain user input as constraints, a deformed mesh is reconstructed by minimizing the changes in the differential information. Since the differential information is encoded in a global coordinate system, it must somehow be transformed to fit the orientations of details in the deformed surface, otherwise distortion will appear. We observe that visually pleasing deformed meshes should preserve both local parameterization and geometry details. We propose to encode these two types of information in the dual mesh domain due to the simplicity of the neighborhood structure of dual mesh vertices. Both sets of information are nondirectional and nonlinearly dependent on the vertex positions. Thus, we present a novel editing framework that iteratively updates both the primal vertex positions and the dual Laplacian coordinates to progressively reduce distortion in parametrization and geometry. Unlike previous related work, our method can produce visually pleasing deformations with simple user interaction, requiring only the handle positions, not local frames at the handles.
Inter-channel demosaicking traces for digital image forensics

Digital image forensics seeks to detect statistical traces left by image acquisition or post-processing in order to establish an images source and authenticity. Digital cameras acquire an image with one sensor overlayed with a color filter array (CFA), capturing at each spatial location one sample from the three necessary color channels. The missing pixels must be interpolated in a process known as demosaicking. This process is highly nonlinear and can vary greatly between different camera brands and models. Most practical algorithms, however, introduce correlations between the color channels, which are often different between algorithms. In this paper, we show how these correlations can be used to construct a characteristic map that is useful in matching an image to its source. Results show that our method employing inter-channel traces can distinguish between sophisticated demosaicking algorithms. It can complement existing classifiers based on inter-pixel correlations by providing a new feature dimension.
Novel RD-Optimized VBSME With Matching Highly Data Re-Usable Hardware Architecture

To achieve superior performance, rate-distortion optimized motion estimation (ME) for variable block size (RDO VBSME) is often used in state-of-the-art video coding systems such as the H.264 JM software. However, the complexity of RDO-VBSME is very high both for software and hardware implementations. In this paper, we propose a hardware-friendly ME algorithm called RDOMFS with a novel hardware-friendly rate-distortion (RD)-like cost function, and a hardware-friendly modified motion vector predictor. Simulation results suggest that the proposed RDOMFS can achieve essentially the same RD performance as RDO-VBSME in JM. We also propose a matching hardware architecture with a novel Smart Snake Scanning order which can achieve very high data re-use ratio and data throughout. It is also reconfigurable because it can achieve variable data re-use ratio and can process variable frame size. The design is implemented with TSMC 0.18 μm CMOS technology and costs 103 k gates. At a clock frequency of 63 MHz, the architecture achieves real-time 1920 × 1080 RDO-VBSME at 30 frames/s. At a maximum clock frequency of 250 MHz, it can process 4096 × 2160 at 30 frames/s.
Parallel rate-distortion optimized intra mode decision on multi-core graphics processors using greedy-based encoding orders

Rate-distortion (RD) optimized intra-prediction mode selection can lead to significant improvement in coding efficiency in intra-frame encoding. However, it would incur considerable increase in encoding complexity. In this paper, we investigate how multi-core Graphics Processing Units (GPUs) can be efficiently utilized to undertake the task of RD optimized intra mode selection in AVS and H.264 video encoding. Achieving efficient GPU-based intra mode decision, however, could be non-trivial. It is because the mode decision of the current block would depend on the reconstructed data of the neighboring blocks. Therefore, the coding modes of neighboring blocks would need to be computed first before that of the current block can be determined. This dependency poses challenge to computation on multi-core GPUs, which rely heavily on parallel data processing to achieve superior speedups. To address this issue, we analyze the data dependency in intra mode decision, and propose novel greedy-based encoding orders to achieve highly parallel processing. We also prove that the proposed greedy-based orders are optimal in terms of execution time. Experimental results suggest that the proposed GPU-based intra mode decision compares favorably to the counterpart implemented on a single-core CPU.
BDCT compressed image deblocking usingweighted adaptive total variation

Images encoded at low bit rate usually exhibit visually annoying coding artifacts, which are commonly referred as blocking artifacts. In this paper, a novel weighted adaptive total variation method is proposed to remove the blocking artifacts. Based on the analysis of image coding process and block-based discrete cosine transform (BDCT) image properties, image deblocking is formulated as an optimization problem which is solved through approximating the objective function by a set of convex functions. Experimental results show that the proposed method can achieve better objective and subjective quality performance compared to other deblocking algorithms.
Rate-distortion optimized 3D reconstruction from noise-corrupted multiview depth videos

Transmitting compactly represented geometry of a dynamic scene from a sender can enable a multitude of 3D imaging functionalities at a receiver, such as synthesis of virtual images from freely chosen viewpoints via depth-image-based rendering (DIBR). While depth maps can now be readily captured using inexpensive depth sensors, they are often corrupted by non-negligible acquisition noise. In this paper, we derive 3D surfaces of a dynamic scene from noise-corrupted depth maps in a rate-distortion (RD) optimal manner. Specifically, unlike previous work that finds the most likely (e.g., maximum likelihood) 3D surface from noisy observations regardless of representation size, we judiciously search for the best fitting (i.e., minimum distortion) 3D surface subject to a bitrate constraint. Our RD-optimal solution reduces to the maximum likelihood solution as the rate constraint is loosened. Using the MVC codec for compression of multiview depth video and MPEG free viewpoint test sequences as input, experimental results show that RD-optimized 3D reconstructions computed by our algorithm outperform unprocessed depth maps by up to 2:42dB in PSNR of synthesized virtual views at the decoder for the same bitrate.
Fast sub-pixel inter-prediction - based on texture direction analysis (FSIP-BTDA) [video coding applications]

Motion estimation (ME) is a core part of modern video coding systems to exploit temporary redundancy in a video. Motion estimation is typically performed firstly with integer pixel accuracy and then at sub-pixel accuracy, which includes half-pixel and quarter-pixel accuracy. When sophisticated fast integer-pixel accuracy motion-search algorithms are used to decrease the number of search points for integer-pixel motion search, thus sub-pixel motion search became another important processing bottleneck in the encoding process. The conventional method is to search 8 half-pixel positions around the motion vector (MV) obtained from integer pixel motion search, then do motion search in the same way on 8 quarter-pixel positions around the MV obtained from half-pixel motion search, therefore totally 16 search points are needed. The proposed algorithm, fast sub-pixel inter-prediction - based on texture direction analysis (FSIP-BTDA), successfully optimizes the sub-pixel motion search on both half and quarter-pixel accuracy, and improves the processing speed with low PSNR penalty.
A Novel Framework for Frame Rate Up Conversion by Predictive Variable Block-Size Motion Estimated Optical Flow

This paper proposes a novel framework for frame rate up conversion. The framework contains a motion field estimator employing a hybrid of a novel predictive variable blocksize motion estimation and robust optical flow algorithm. This motion field is more robust and accurate than pure optical flow or pure motion estimation methods. Therefore, in-between frames can be estimated more accurately by the new hybrid method. After the motion field estimation, pixel interpolation following motion field interpolation is done for in-between frames. It is shown the proposed method has better than pure motion estimation and pure optical flow methods in terms of both subjective and objective measurements.
Modified one-bit transform for motion estimation

Motion estimation using the one-bit transform (1BT) was proposed by Natarajan, Bhaskaran and Konstantinides (see ibid., vol.7, p.702-06, 1997) to achieve large computation reduction. However, it degrades the predicted image by almost 1 dB as compared with full search. We propose a modification to the 1BT by adding conditional local searches. Simulation results show that the proposed modification improves the peak signal-to-noise ratio (PSNR) significantly at the expense of slightly increased computational complexity. A variant of the proposed modification called the multiple-candidate two-step search (M2SSFS) is found to be particularly good for high quality, high bit rate video coding. In the MPEG-1 simulation, its PSNR is within 0.1 dB from that of full search at bit rates higher than 1 Mbit/s with a computation reduction factor of ten.