nils-herrmann/clean_scirepeval huggingface dataset:
From 2D Extrapolation to 1D Interpolation: Content Adaptive Image Bit-Depth Expansion

In this paper, we address the problem of image bit-depth expansion and present a novel method to generate high bit-depth (HBD) images from a single low bit-depth (LBD) image. We expand image bit-depth by reconstructing the least significant bits (LSBs) for the LBD image after it is rescaled to high bit-depth. For image regions whose intensities are neither locally maximum nor minimum, neighborhood flooding is applied to convert 2D interpolation problem into 1D interpolation, for local maxima/minima (LMM) regions where interpolation is not applicable, a virtual skeleton marking algorithm is proposed to convert problematic 2D extrapolation problem into 1D interpolation. At last, a content-adaptive reconstruction model is proposed to obtain the output HBD image. The experimental results show that proposed method significantly outperforms existing methods in PSNR and SSIM without contouring artifacts.
Color image denoising based on multichannel non-local means fusion

In this paper, we investigate the problem of color image denoising, and propose a novel algorithm called multichannel non-local means fusion (MNLMF), building on the grayscale denoiser non-local means filter. By analyzing and modeling the inter-channel correlation in color images, we formulate the color noise reduction as a minimization problem with a specifically-designed penalty function which fully takes advantages of the inter-channel prior information. The optimal solution is derived consisting of constructing multiple non-local means spanning all three channels and fusing them together. The weights in the fusion are optimized to minimize the overall denoising error. Simulation results under various noise levels demonstrate that when compared to other state-of-the-art algorithms, the proposed MNLMF achieves competitive performance both in terms of the color peak signal-to-noise ratio (cPSNR) and in perceptual quality.
Fast motion estimation for video resolution down-conversion using spatial-variant filter

To convert a compressed video sequence to a lower-resolution compressed video, one typically needs to decompress the original sequence, down-sample each frame, and recompress it. It involves motion estimation in the reduced sequence, which is computational intensive. In this paper, a novel fast motion estimation algorithm is proposed to predict the motion vectors of the reduced-resolution video without performing any search. The motion vectors are predicted by applying spatial-variant filters to the motion vectors of the original compressed high-resolution video. In simulations, our method outperforms other existing fast predictive algorithms which involve no searching.
Image compression via dense descriptors assisted synthesis

In this paper, we propose a novel image compression approach towards visual quality rather than pixel fidelity. We intentionally remove several blocks at the encoder and reconstruct them at the decoder to get bits reduction. The removal blocks are wisely and adaptively selected based on blocks clustering, patch similarity and removal priority. A well-suited similarity measurement is defined to capture the common pattern between patches as well as tell their substitutability based on boundary consistency. To assist the removal blocks reconstruction at the decoder, we extract some dense descriptors as the side information to the decoder. Encouraging experimental results show that our compression scheme achieves up to 20.26%bits reduction with a comparable visual quality compared to the most recent standard High Efficiency Video Coding (HEVC).
Fast Fractal Image Encoding Via One-bit Transforms

In this paper we present an algorithm for faster Fractal image coding than traditional algorithms. The algorithm involves a method of transforming the image from a multibit to a one-bit/pixel representation, applying MPEG like full search block based estimation on the new image and then refining the result with the use of statistical normalization and fast block-based motion estimation techniques for MPEG encoding. Simulation results suggest that the algorithm can significantly reduce the computation compared to traditional methods.
Towards robust and efficient segmentation: An approach based on inter-region contour and intra-region content analysis

We address the problem of boundary estimation by formulating it as inter-region contour and intra-region information analysis in the framework of graph-based segmentation. Given an image without any prior information about object model and class, we seek to approximate one's instant perception of visual similarity. The method can serve as a preprocessing step for many higher level operations that require regional support, such as scene understanding and object recognition. We show in this paper that the defined region comparison predicate makes a better boundary estimator than efficient graph-based image segmentation (EGS) - a well known and widely used segmentation method. We further illustrate, by making a small relaxation, further improvement of segmentation performance can be achieved. Experimental results have demonstrated the effectiveness of our proposed method.
Depth Map Super-Resolution Using Synthesized View Matching for Depth-Image-Based Rendering

In texture-plus-depth format of 3D visual data, texture and depth maps of multiple viewpoints are coded and transmitted at sender. At receiver, decoded texture and depth maps of two neighboring viewpoints are used to synthesize a desired intermediate view via depth-image-based rendering (DIBR). In this paper, to enable transmission of depth maps at low resolution for bit saving, we propose a novel super-resolution (SR) algorithm to increase the resolution of the received depth map at decoder to match the corresponding received high resolution texture map for DIBR. Unlike previous depth map SR techniques that only utilize the texture map of the same view 0 to interpolate missing depth pixels of view 0, we use texture maps of the same and neighboring viewpoints, 0 and 1, so that the error between the original texture map of view 1 and the synthesized image of view 1 (interpolated using texture and depth maps of view 0) can be used as a regularization term during depth map SR of view 0. Further, piecewise smoothness of the reconstructed depth map is enforced by computing only the lowest frequency coefficients in Graph based Transform (GBT) domain for each interpolated block. Experimental results show that our SR scheme out-performed a previous scheme by up to 1.7dB in synthesized view quality in PSNR.
SSIM-based rate-distortion optimization in H.264

In the current video coding standards, rate-distortion optimization (RDO) plays an important role in achieving best tradeoff between the perceived distortion and transmission rate. It is widely used in all kinds of encoder decisions, including block mode decision, motion vector selection and so on. Generally, the sum of absolute difference (SAD) or the sum of square difference (SSD) is used as the distortion measurement. However, it is well known that both of them cannot always reflect the perceptual quality of the encoded video. In this paper, an objective quality measurement structural similarity (SSIM) index is proposed as the distortion measurement in the RDO framework for video coding standards. By fully exploiting the relationship between SSIM and mean square error (MSE), the SSIM-based RDO framework can be approximated by the original SSD-based RDO framework with only a scaling of the Lagrange multiplier. Experimental results show that the proposed method outperforms the latest H.264 codec and also the state-of-the-art SSIM-based RDO video codec.
Low Bit-Rate Subpixel-Based Color Image Compression

We propose a novel low bit-rate compression scheme with sub pixel-based down-sampling and reconstruction (SPDR) for full color images. In the encoder stage, a decoder-dependent multi-channel sub pixel-based down-sampling is proposed, which is more effective in retaining high frequency detail than conventional pixel-based process. The decoder first decompresses the low-resolution image and then up-converts it to the original resolution using encoder dependent sub pixel-based reconstruction scheme by jointly considering the sub pixel-based down-sampling effect and the compression degradation. Compared to existing algorithms with comparable encoder and decoder complexity, the proposed SPDR offers complete standard compliance, competitive rate-distortion performance, and superior subjective quality.
Error recovery of variable length codes over BSC with arbitrary crossover probability

The error recovery capability of variable length code (VLC) has been considered as an important performance and design criterion in addition to its coding efficiency. However, almost all of the existing methods for evaluating the error recovery capability of VLC assume that the transmission fault is a random single bit inversion. In this paper, we consider a more generalized problem of precisely evaluating the error recovery capability of VLC in the case that the encoded bit stream is transmitted over a BSC with arbitrary crossover probability. By making use of the Perron-Frobenius Theorem, we derive a very simple expression for the exact mean symbol error rate (MSER) in Levenshtein distance sense. We also prove that in the very low crossover probability region, the mean error propagation length (MEPL) derived for single inversion error case approaches a scaled value of MSER. In addition, we briefly discuss the error recovery of VLC over Gilbert-Elliott channel, which is one of the simplest and practical models for a channel with memory.
Automatic white balancing using luminance component and standard deviation of RGB components [image preprocessing]

Automatic white balancing is an essential image preprocessing component in consumer digital still cameras, and it can greatly improve the final image quality of the captured image. In this paper, a novel automatic white balancing algorithm based on both the luminance component and standard deviation of RGB components of the pre-captured image is proposed. A light source model for evaluation of an automatic white balancing is also described. The simulation results indicate that the proposed algorithm can improve the final image quality of the captured image.
A convex-optimization approach to dense stereo matching

We present a novel convex-optimization approach to solving the dense stereo matching problem in computer vision. Instead of directly solving for disparities of pixels, by establishing the connection between a permutation matrix and a disparity vector, we directly formulate the stereo matching problem as a continuous convex quadratic program in a simple, elegant and straightforward manner without performing any complicated relaxation or approximation. By using CVX, the Matlab software for disciplined convex programming, our method is extremely simple to implement.
Complexity adaptive H.264 encoding using multiple reference frames

The state-of-the-art H.264/AVC video coding standard achieves significant improvements in coding efficiency by introducing many new coding techniques. However the computation complexity is inevitably increased during both the encoding and decoding process. Many previous works, such as fast motion estimation and fast mode decision algorithms, have been proposed aiming at reducing the encoder complexity while maintaining the coding efficiency. In this paper, we propose a new encoding approach which accounts for the decoding complexity. Simulation results show that the decoding complexity can be reduced by up to 15% in terms of motion compensation operations, which is the most complex part of the decoder, while maintaining the R-D performance with only about 0.1 dB degradation.
Multimedia human computer interface for oriental calligraphies

Most researches on oriental calligraphies simulation are based on the texture rendering methods with input from pointing devices. However, the traditional way is by using the hairy brush which is not employed in these applications. In this paper, we proposed a novel multimedia human computer interface that allows users which including artists to use the physical hairy brushes to write without training, so as to provide the same user experience. A virtual paper is projected on the transparent screen, the interface for using the hairy brush. An infra-red camera is used in capturing the image as the system input. A software framework was proposed to compute the contact region and the ink absorption between the ink-brush and the virtual paper. This information is highly useful in further rendering the texture of the brushstrokes and the ink spreading on the paper.
[Demo paper] learning to beautify facial image

In this demo, we demonstrate a data-driven facial image beautification system that learns how to beautify portraits from facial image database, and enhances the facial texture of arbitrary portraits automatically by modifying its pigment distribution and correcting its color. Specifically, as human skin can be treated as a turbid medium with multilayered structure, we decompose facial image into melanin and hemoglobin layers. With the extracted attractiveness features, a data-driven qualitative beautify model serves for the guidance of beautification through optimizing hemoglobin and melanin layers. Our beautification operations are conducted in completely automatic and time-efficient way, leading to customized realistic beautified portraits that follows users' preferences.
Edge-based Adaptive Directional Intra Prediction

H.264/AVC employs intra prediction to reduce spatial redundancy between neighboring blocks. Different directional prediction modes are used to cater diversified video content. Although it achieves quite high coding efficiency, it is desirable to analyze its drawbacks in the existing video coding standard, since it allows us to design better ones. Basically, even after intra prediction, the residue still contains a lot of edge or texture information. Unfortunately, these high frequency components consume a large quantity of bits and the distortion is usually quite high. Based on this drawback, an Edge-based Adaptive Directional Intra Prediction is proposed (EADIP) to reduce the residue energy especially for the edge region. In particular, we establish an edge model in EADIP, which is quite flexible for natural images. Within the model, the edge splits the macroblock into two regions, each being predicted separately. In implementation, we consider the current trend of mode selection and complexity issues. A mode extension is made on INTRA 16×16 in H.264/AVC. Experimental results show that the proposed algorithm outperforms H.264/AVC. And the proposed mode is more likely to be chosen in low bitrate situations.
Low-Complexity Rate Control for Efficient H.263 to H.264/AVC Video Transcoding

Rate control is a complicated problem in the H.264/AVC coding standard, extra computation is usually needed for the existing rate control schemes to estimate the complexity of frames or macroblocks (MBs). However, during transcoding, information from preceded video could be used to simplify the rate control. In this paper, we propose a low-complexity rate control scheme for transcoding from H.263 to H.264/AVC. The relationship between the rate of the pre-coded video and both the rate and distortion of the transcoded video are studied. By using only the rate information from the preceded video, we introduce a row-layer bit allocation and perform average rate shaping across a row of MBs. Estimation error diffusion is also introduced. The proposed scheme has sufficiently lower computational complexity than other methods as there is no explicit complexity measurement of MBs and complicated parameters updating. Experimental results show the effectiveness of the proposed scheme.
View Synthesis Prediction in the 3-D Video Coding Extensions of AVC and HEVC

Advanced multiview video systems are able to generate intermediate viewpoints of a 3-D scene. To enable low-complexity free view generation, texture and its associated depth are used as input data for each viewpoint. To improve the coding efficiency of such content, view synthesis prediction (VSP) is proposed to further reduce interview redundancy in addition to traditional disparity compensated prediction. This paper describes and analyzes rate-distortion optimized VSP designs, which were adopted in the 3-D extensions of both Advanced Video Coding (AVC) and High Efficiency Video Coding (HEVC). In particular, we propose a novel backward-VSP scheme using a derived disparity vector, as well as efficient signalling methods in the context of AVC and HEVC. In addition, we put forward a novel depth-assisted motion vector prediction method to optimize the coding efficiency. A thorough analysis of coding performance is provided using different VSP schemes and configurations. Experimental results demonstrate average bit rate reductions of 2.5% and 1.2% in AVC and HEVC coding frameworks, respectively, with up to 23.1% bit rate reduction for dependent views.
Data hiding by smart pair toggling for halftone images

There is growing interest in hiding data for authentication and copyright control in halftone images printed in books, newspapers and by computer printers. A previous data hiding method, the data hiding by pair-toggling (DHPT), is reasonably good but introduce considerable visual artifacts. In this paper, we analyze the sources of the artifacts in DHPT and propose an improvement by using smart pair toggling. Simulation results suggest that the proposed data hiding by smart pair-toggling (DHSPT) algorithm can hide the same amount of data while generating halftone images with considerably better visual quality than DHPT.
Compressing similar image sets using low frequency template

In advance of the imaging capturing technology, large amount of similar images are created. Instead of compressing each similar image individually, removing the inter-image redundancy would reduce the storage and transmission time. However, only a few set redundancy methods are proposed to deal with the problem. In this paper, a new method was derived from a theoretical model by extracting the low frequency in an image set. For the similar images, the values of their low frequency components are very close to that of their neighboring pixel in the spatial domain. In our model, a low frequency template is created and used as a prediction for each image to compute its residue. This model proves the reduction in the entropy and hence the bit rates. Experiments were conducted and proved there were up to 30% gains over the existing methods.
Chroma Replacing and adaptive Chroma Blending for subpixel-based downsampling

Subpixel-based downsampling generates images with higher apparent resolution with the expense of annoying color-fringing artifacts near strong edges. In this paper we propose two methods that find a balance in the tradeoff of apparent resolution and color-fringing artifacts. The first method is called Chroma Replacing in which the color-fringing artifacts are completely removed but the subpixel rendering effect is also removed. The second one is called Chroma Blending in which only the color-fringing artifacts that are strong enough to be noticed are removed, and also the subpixel rendering effect is retained. We also propose two objective measures for measuring the similarity of downsampled image to the original image. Experiment results show that the proposed methods are effective in removing color-fringing artifacts, without harming the high apparent resolution.
Depth map denoising using graph-based transform and group sparsity

Depth maps, characterizing per-pixel physical distance between objects in a 3D scene and a capturing camera, can now be readily acquired using inexpensive active sensors such as Microsoft Kinect. However, the acquired depth maps are often corrupted due to surface reflection or sensor noise. In this paper, we build on two previously developed works in the image denoising literature to restore single depth maps-i.e., to jointly exploit local smoothness and nonlocal self-similarity of a depth map. Specifically, we propose to first cluster similar patches in a depth image and compute an average patch, from which we deduce a graph describing correlations among adjacent pixels. Then we transform similar patches to the same graph-based transform (GBT) domain, where the GBT basis vectors are learned from the derived correlation graph. Finally, we perform an iterative thresholding procedure in the GBT domain to enforce group sparsity. Experimental results show that for single depth maps corrupted with additive white Gaussian noise (AWGN), our proposed NLGBT denoising algorithm can outperform state-of-the-art image denoising methods such as BM3D by up to 2.37dB in terms of PSNR.
Subpixel rendering: from font rendering to image subsampling [Applications Corner]

Subpixel rendering technologies take advantage of the subpixel structure of a display to increase the apparent resolution and to improve the display quality of text, graphics, or images. These techniques can potentially improve the apparent resolution because a single pixel on color liquid crystal display (LCD) or organic light-emitting diode (OLED) displays consists of several independently controllable colored subpixels. Applications of subpixel rendering are font rendering and image/video subsampling. By controlling individual subpixel values of neighboring pixels, it is possible to microshift the apparent position of a line to give greater details of text. Similarly, since the individual selectable components are increased threefold by controlling subpixels rather than pixels, subpixel-based subsampling can potentially improve the apparent resolution of a down-scaled image. However, the increased apparent luminance resolution often comes at the price of color fringing artifacts. A major challenge is to suppress chrominance distortion while maintaining apparent luminance sharpness. This column introduces subpixel arrangement in color displays, how subpixel rendering works, and several practical subpixel rendering applications in font rendering and image subsampling.
Temporal interpolation using wavelet domain motion estimation and motion compensation

Temporal frame subsampling is a simple but efficiency technique to achieve very high compression ratio in slow motion video. The discrete wavelet transform (DWT) is successful in still image compression and many researchers try to extend the usage of DWT to a video coder. In this paper, we incorporate these two ideas, together with the newly developed wavelet domain motion estimation method - low-band-shift motion estimation (LBSME) to develop a novel temporal interpolation scheme in the wavelet domain.
Content-adaptive Temporal Search Range Control Based on Frame Buffer Utilization

Multiple reference frame selection adopted by the state-of-art H.264 video compression standard offers substantial performance gain. The temporal search range control, as a consequence, is crucial for maintaining the coding performance with minimum complexity. In this paper, we investigate the relationships between the reference frame buffer utilization and the optimal search range. A content-adaptive algorithm is proposed to control the search range dynamically during the encoding process. Experimental results show that our algorithm can rapidly adapt to the video characteristics and effectively reduce the complexity with negligible coding performance penalty