nils-herrmann/clean_scirepeval huggingface dataset:
Depth Map Super-Resolution Considering View Synthesis Quality

Accurate and high-quality depth maps are required in lots of 3D applications, such as multi-view rendering, 3D reconstruction and 3DTV. However, the resolution of captured depth image is much lower than that of its corresponding color image, which affects its application performance. In this paper, we propose a novel depth map super-resolution (SR) method by taking view synthesis quality into account. The proposed approach mainly includes two technical contributions. First, since the captured low-resolution (LR) depth map may be corrupted by noise and occlusion, we propose a credibility based multi-view depth maps fusion strategy, which considers the view synthesis quality and interview correlation, to refine the LR depth map. Second, we propose a view synthesis quality based trilateral depth-map up-sampling method, which considers depth smoothness, texture similarity and view synthesis quality in the up-sampling filter. Experimental results demonstrate that the proposed method outperforms state-of-the-art depth SR methods for both super-resolved depth maps and synthesized views. Furthermore, the proposed method is robust to noise and achieves promising results under noise-corruption conditions.
A real-time HDTV video decoder

We present an architecture for real-time digital HDTV video decoding. Our technique is based on a dual decoding datapath controlled in a fixed schedule with an efficient write-back scheme for anchor pictures. Unlike other decoding approaches such as the slice bar decoding method and the crossing-divided method, our scheme reduces the memory access contention problem to achieve real-time HDTV decoding without a high cost in overall decoder buffers, architecture, and bus. Our simulation shows that with a relatively low rate 81 MHz clock, our decoder can decode MPEG-2 MP@HL HDTV in real-time, based on an ATSC video format of 1920/spl times/1080 pixels/frame at 30 frames/s, at a bit rate of 18 to 20 Mbit/s.
Fast merge mode decision for diamond search in High Efficiency Video Coding

This paper proposes a merge mode decision algorithm while maintaining the accuracy of a diamond search (DS) in motion estimation and compensation. In High Efficiency Video Coding (HEVC), the merge mode is used to reduce the bit rate in order to carry motion information. The rate-distortion (RD) cost of the merge mode is compared with the RD cost of the inter-prediction mode in the course of the motion estimation which can be terminated early when the merge cost is smaller than the estimated cost of the inter-prediction mode. To this end, this paper proposes a fast merge mode prediction algorithm when the DS is used for motion estimation. The main purpose of this work is to estimate the RD cost of the merge mode in advance by utilizing the distortion information of the RD cost of the motion vector prediction (MVP) so as to early terminate the motion search operation. Experimental results show that the proposed fast merge mode estimation achieves a comparable RD performance but reduces the amount of computation by 16.6% on the average when compared to the fast motion estimation with the diamond search implemented in the HM 8.0 reference software.
Rate control using enhanced frame complexity measure for H.264 video

Bit rate control is an important issue for wireless and Internet video streaming. The paper presents a revised rate control scheme based on an improved frame complexity measure. Rate control adopted by both MPEG-4 VM18 and H.264/AVC uses a quadratic rate-distortion (R-D) model that determines quantization parameters (QPs). The classical quadratic R-D model is suitable for MPEG-4, but it performs poorly for H.264/AVC because one of the important parameters, mean absolute difference (MAD), is predicted through a linear model, whereas the MAD used in MPEG-4 VM18 is the actual MAD. Inaccurately predicted MAD results in the wrong QP and consequently degrades rate distortion optimization (RDO) performance in H.264/AVC. To overcome the limitation of the existing rate control schemes, we introduce an enhanced linear model for predicting MAD, utilizing some knowledge of current frame complexity. Moreover, we propose a more accurate frame complexity measure, namely, normalized MAD, to replace the current use of MAD parameters. Normalized MAD has a stronger correlation with optimally allocated bits than that of the predicted MAD. Finally, a dynamic bit allocation scheme among basic units is implemented. Extensive simulation results show that our method, with inexpensive added computational complexity, improves the average peak signal-to-noise ratio (PSNR) considerably, by up to 1.2 dB, and reduces PSNR variances significantly, by up to 63%.
On speed optimization of MPEG-4 decoder for real-time multimedia applications

In this paper, we present several speed optimizing techniques in our MPEG-4 video decoder for real-time multimedia applications. Accessor functions and a simple data structure are used. The use of pointers is minimized. Fast algorithms and fixed-point arithmetic are applied whenever possible. Memory accesses are kept to the minimum. Our results show that our software can decode and display MPEG-4 QCIF videos about eight times faster than the existing MoMuSys decoder.
On Model Parameter Estimation for H.264/AVC Rate Control

We present an idea of improving the rate quantization (R-Q) model based rate control for H.264/AVC from a novel model parameter estimation perspective. Our idea is different from the conventional belief that emphasizes the statistical accuracy property of an R-Q model. Instead, in our opinion, a good R-Q model should also balance the model accuracy with dynamic properties such as the predictability of its model parameter. This new angle is important for two primary reasons: First, due to the non-stationary nature of video signals, deviation from the assumptions for an R-Q model is almost inevitable. Second, for timing-sensitive applications, model parameters are not known in advance and thus have to be estimated on the fly where estimation errors cannot be avoided. We found that compared to the quadratic R-Q model (Lee et al., 2000), while the linear R-Q model is a bit less accurate when the statistical data is known, its model parameter has much better predictability and faster to estimate. Taking advantage of these nice properties of the linear R-Q model, the rate control for H.264/AVC yields significantly better rate distortion performance than that of JM 10.2 reference software where the quadratic R-Q model is used. The PSNR improvement is up to 0.63 dB and about 0.40 dB on the average for our test sequences. Furthermore, the proposed rate control achieves the target bit rate much more accurately
Transcoding with quality enhancement and irregular sampling

The need to move data among devices with different capabilities creates a need for transcoding. In this paper, an efficient transcoding algorithm from VC-1 to H.264 video with down/up sampling is discussed. Transcoding with resolution conversion usually results in lower video output quality. To address this problem an algorithm for quality enhancement with the use of irregular sampling is presented to use with transcoding. The quality enhancement module can also be used independently from the transcoding step. Experimental results show that the proposed algorithm can successfully remove artifacts and generate higher video quality. The main contribution of this paper is the use of irregular sampling techniques for video quality enhancement.
A real-time realization of MPEG-4 video decoder

In this paper, several techniques for real-time realization of an MPEG-4 video decoder software is presented. The structure of the decoder is also discussed. Our methods improve the processing of three computationally intensive modules, context-based arithmetic decoding, padding, and variable length decoding. The decoding speed of our developed software is a few times faster than the existing MPEG-4 reference software.
Flat hex Pattern for Large Range Fast Motion Estimation

Fast motion estimations achieve reasonable computation power, by using search patterns to find a potential ideal match. This paper presents a flat hex fast motion estimation search pattern, which favors horizontal displacement over vertical displacement. Experimental results show that due to the enlargement of the horizontal dimension in the greatest conventional "circular" search pattern, the flat hex maintains similar PSNR, with a little reduced bit rate, but achieves a decrease computation time.
A vector quantizer with increased codebook patterns

In this paper, we present an approach to increase the amount of VQ codebook patterns to improve image quality without a large overhead on the memory and search time. Basically, our approach characterizes a pattern into a key value, which is used in codebook search; the "reflection operation" was performed on codewords using the key values to get more patterns to improve image quality. Our basic VQ system is a 2-step VQ where VQ is used to compress prediction errors resulting from simple scalar quantization of pixel blocks. The codebook search time for our technique is O(logN+K) for each input vector, where N is the codebook size and K is the Voronoi range. Experimental results are presented in the paper.
Integration of Plane Mode with Multiple Predictor Sets in Intra Prediction for video coding

Current default intra prediction method in the current test model for the next generation video coding standard JCT-VC, Unified Intra Prediction, has up to 33 directional modes and only one non-directional mode, which is Intra DC mode. Having only one non-directional mode is insufficient for maintaining accuracy on prediction units with complex texture. In this contribution, Plane Mode is integrated as an additional non-directional mode and an improved Multiple Predictor Set method is introduced; the proposed new method for intra mode prediction is used to obtain higher accuracy for intra mode prediction in video coding. Experimental results have shown that coding efficiency is improved with the proposed method.
Adaptively weighted update steps using chrominance for scalable video coding

Scalable video coding using motion-compensated temporal filtering is one of the latest trends in video coding standardization. In the lifting based motion-compensated temporal filtering framework, such as that in the joint scalable video model (JSVM), the data used in the update steps are basically the residuals from the motion-compensated prediction. When the motion model used in the prediction steps fails to capture the true motion, energy in the high-pass temporal frames becomes substantial and strong ghosting artifacts may be introduced to the low-pass frames during the update steps. In this paper we propose a new block-based update approach, which takes advantage of the chrominance information of the video sequence to further reduce ghosting artifacts in low-pass temporal frames. We adaptively weight the update steps according to the energy not only of luminance pixels, but also of chrominance pixels in the high-pass temporal frames at the corresponding locations. Experimental results show that the proposed algorithm can significantly improve the quality of the reconstructed video sequence, in PSNR and visual quality.
Improved update steps through motion vector correlation analysis for scalable video coding

Recent breakthroughs in motion-compensated temporal filtering (MCTF) has enabled the realization of scalable video coding (SVC). In this paper, we propose a new scheme to improve the update steps of MCTF by analyzing the correlation of the motion vectors (MVs) of the neighboring blocks as well as the correlation of the derived update MVs in the low-pass frames. Results show that the proposed algorithm improves the visual quality of reconstructed video sequences at different bit rates.
Raptor Codes Based Unequal Protection for Compressed Video According to Packet Priority

Raptor codes are state-of-the-art forward error correction (FEC) solutions for multimedia transmission, which have been applied to unequal error protection (UEP) of multi-layered media such as scalable video coding. In this paper, we address the problem of UEP for single-layered video over packet erasure channels. By exploiting the different priorities of video packets inside a group of pictures (GOP) and making full use of the good characteristics of standardized Raptor codes at large block length, we propose an optimized UEP framework for single-layered video and develop an efficient algorithm to solve it. Simulation results show that significant gains can be obtained by our method in case of packet losses.
Frame layer bit allocation scheme for constant quality video

In order to achieve constant quality across the whole video sequence under the channel bandwidth and buffer constraints, it is necessary to allocate more bits to frames with scene changes or high complexity and fewer bits to low complexity frames. In this work, we propose a new frame layer bit allocation scheme for H.264 video coding using mean absolute difference (MAD) ratio, which is the ratio of MAD of current frame to the average MAD from the starting frame up to the previous frame in a GOP. We provide a theoretical justification of MAD ratio as a measure of frame complexity. Bit budget is allocated to frames according to their MAD ratios, combined with the bits computed based on their buffer status. Simulation results show that the H.264 coder, using our proposed algorithm with virtually little computational complexity added, effectively alleviates visual quality degradation caused by high motion or scene changes. Our proposed algorithm significantly reduces the standard deviation of PSNR, hence producing a nearly constant video quality throughout the whole video sequence, when compared with other existing video schemes
Adaptive RD Optimal Sparse Coding With Quantization for Image Compression

In image and video compression for many multimedia applications, an image/frame is divided into component blocks or patches and is then encoded using some type of transform. Traditional transforms use a complete dictionary of basis functions. A recent technique of growing interest is signal approximation using a linear combination of basis functions from an overcomplete dictionary. The result is a sparse set of coefficients that can represent the original signal and is called sparse coding. This is an NP-hard problem. Orthogonal matching pursuit is a greedy algorithm that is effectively used to address this problem. Keeping in mind the iterative nature of this algorithm, in a recent conference publication, we proposed a rate distortion optimization (RDO) method to select the best sparse representation among iterations up to a target sparsity level. In this paper, we expand the work and consider an adaptive coding scheme that takes advantage of both discrete cosine transform (DCT) and sparse coding. This scheme shows a better performance over plain DCT or sparse coding schemes. We further propose a scheme to increase the coding efficiency of sparse coding by quantizing the sparse coefficients. We investigate an RDO method to select the value of the quantization parameter from a range, balancing distortion, and bit rate. Based on experimental results, we provide a comparison between conventional DCT-based coding, sparse coding scheme, our mixed coding scheme, and the proposed method that includes quantization of the sparse coefficients.
Wavelet-based embedded memory reduction for MPEG decoder

In MPEG-2 HDTV, the bandwidth and size of memory requirements are very large. These requirements increase the system design difficulty and the hardware cost is high. There are a lot of compression algorithms that can reduce the size of the memory, but the algorithms are too complex. We propose a simple algorithm to provide low complexity hardware and high quality video. Our algorithm is based on the wavelet transform (WT) and we use two integer wavelet bases, so there are no multipliers needed. This will reduce the cost a lot in VLSI implementation. Following the WT, we use a variable-length coder to encode the data. For the memory access problem, we use group of macroblock segment coding and a hierarchical access method to reduce the access time.
Two polyphase filter architectures for MPEG audio

Polyphase subband filtering is a computational-intensive operation in MPEG audio coding. In this paper, we present two fast architectures by using a fast IDCT algorithm with and without recursive formula for polyphase analysis subband filtering. Our basic approach is to use only one adder/subtracter and one multiplier, but each has its own data bus, and a process controller is used to control the overall processing. We compare our design by using timing constraints, memory size, circuit complexities, and the number of operations. Low cost adder/subtracters and multipliers in VLSI implementation can be designed for real-time applications.
A massively parallel and versatile architecture for computer vision

A massively parallel architecture for vision that can be efficiently implemented as a dense and regular VLSI circuit is proposed. It consists of locally connected array of simple processors possessing certain capabilities, making it possible to form a complete vision system for binary images, that can operate in real time. For low level processing both a systolic implementation on a 2D array, as well as a non-systolic 'retinotopic' processing is possible. For the intermediate level, support for pyramidal hierarchy on the array is provided. To facilitate high level vision processing on the same architecture both AI, as well as the binary neural network approaches are applied.<<ETX>>
A Universal Framework for Salient Object Detection

In this paper, we propose a novel universal framework for salient object detection, which aims to enhance the performance of any existing saliency detection method. First, rough salient regions are extracted from any existing saliency detection model with distance weighting, adaptive binarization, and morphological closing. With the superpixel segmentation, a Bayesian decision model is adopted to refine the rough saliency map to obtain a more accurate saliency map. An iterative optimization method is designed to obtain better saliency results by exploiting the characteristics of the output saliency map each time. Through the iterative optimization process, the rough saliency map is updated step by step with better and better performance until an optimal saliency map is obtained. Experimental results on the public salient object detection datasets with ground truth demonstrate the promising performance of the proposed universal framework subjectively and objectively.
Low complexity rate control for VC-1 to H.264 transcoding

In this paper we focus on rate-control VC-1 to H.264 video transcoding in which video quality is obtained while maintaining a target bit rate for the transcoded output. The relationship between rate control in VC-1 and H.264 is studied and we use the rate control information obtained from VC-1 encoding to simplify the rate control algorithm in H.264. The key idea is to find the relationship between the quantization parameters (QPs) in VC-1 to the QPs in H.264 so that the first can be used to estimate the second, short cutting the need for a complex estimation in H.264. Experimental results show that the proposed rate control for transcoding is less complex than that for a full cascaded decoder while maintaining the target bit rate and PSNR.
Fast Mode Decision Based on Grayscale Similarity and Inter-View Correlation for Depth Map Coding in 3D-HEVC

The 3D extension of High Efficiency Video Coding significantly improves the coding efficiency of 3D video at the expense of computational complexity. This paper presents a novel fast mode decision algorithm for depth map coding based on the grayscale similarity and inter-view correlation. First, depth map grayscale similarity is adopted to judge whether the reference frame could assist the coding of the current frame. When the difference in the average grayscale between the co-located coding unit (CU) and the current CU is smaller than the similarity threshold, the depth level of the current CU will be restricted by that of the coded reference CU. Second, the grayscale similarity and inter-view correlation are jointly used for dependent views to achieve early decision on the best prediction unit (PU) mode. The mode decision procedure will be determined early when the co-located CU, which has a grayscale similarity with the current CU, selects Merge or Inter <inline-formula> <tex-math notation="LaTeX">$2N \times 2N$ </tex-math></inline-formula> as the best prediction mode. Moreover, when the corresponding CU in the independent view selects Merge or Inter <inline-formula> <tex-math notation="LaTeX">$2N \times 2N$ </tex-math></inline-formula> as the best prediction mode, the current CU will skip other PU modes checking based on the strong inter-view correlation. Finally, different strategies are proposed for the P-frames and B-frames of dependent views in view of the characteristics of different prediction structures. For B frames, the PU mode information of the coded independent view is utilized as reference to skip the unnecessary mode decision processes. For P frames, the spatial–temporal correlation is considered in the process of early mode decision to determine whether to choose the Merge mode or Inter <inline-formula> <tex-math notation="LaTeX">$2N \times 2N$ </tex-math></inline-formula> as the best mode. Experimental results show that our proposed scheme achieves considerable time saving with negligible degradation of coding performance.
Architecture and bus-arbitration schemes for MPEG-2 video decoder

An efficient MPEG-2 video decoder architecture together with several effective bus-arbitration schemes designed to meet the main profile at main level (MP@ML) real-time decoding requirement is presented. The overall architecture, as well as the design of major function-specific processing blocks (variable-length decoder, inverse two-dimensional discrete cosine transform unit, and motion-compensation unit), is discussed. A hierarchical and distributed controller approach is used, a bus-monitoring model for different bus-arbitration schemes to control external DRAM accesses is developed, and the system is simulated. Practical issues and buffer sizes are addressed and evaluated. With a 27 MHz clock, our architecture uses many fewer than the 667 cycles, the upper bound for the MP@ML decoding requirement, to decode each macroblock with a single external bus and DRAM.
Hardware module for an adaptive modeling unit of multi-symbol multiplication-free arithmetic encoder

An efficient hardware module for an adaptive modeling unit of a multi-symbol multiplication-free arithmetic coder is presented. By adopting the Fenwick (1994) data structure, our model can access, store, and update the cumulative frequencies with fewer memory accesses, despite a much simpler architecture compared to existing architectures of Witten, Neal and Cleary (1987) and of Peon, Osorio and Bruguera (see EUROMICRO 1997, New Frontiers of Information Technology, . The original multiplication-free algorithm has been revised to fit hardware implementation.