nils-herrmann/clean_scirepeval huggingface dataset:
The latest open-source video codec VP9 - An overview and preliminary results

Google has recently finalized a next generation open-source video codec called VP9, as part of the libvpx repository of the WebM project (http://www.webmproject.org/). Starting from the VP8 video codec released by Google in 2010 as the baseline, various enhancements and new tools were added, resulting in the next-generation VP9 bit-stream. This paper provides a brief technical overview of VP9 along with comparisons with other state-of-the-art video codecs H.264/AVC and HEVC on standard test sets. Results show VP9 to be quite competitive with mainstream state-of-the-art codecs.
Format independent encryption of generalized scalable bit-streams enabling arbitrary secure adaptations [multimedia communication applications]

Secure format-independent adaptation of bit-streams during delivery is becoming increasingly important to cope with content piracy while still accommodating diverse networks, terminals and formats. Generalized scalable bit-streams are particularly advantageous in this regard, since they enable a variety of efficient and secure adaptations. Further, by associating such a bit-stream with appropriate metadata, such as those standardized in MPEG-21 Part 7, entitled digital item adaptation (DIA), the adaptation process can be fully format-independent. In this paper, to maximally secure a generalized scalable bit-stream while allowing arbitrary encrypted domain adaptations, strong progressive encryption methods are extended to multiple dimensions. Further it is shown that by appropriate modeling of such bit-streams and re-use of some DIA descriptions, the encryption and decryption engines themselves can be entirely metadata-driven and format-independent. This leads to end-to-end format-independent secure and adaptive delivery architectures for scalable bit-streams.
Distributed Image Coding for Digital Image Recovery From the Print-Scan Channel

A printed digital photograph is difficult to reuse because the digital information that generated the print may no longer be available. This paper describes a method for approximating the original digital image by combining a scan of the printed photograph with digital auxiliary information kept together with the print. We formulate and solve the approximation problem using a Wyner-Ziv coding framework. During encoding, the Wyner-Ziv auxiliary information consists of a small amount of digital data composed of a number of sampled luminance pixel blocks and a number of sampled color pixel values to enable subsequent accurate registration and color-reproduction during decoding. The registration and color information is augmented by an additional amount of digital data encoded using Wyner-Ziv coding techniques that recovers residual errors and lost high spatial frequencies. The decoding process consists of scanning the printed photograph, together with a two step decoding process. The first decoding step, using the registration and color auxiliary information, generates a side-information image which registers and color corrects the scanned image. The second decoding step uses the additional Wyner-Ziv layer together with the side-information image to provide a closer approximation of the original, reducing residual errors and restoring the lost high spatial frequencies. The experimental results confirm the reduced digital storage needs when the scanned print assists in the digital reconstruction.
A butterfly structured design of the hybrid transform coding scheme

The hybrid transform coding scheme that alternates amongst the asymmetric discrete sine transform (ADST) and the discrete cosine transform (DCT) depending on the boundary prediction conditions, is an efficient tool for video and image compression. It optimally exploits the statistical characteristics of prediction residual, thereby achieving significant coding performance gains over the conventional DCT-based approach. A practical concern lies in the intrinsic conflict between transform kernels of ADST and DCT, which prevents a butterfly structured implementation for parallel computing. Hence the hybrid transform coding scheme has to rely on matrix multiplication, which presents a speed-up barrier due to under-utilization of the hardware, especially for larger block sizes. In this work, we devise a novel ADST-like transform whose kernel is consistent with that of DCT, thereby enabling butterfly structured computation flow, while largely retaining the performance advantages of hybrid transform coding scheme in terms of compression efficiency. A prototype implementation of the proposed butterfly structured hybrid transform coding scheme is available in the VP9 codec repository.
On Macroblock Partition for Motion Compensation

In the H.264/AVC video coding standard, motion compensation can be performed by partitioning macroblocks into square or rectangular sub-macroblocks in a quadtree decomposition. This paper studies a motion compensation method using wedges, i.e. partitioning macroblocks or sub-macroblocks into two regions by an arbitrary line segment. This technique allows the shapes of the divided regions to better match the boundaries between moving objects. However, there are a large number of ways to slice a block and searching exhaustively over all of them would be an extremely computer-intensive task. Thus, we propose a fast algorithm which detects the predominant edge orientations within a block in order to pre-select candidate wedge lines. Finally a comparison among macroblock partition methods is performed, which points to the higher performance of the wedge partition method.
Successive refinement lattice vector quantization

Lattice Vector quantization (LVQ) solves the complexity problem of LBG based vector quantizers, yielding very general codebooks. However, a single stage LVQ, when applied to high resolution quantization of a vector, may result in very large and unwieldy indices, making it unsuitable for applications requiring successive refinement. The goal of this work is to develop a unified framework for progressive uniform quantization of vectors without having to sacrifice the mean- squared-error advantage of lattice quantization. A successive refinement uniform vector quantization methodology is developed, where the codebooks in successive stages are all lattice codebooks, each in the shape of the Voronoi regions of the lattice at the previous stage. Such Voronoi shaped geometric lattice codebooks are named Voronoi lattice VQs (VLVQ). Measures of efficiency of successive refinement are developed based on the entropy of the indices transmitted by the VLVQs. Additionally, a constructive method for asymptotically optimal uniform quantization is developed using tree-structured subset VLVQs in conjunction with entropy coding. The methodology developed here essentially yields the optimal vector counterpart of scalar "bitplane-wise" refinement. Unfortunately it is not as trivial to implement as in the scalar case. Furthermore, the benefits of asymptotic optimality in tree-structured subset VLVQs remain elusive in practical nonasymptotic situations. Nevertheless, because scalar bitplane- wise refinement is extensively used in modern wavelet image coders, we have applied the VLVQ techniques to successively refine vectors of wavelet coefficients in the vector set-partitioning (VSPIHT) framework. The results are compared against SPIHT and the previous successive approximation wavelet vector quantization (SA-W-VQ) results of Sampson, da Silva and Ghanbari.
Learning-Based, Automatic 2D-to-3D Image and Video Conversion

Despite a significant growth in the last few years, the availability of 3D content is still dwarfed by that of its 2D counterpart. To close this gap, many 2D-to-3D image and video conversion methods have been proposed. Methods involving human operators have been most successful but also time-consuming and costly. Automatic methods, which typically make use of a deterministic 3D scene model, have not yet achieved the same level of quality for they rely on assumptions that are often violated in practice. In this paper, we propose a new class of methods that are based on the radically different approach of learning the 2D-to-3D conversion from examples. We develop two types of methods. The first is based on learning a point mapping from local image/video attributes, such as color, spatial position, and, in the case of video, motion at each pixel, to scene-depth at that pixel using a regression type idea. The second method is based on globally estimating the entire depth map of a query image directly from a repository of 3D images ( image+depth pairs or stereopairs) using a nearest-neighbor regression type idea. We demonstrate both the efficacy and the computational efficiency of our methods on numerous 2D images and discuss their drawbacks and benefits. Although far from perfect, our results demonstrate that repositories of 3D content can be used for effective 2D-to-3D image conversion. An extension to video is immediate by enforcing temporal continuity of computed depth maps.
Adaptive interpolation filter scheme in AV1

Video codecs heavily depend on sub-pixel level motion compensation to achieve superior compression performance. Interpolation filters with both anti-aliasing and denoising properties play a critical role in producing high quality prediction at sub-pixel positions. Prior research has developed many adaptive filtering schemes to improve the prediction precision for compression gains. On the other hand, such filtering operations require intense computation and may lead to scattered cache footprints, therefore, account for a major portion of the overall decoding cost in both software and hardware implementations. An adaptive interpolation filtering scheme is proposed in this work to optimize the trade off between prediction quality and decoding performance. It employs a separable model and selects filter kernels independently for horizontal and vertical directions to better capture statistical variations. In order to obtain sharper transition and reduce the ripple effect in the passband in frequency domain, a 12-tap filter is introduced in conjunction with a complimentary operation design that minimizes its impact on the decoding performance. The scheme achieves on average 1.3% coding gains across a wide range of test settings, with fairly limited additional hardware cost.
On the robustness of vector set partitioning image coders to bit errors

A vector enhancement of Said and Pearlman's Set Partitioning in Hierarchical Trees (SPIHT) methodology, named VSPIHT has recently been proposed for embedded wavelet image compression. A major advantage of vector based embedded coding with fixed length VQs over scalar embedded coding, is its superior robustness to noise. In this work we show that vector set partitioning can effectively alter the balance of bits in the bit stream so that significantly fewer critical bits carrying significant information is transmitted, thereby improving inherent noise resilience. Additionally, the degradation in reconstruction quality caused by errors in non-critical quantization information, can be reduced by appropriate VQ indexing, or designing channel optimized VQs for the successive refinement systems. For very noisy channels unequal error protection to the critical and non critical bits with either block codes or convolution codes are used. Extensive simulation results are presented.
Vector set partitioning with classified successive refinement VQ for embedded wavelet image and video coding

The set partitioning in hierarchical trees (SPIHT) approach for still image compression proposed by Said and Pearlman (see  1996) is one of the most efficient embedded gray image compression schemes to date. The algorithm relies on a very efficient scanning cum bit-allocation scheme for quantizing the coefficients obtained by a wavelet decomposition of an image. We adopt this scheme to scan vectors of wavelet coefficients, and use successive refinement VQ techniques with staggered bit-allocation to quantize several wavelet coefficients at once. The new scheme is named VSPIHT (vector SPIHT). We present some coding results comparing VSPIHT to the scalar counterpart in the mean-squared-error sense. The method readily generalizes to color images and video where the vector-based approach makes more sense. We present the coding results on intra frames of QCIF sequences as compared against H.263.
Exploiting patterns of data magnitude for efficient image coding

In order to increase the computational efficiency of compression methods we have to consider that new hardware architectures increasingly rely more on wider data paths and parallel processing (e.g., SIMD and multi-core), than on faster clocks. Higher data throughputs are achieved with entropy coding methods that process larger amounts of information each time, and use context dependencies that are less complicated and that can be quickly updated. We propose a coding method with properties more suited to the new processors, that achieves better compression by exploiting patterns of data magnitude. We present experimental results on image coding implementations that take advantage of the fast decay of transform coefficient variance with frequency.
A Wyner-Ziv Video Transcoder

Wyner-Ziv (WZ) coding of video utilizes simple encoders and highly complex decoders. A transcoder from a WZ codec to a traditional codec can potentially increase the range of applications for WZ codecs. We present a transcoder scheme from the most popular WZ codec architecture to a differential pulse code modulation/discrete cosine transform codec. As a proof of concept, we implemented this transcoder using a simple pixel-domain WZ codec and the standard H.263+. The transcoder design aims at reducing complexity as a large amount of computation is saved by reusing the motion estimation, calculated at the side information generation process, and the l-frame streams. New approaches are used to generate side information and to map motion vectors for the transcoder. Results are presented to demonstrate the transcoder performance.
Mapping motion vectors for Awyner-Ziv video transcoder

Wyner-Ziv (WZ) coding of video utilizes simple encoders and highly complex decoders. A transcoder from a WZ codec to a traditional codec can potentially increase the range of applications for WZ codecs. We present a transcoder scheme from the most popular WZ codec architecture to a DPCM/DCT codec. As a proof of concept, we implemented this transcoder using a simple pixel domain WZ codec and the standard H.263+. The transcoder design aims at reducing complexity, since the transcoder has to perform both WZ decoding and DPCM/DCT encoding, including motion estimation. New approaches are used to map motion vectors for such a transcoder. Results are presented to demonstrate the transcoder performance.
Video Super-Resolution Using Codebooks Derived From Key-Frames

Example-based super-resolution (SR) is an attractive option to Bayesian approaches to enhance image resolution. We use a multiresolution approach to example-based SR and discuss codebook construction for video sequences. We match a block to be super-resolved to a low-resolution version of the reference high-resolution image blocks. Once the match is found, we carefully apply the high-frequency contents of the chosen reference block to the one to be super-resolved. In essence, the method relies on “betting” that if the low-frequency contents of two blocks are very similar, their high-frequency contents also might match. In particular, we are interested in scenarios where examples can be picked up from readily available high-resolution images that are strongly related to the frame to be super-resolved. Hence, they constitute an excellent source of material to construct a dynamic codebook. Here, we propose a method to super-resolve a video using multiple overlapped variable-block-size codebooks. We implemented a mixed-resolution video coding scenario, where some frames are encoded at a higher resolution and can be used to enhance the other lower-resolution ones. In another scenario, we consider the framework where the camera captures a video at a lower resolution and also takes periodic snapshots at a higher resolution. Results indicate substantial gains over interpolation and fixed-codebook SR, and significant gains over previous works as well.
Generalized variable dimensional set partitioning for embedded wavelet image compression

A vector enhancement of Said and Pearlman's (1996) set partitioning in hierarchical trees (SPIHT) methodology, named VSPIHT, has recently been proposed for embedded wavelet image compression. While the VSPIHT algorithm works better than scalar SPIHT for most images, a common vector dimension to use for coding an entire image may not be optimal. Since statistics vary widely within an image, a greater efficiency can be achieved if different vector dimensions are used for coding the wavelet coefficients from different portions of the image. We present a generalized methodology for developing a variable dimensional set partitioning coder, where different parts of an image may be coded in different vectoring modes, with different scale factors, and up to different number of passes. A Lagrangian rate-distortion criterion is used to make the optimum coding choices. Coding passes are made jointly for the vectoring modes to produce an embedded bitstream.
MRC Compression of Compound Documents Using Threshold Segmentation, Iterative Data-Filling and H.264/AVC-INTRA

The Mixed Raster Content (MRC) ITU document compression standard (T.44) specifies a multi-layer representation of a compound document. It is expected that higher compression can be achieved if more efficient compression standards are used to compress each layer. In this paper we present an MRC compound document codec that uses H.264/AVC operating in INTRA mode to encode Background/Foreground layers and JBIG2 to encode the binary Mask layer. We also present a threshold-based layer segmentation algorithm and an iterative data-filling algorithm for redundant regions. The result is an unrivaled performance for compressing compound documents as demonstrated by our experiments.
Parameter Selection for Wyner–Ziv Coding of Laplacian Sources with Additive Laplacian or Gaussian Innovation

A large number of practical coding scenarios deal with sources such as transform coefficients that can be well modeled as Laplacians. For regular coding of such sources, samples are often quantized by a family of uniform quantizers possibly with a deadzone, and then entropy coded. For the Wyner-Ziv coding problem when correlated side-information is available at the decoder, the side-information can be modeled as obtained by independent additive Laplacian or Gaussian innovation on the source. This paper deals with optimal choice of parameters for practical Wyner-Ziv coding in such scenarios, using the same quantizer family as in the regular codec to cover a range of rate-distortion tradeoffs, given the variances of the source and additive noise. We propose and analyze a general encoding model based on multilevel coset codes that combines source coding and channel coding and show that at practical block lengths and code complexities, not pure channel coding but a hybrid combination of source coding and channel coding with right parameters provide optimal rate-distortion performance. We also provide a framework for on-the-fly parameter choice based on nonparametric representation of a set of seed functions, for use in scenarios where variances are estimated during encoding. A good insight in the optimal parameter selection mechanism is essential for building practical distributed codecs.
A Technical Overview of VP9—The Latest Open-Source Video Codec

Google has recently finalized a next-generation open-source video codec called VP9, as part of the libvpx repository of the WebM project (http://www.webmproject.org/). Starting from the VP8 video codec released by Google in 2010 as the baseline, various enhancements and new tools were added, resulting in the next-generation bit stream VP9. The bit stream was finalized with the exception of essential bug fixes in June 2013. Prior to the release, however, all technical developments were being conducted openly in the public experimental branch of the repository for many months. This paper provides a brief technical overview of the coding tools included in VP9, along with coding performance comparisons with other state-of-the-art video codecs—namely, H.264/AVC and HEVC—on standard test sets. While a completely fair comparison is impossible to conduct because of the limitations of the respective encoder implementations, the tests show VP9 to be quite competitive with mainstream state-of-the-art codecs.
Complexity-scalable H.264/AVC in an IPP-based video encoder

Real-time high-definition video encoding is a computation-hungry task that challenges software-based solutions. For that, in this work we adopted an Intel software implementation of an H.264 video encoder and optimized its prediction stage in the complexity sense (C). Thus, besides looking for the coding options which lead to the best coded representation in terms of rate and distortion, we constrain the process to fit within a certain time budget. We present an RDC-optimized framework which allows for real-time HD video compression.
H.264/AVC hardware encoders and low-power features

Because of significant bit rate reduction in comparison to the previous video compression standards, the H.264/AVC has been successfully used in a wide range of applications. In hardware design for H.264/AVC video encoders, power reduction is currently a tremendous challenge. This paper presents a survey of different H.264/AVC hardware encoders focusing on power features and power reduction techniques to be applied. A new H.264/AVC hardware encoder, named VENGME, is proposed. This low power encoder is a four-stage architecture with memory access reduction, in which, each module has been optimized. The actual total power consumption, estimated at RTL level, is 19.1mW.
A Study on Prosody of Vietnamese Emotional Speech

This paper describes the analyses of the prosody of Vietnamese emotional speech, accomplished to find the relations between prosodic variations and emotional states in Vietnamese speech. These relations were obtained by investigating the variations of prosodic features in Vietnamese emotional speech in comparison with prosodic features of neutral speech. The analyses were performed on a multi-style emotional speech database which consisted of Vietnamese sentences uttered in different styles. Specifically, four emotional styles were considered: happiness, sadness, cold anger, and hot anger. Speech data in the neutral style were also collected, and prosodic differences of each style with respect to this neutral baseline were quantified. The acoustic features related to prosody which were investigated were fundamental frequency, power, and duration. According to the analysis results, for each speaker of the database, a set of prosodic variation coefficients was produced for each emotional style. This will help for bringing emotions into Vietnamese synthesized speech, making them more natural.