nils-herrmann/clean_scirepeval huggingface dataset:
Joint view filtering for multiview depth map sequences

Multi-view depth data representing the real-world 3D scenery can be inconsistent in the inter-view direction. Such inconsistency of depth data can distort the performance of the inter-view motion-compensated prediction (MCP), if the Multiview Video Coding extension of the Advanced Video Coding standard (H.264/MVC) is used for coding of multiview depth map data. In this paper we propose novel joint-view depth filtering (JVDF), which improves inter-view consistency of the multiview depth map data. Depth map images from all available views are warped to the same viewpoint, which results in multiple estimates of a real-world depth value. Estimates that belong to a confidence interval are fused together through a weighted average to produce a “noise-free” depth value estimate. In the provided simulation results the JVDF applied to 3-view depth map data significantly improves the performance of H.264/MVC coding, providing up to 13% of bitrate reduction.
Seamless switching of H.265/HEVC-coded dash representations with open GOP prediction structure

The Dynamic Adaptive Streaming over HTTP (DASH) enables bitrate adaptation through different representations of the same content. It is common to encode random access point (RAP) pictures at segment boundaries to support representation switching. As an open group of pictures (GOP) results into a temporary discontinuity of the video playback due to the inability to decode some pictures when switching representations, closed GOP prediction structures are normally used in DASH. This paper proposes two similar methods for using the open GOP prediction structure in DASH representations while a full picture rate is maintained also during representation switching. The first method is enabled with straightforward changes in the decoding of the High Efficiency Video Coding (H.265/HEVC) standard, whereas the second method utilizes the adaptive resolution change feature of the scalable (SHVC) extension of H.265/HEVC. Experiments show that the proposed methods outperform the use of closed GOPs by 5.6% on average in terms of Bjontegaard delta bitrate (BD-rate).
Parallel Encoding - Decoding Operation for Multiview Video Coding with High Coding Efficiency

Multiview Video Coding (MVC) standardization is an ongoing effort aiming to extend H.264/AVC by developing novel tools optimized for 3D and multiview video use cases. One of the key identified requirements for the MVC standard is its ability to support parallel processing of different views. The parallel operation is known to be especially important for 3DTV applications, where the display needs to output many views simultaneously to support head-motion parallax. In this paper, we present a novel coding structure that enables parallel encoder/decoder operation for different views, without compromising from the coding efficiency. This is achieved by systematically restricting the reference area of each view, so that encoding and decoding of macroblocks from different views could be efficiently pipelined and parallel operation of separate views becomes possible. As the inter-view prediction is still used, proposed structure achieves up to 0.9 dB gain compared to simulcast, maintaining very similar desirable parallelism characteristics.
Distributed lossless coding of hyperspectral images

In this paper we propose a novel distributed lossless compression scheme for hyperspectral images. All the images/bands are encoded independently, and the spectral correlation is exploited using distributed coding technologies in order to achieve low encoding complexity. At the encoder, sub-sampled images are successively encoded and transmitted. At the decoder, side information is generated with the knowledge of decoded sub-sampled images and other previously decoded bands. Reference bands are adaptively selected, and sliding window prediction or k nearest neighbor prediction is performed to capture the spatially varying spectral characteristics. Experimental results on AVRIS data demonstrate that the proposed scheme achieves competitive compression performance with respect to other state-of-the-art 3D codecs and with even lower encoding complexity than 2D codecs.
Frame loss error concealment for multiview video coding

The multiview video coding (MVC) standard is currently under development by the Joint Video Team as an extension of the advanced video coding (H.264/AVC) standard. An MVC encoder compresses more than one viewpoint of a scene captured by different cameras. Redundancies between views can be used for inter-view prediction in encoding as well as error concealment in decoding. In this paper, a new algorithm utilizing motion information of pictures from other views to conceal a lost picture is proposed. The algorithm first derives motion information for a lost picture based on motion fields of pictures in adjacent views. Then, traditional motion compensation is invoked within the view containing the lost picture to derive a concealed frame. Experimental results show that the proposed algorithm can improve video quality with a negligible computational complexity overhead compared to simple temporal error concealment algorithms.
Optimized reference frame selection for video coding by cloud

We investigate how to improve video coding efficiency via optimized reference frame selection using large-scale computation resources, e.g., a cloud. We first formulate the optimization problem for reference frame selection in video coding, which can be simplified to a manageable level. Given the maximum number of reference frames for encoding one frame, we give the upper bound of the coding efficiency on the High Efficiency Video Coding (HEVC) platform, which, although ideal, may require a huge amount of reference frame buffering at the decoder. Then we give a solution and the corresponding performance when the reference frame buffer size at the decoder is constrained. Experimental results show that when the number of reference frames is four, the proposed encoding scheme can achieve up to 16.9% bit-saving compared to HEVC, the state-of-the-art video coding system. The proposed encoding scheme is standard-compliant and can also be applied to H.264/AVC to improve coding efficiency.
Content-based hierarchical motion description for multiple video adaptation

Video adaptation has been considered as a promising technique to tackle challenging problems in pervasive multimedia applications. However, the styles of video representation and description in existing framework are not flexible enough to adapt diversified application environment. In this paper, we propose a novel solution based on intermediate description, which can support fast multiple video adaptation operations in signal level (e.g., temporal resolution reduction, bit-rate adaptation), structural level (e.g., random access of any shots, fast preview of any key-frames or thumbnails), as well as joint level. Experimental results show that the presented solution can support the operations in real-time environment, while maintaining coding efficiency, which demonstrates its feasibility and effectiveness.
Inter-view-predicted redundant pictures for viewpoint switching in multiview video streaming

Interactive selection of the desired viewpoint is one important feature in multiview video streaming. When receivers differ in the number of simultaneously displayed views, it is challenging to optimize the multiview coding structure particularly when it comes to the use of inter-view prediction and anchor picture frequency. This paper presents a coding and transmission scheme for streaming to clients of different display capabilities. The scheme saves transmission bandwidth by sending only those views that are displayed in a receiver. Furthermore, the scheme enables viewpoint switching through inter-view-predicted redundant pictures representing different sets of views being switched from and switched to. When switching happens at a switch point, an appropriate redundant picture is adaptively chosen based on which views were transmitted before and will be transmitted after the switch point. Experimental results show that the proposed scheme improves the rate-distortion (RD) performance compared to simple anchor picture insertion.
Quasi rate distortion optimization for binary hashing

Rate-distortion optimization has been a successful and significant method in video coding. By introducing Lagrange multiplier optimization into compress procedure, we can choose coding parameters simply and effectively. In nearest neighbor search problem, hashing has been a popular method to reduce computation and storage cost, which is consistent with video coding method. Conventionally, we evaluate a hashing method with mAP (mean average precision) w.r.t. different bit number, but leave bit cost as an independent measure index. In this paper, we make an attempt to combine retrieval accuracy and bit cost to make evaluation more comprehensive, using the concept of rate distortion optimization. Consequently, we obtain an evaluation criterion to judge which work point of a specific hashing method is better, taking both the accuracy and the bit cost into account. The exertion of an algorithm can be then determined.
Source-Channel Secrecy for Shannon Cipher System

Recently, a secrecy measure based on list-reconstruction has been proposed, in which a wiretapper is allowed to produce a list of $2^{mR_{L}}$ reconstruction sequences and the secrecy is measured by the minimum distortion over the entire list. In this paper, we show that this list secrecy problem is equivalent to the one with secrecy measured by a new quantity lossy equivocation , which is proved to be the minimum optimistic one-achievable source coding rate (the minimum coding rate needed to reconstruct the source within target distortion with positive probability for infinitely many blocklengths ) of the source with the wiretapped signal as two-sided information, and also can be seen as a lossy extension of conventional equivocation. Upon this (or list) secrecy measure, we study source-channel secrecy problem in the discrete memoryless Shannon cipher system with noisy wiretap channel. Two inner bounds and an outer bound on the achievable region of secret key rate, list rate, wiretapper distortion, and distortion of legitimate user are given. The inner bounds are derived by using uncoded scheme and (operationally) separate scheme, respectively. Thanks to the equivalence between lossy-equivocation secrecy and list secrecy, information spectrum method is leveraged to prove the outer bound. As special cases, the admissible region for the case of degraded wiretap channel or lossless communication for legitimate user has been characterized completely. For both these two cases, separate scheme is proved to be optimal. Interestingly, however, separation indeed suffers performance loss for other certain cases. Besides, we also extend our results to characterize the achievable region for Gaussian communication case. As a side product, optimistic lossy source coding has also been addressed.
QP refinement according to Lagrange multiplier for High Efficiency Video Coding

This paper presents a Quantization Parameter (QP) refinement algorithm used in Rate-Distortion Optimization (RDO) process to improve the coding efficiency for High Efficiency Video Coding (HEVC). To minimize the RD cost, QP is one of the parameters that can be optimized. Usually, multiple-QP optimization can be applied to choose the best QP value. However, this kind of optimization increases the encoding complexity significantly. We analyze the relationship between the QP value and Lagrange multiplier in this paper firstly. And then we apply the relationship between QP and Lagrange multiplier to the encoding process of HEVC. The proposed QP refinement method has already been adopted into HM (the HEVC reference software). The experimental results show that proposed algorithm can achieve about 1.4%~1.9% bit saving for luma component on average compared with the default anchor of HM-8.0. The bit saving on chroma components are larger than that of luma. The proposed method improves the coding efficiency without increasing the processing time.
A novel tracking-by-encoding scheme based on linear programming matching

In this paper, we present a new object tracking scheme. Different from conventional tracking process, the proposed method is combined with video encoding and can be easily embedded into video encoding system. The only information the tracker need is Summed Absolute Difference (SAD) generated by motion estimation during encoding. These SADs are used as soft votes in a fast Hough-Transform (HT) algorithm in parameter space of translation. This can be much faster than conventional tracking algorithm such as particle filter and more accurate than recent compress-domain approaches. Taking the advantage of HT, this approach is also robust to unstable camera, background moving and partial occlusion. The parameter space can be also extended to affine transform space with those soft votes to track scale change and partially rotation moving. We solve the latter multi-parameter problem by Linear Programming (LP) algorithm. The tracking result can also direct encoding behavior: detecting Region of Interest (ROI), adjusting QP, and etc.
A new system for Chinese sign language recognition

In this paper, we propose a new system for isolated sign language recognition (SLR) and continuous SLR. In isolated SLR, Histogram of Oriented Displacement is used to describe the trajectories, and multi-SVM is adopted for classification. In continuous SLR, we propose a Dynamic Programming method with warping templates obtained by Dynamic Time Warping (DTW) algorithm. We evaluate our approach with 450 phrases and 180 sentences recorded by Kinect and compare with classical methods, including Hidden Markov Models and state-of-the-art Conditional Random Fields (CRF), Hidden CRF and Latent Dynamic CRF. The experiments demonstrate the effectiveness of our method.
Video Inpainting for Largely Occluded Moving Human

In this paper, a video inpainting approach is proposed, which targets at repairing a video containing moving humans that are largely or completely occluded or missing for some of the frames. The proposed approach first categorizes typically periodic human motion in a video into a set of temporal states (called motion states), and then estimates the motion states for the frames with missing humans so as to repair the missing parts using other undamaged frames with the same motion states. This deviates from common approaches that directly repair the pixels of the damaged parts. Experiments demonstrate that the proposed method can well repair the damaged video sequences without introducing strong artifacts that exist in many existing techniques.
Diagonal motion partitions for inter prediction in HEVC

This paper presents diagonal motion partitions (DMP) for inter prediction in HEVC. In addition to the square and rectangular partitions, we propose to add diagonal shaped partitions to match different motion parts with oblique boundaries. Considering the overlap of pixels along the partition boundaries, the calculation of sum of absolute differences (SAD) and the motion compensation for the pixels on the boundaries are weighted. Besides, the residues of a diagonal prediction unit (PU) are augmented to form a rectangular one, so as to perform Hadamard transform on the residues. We also revise the advanced motion vector prediction (AMVP) and merge candidates based on the diagonal motion partitions. Experimental results show that on average 0.8%-1.0% BD-rate reduction can be achieved by DMP.
Intra coding for depth maps using adaptive boundary location

Depth maps, an essential part in the new generation of 3D video coding, allow rendering of arbitrary viewpoints of a video scene. Depth maps are characterized by sharp object boundaries, which significantly affect the rendering quality and account for the most bitrate for depth map coding. This paper proposes a novel intra coding method for depth maps based on a two-step adaptive boundary location process. By extracting a series of sub-blocks along a depth boundary and refining the boundary within sub-blocks, accurate predictions for blocks with arbitrary edge shapes can be realized. Experimental results show that the proposed scheme achieves bitrate reductions of up to 28% and 13% on average for seven test sequences of MPEG 3DV compared to original intra coding of H.264/AVC considering the same quality of synthesized views. Besides, subjective quality of virtual views is improved owning to well preserved boundary information.
A new approach to pattern recognition of remote sensing image using artificial neural network

The performance is analyzed of Multilayered Neural Networks (MLNs) and its training rule-back propagation (BP) algorithm. Some new methods are used to improve its training speed and to avoid local minima. Spatial information is introduced into pattern recognition of remote sensing image using MLNs. The authors use three-layer modified MLNs as classifiers in pattern recognition of remote sensing images (Landsat-4 TM Data). The experimental results verify the usefulness of the approach.<<ETX>>
OMP-based transform for inter coding in HEVC

Discrete Cosine Transform (DCT) has been the commonly used transform for a few decades in image/video coding. However, DCT does not work well on the blocks having anisotropic correlations. In this paper, based on the adaptive dictionary, we propose a new online transform scheme using Orthogonal Matching Pursuit (OMP) for High Efficiency Video Coding (HEVC). For a coding block, we construct its dictionary by exploiting non-local correlations from the reconstructed regions. The OMP algorithm is implemented to obtain the sparse transform coefficients. Experimental results show that the BD-rate savings of the proposed scheme for the sequences with strong edges can be up to 19.9%.
Depth-based motion vector prediction in 3D video coding

There are several data formats available for 3D video, among which is the Multiview Video plus Depth (MVD) representation that enables depth-image-based rendering (DIBR). In addition to DIBR, the MVD data format can enable more efficient compression for texture video coding. In this paper, we propose advanced motion vector prediction (MVP) techniques which utilize the availability of depth information in the MVD format for more efficient coding of the corresponding texture pictures. The proposed MVP was implemented on top of the Multiview Video Coding (MVC) extension of the Advanced Video Coding (H.264/AVC) standard and its reference software and tested under the simulation conditions of the Call for Proposals of 3D video coding technology issued by the Moving Picture Experts Group (MPEG). In the provided simulation results the proposed scheme significantly outperformed the conventional MVP (by up to 9.9% and on average by 7.5% in Bjontegaard delta bitrate) when it was applied for coding of 3-view MVD data.
An Efficient Fast Mode Decision Method for Inter Prediction in HEVC

The emerging High Efficiency Video Coding (HEVC) standard adopts many advanced techniques with flexible combinations, which enables HEVC to achieve about 50% bit-rate reduction for similar perceptual video quality relative to the prior video coding standard H.264/Advanced Video Coding. However, the enormously increased encoding complexity of HEVC inevitably becomes one of the greatest challenges for real-time applications. Among all the factors resulting in the increase in encoding complexity of HEVC, the quad-tree structure for coding units (CUs) with different sizes and accordingly a large number of prediction modes is one critical reason. Thus, it is greatly desired to develop a fast mode decision method for HEVC to reduce the computational complexity. In this paper, considering that HEVC employs the quad-tree structure, and the distortion of each sub-CU can indicate whether the current mode is suitable for current CU, we explore the relationship between the impossible modes and the distribution of the distortions to help the encoder skip checking the unnecessary modes. Besides, since the residual values can reflect the prediction result directly, we propose a method to skip some motion estimation operations according to the distribution of the residuals. Experimental results show that the proposed method can save about 77% of encoding time with only about a 4.1% bit-rate increase compared with HM16.4 anchor, while compared with the fast mode decision method adopted in HM16.4, the proposed algorithm can save about 48% of encoding time with only about a 2.9% bit-rate increase.
Single image super-resolution based on nonlocal similarity and sparse representation

Images with high resolution (HR) are often desired and required in imaging applications. Super-resolution (SR) is a promising but still challenging task to obtain an HR image (or sequence) from the observed low-resolution (LR) image (or sequence). In this paper, we present an SR approach for single image, by combining the image observation model, image nonlocal similarity, and sparse representation of image patches. First, we regularize low rank of matrix that is composed of nonlocal similar patches, to obtain an initial estimate of HR image. Then, each patch of the estimated image is sparsely represented in terms of LR dictionary, and the sparse coding coefficients are used to generate an output HR patch in terms of HR dictionary, resulting in the enhancement of high-frequency components in the HR estimate. The two over-complete dictionaries are jointly learned from example patch pairs in advance. Finally, the refined HR estimate is further constrained by the observation model. Experimental results have shown that the proposed SR method has obvious visual improvement in preserving edges and structures, while achieving comparable overall objective quality to the state-of-the-art methods.
Improved downstream rate-distortion performance of SHVC in DASH using sub-layer-selective interlayer prediction

Dynamic Adaptive Streaming over HTTP (DASH) has gained wide acceptance due to its ability of bitrate adaptation in diverse network conditions. In the meanwhile, it is asserted that the Scalable Extension (SHVC) of High Efficiency Video Coding (HEVC) can bring more efficient storage and caching compared with traditional single-layer video coding. However, using of SHVC in DASH raises the downstream bitrate relative to encoding each representation independently. This paper proposes a method to improve the downstream rate-distortion performance of SHVC in DASH. Experimental results show that the proposed method can bring up to 16.9 %-unit downstream bitrate reduction compared with directly use SHVC in DASH, while the efficiency in storage space is maintained.
View order alternation symmetrization and temporal inter-view combined prediction for 3D video coding

Asymmetric stereoscopic video coding, where one view has a higher quality than the other, has been researched widely. In order to balance the quality difference between views, it has been suggested to alternate the higher-quality view periodically. However, it has also been found out that inter-view prediction is more effective when the source of prediction is the higher-quality view. In this paper, it is proposed to change the inter-view prediction direction periodically together with gradual view refresh. The proposal was implemented on top of the reference software for the multiview and multiview-plus-depth extensions of the High Efficiency Video Coding (H.265/HEVC) standard. Experimental results show that up to 9.1% and an average of 3.1% bitrate reduction are achieved with respect to the MPEG Anchor.