nils-herrmann/clean_scirepeval huggingface dataset:
Drift reduction for a H.264-AVC fine grain scalability with motion compensation architecture

The recent advances in nonscalable video encoding brought by the H.264-AVC standard offered significant improvements in terms of rate-distortion performance. This paper proposes a H.264-AVC based fine grain scalable video encoder which also exploits the motion compensation tools of the H.264-AVC standard to explore the temporal redundancy in the enhancement layer. The enhancement layer is predicted from a high quality reference obtained from past information of the enhancement and base layers. One of the drawbacks of this architecture is the drift effect, which occurs when part of the enhancement layer used for prediction is not received by the decoder. The drift reduction approaches here proposed simultaneously allow improvements in the coding efficiency and a reduction of the drift effect. The experimental results show improvements up to 2 dB in coding efficiency in comparison to Intra coding (like used by the MPEG-4 FGS standard) using the MPEG-4 testing conditions.
Low complexity intra mode selection for efficient distributed video coding

Motion compensated frame interpolation (MCFI) is one of the most efficient solutions to generate side information (SI) in the context of distributed video coding. However, it creates SI with rather significant motion compensated errors for some frame regions while rather small for some other regions depending on the video content. In this paper, a low complexity Intra mode selection algorithm is proposed to select the most ‘critical’ blocks in the WZ frame and help the decoder with some reliable data for those blocks. For each block, the novel coding mode selection algorithm estimates the encoding rate for the Intra based and WZ coding modes and determines the best coding mode while maintaining a low encoder complexity. The proposed solution is evaluated in terms of rate-distortion performance with improvements up to 1.2 dB regarding a WZ coding mode only solution.
Adaptive deblocking filter for transform domain Wyner-Ziv video coding

Wyner-Ziv (WZ) video coding is a particular case of distributed video coding, the recent video coding paradigm based on the Slepian-Wolf and Wyner-Ziv theorems that exploits the source correlation at the decoder and not at the encoder as in predictive video coding. Although many improvements have been done over the last years, the performance of the state-of-the-art WZ video codecs still did not reach the performance of state-of-the-art predictive video codecs, especially for high and complex motion video content. This is also true in terms of subjective image quality mainly because of a considerable amount of blocking artefacts present in the decoded WZ video frames. This paper proposes an adaptive deblocking filter to improve both the subjective and objective qualities of the WZ frames in a transform domain WZ video codec. The proposed filter is an adaptation of the advanced deblocking filter defined in the H.264/AVC (advanced video coding) standard to a WZ video codec. The results obtained confirm the subjective quality improvement and objective quality gains that can go up to 0.63-dB in the overall for sequences with high motion content when large group of pictures are used.
Improving enhancement layer merge mode for HEVC scalable extension

In the HEVC scalable extension (SHVC), the merge mode prediction plays an important role due to its high selection probability and its low associated bitrate. In the SHVC enhancement layer (EL) merge mode prediction, the motion information is selected from the merge candidates, in this case the spatial, temporal, and inter-layer candidates. Therefore, the merge mode prediction is usually inefficient when the motion vector field (MVF) correlation between the merge candidates and the current EL block is low, especially in video sequences containing high motion activity. To address this problem, this paper proposes an improved EL merge mode prediction solution that adaptively refines the motion vector candidates by using both base layer (BL) and EL decoded information and after linearly combines the motion compensated samples with the BL reconstructed samples to achieve a better merge mode prediction quality. Experiments conducted for a rich set of test conditions have shown that significant compression efficiency gains can be achieved with the proposed improved scalable coding solution, notably up to 4.57% in BD-rate savings regarding the standard SHVC quality scalable codec.
Correlation modeling for a distributed scalable video codec based on the HEVC standard

The growing heterogeneity of networks, devices and consumption conditions asks for flexible and adaptive video coding solutions. The compression power of the HEVC standard and the benefits of the distributed video coding paradigm allow designing novel scalable coding solutions with improved error robustness and low encoding complexity while still achieving competitive compression efficiency. In this context, this paper proposes a novel scalable video coding scheme using a HEVC Intra compliant base layer and a distributed coding approach in the enhancement layers (EL). This design inherits the HEVC compression efficiency while providing low encoding complexity at the enhancement layers. The temporal correlation is exploited at the decoder to create the EL side information (SI) residue, an estimation of the original residue. The EL encoder sends only the data that cannot be inferred at the decoder, thus exploiting the correlation between the original and SI residues; however, this correlation must be characterized with an accurate correlation model to obtain coding efficiency improvements. Therefore, this paper proposes a correlation modeling solution to be used at both encoder and decoder, without requiring a feedback channel. Experiments results confirm that the proposed scalable coding scheme has lower encoding complexity and provides BD-Rate savings up to 3.43% in comparison with the HEVC Intra scalable extension under development.
Improved B-slices DIRECT mode coding using motion side information

The so-called DIRECT coding mode plays an important role in the RD performance of predictive video coding such as the H.264/AVC and MPEG-4 standards because there is typically a large probability that the DIRECT mode is selected in B-slices by the rate-distortion optimization (RDO) process. Although the current H.264/AVC DIRECT coding procedure exploits the motion vectors (MV) obtained from the reference frames in a rather effective way, it may still be improved by considering better motion information such as motion data derived by the side information (SI) creation process typical of distributed video coding. Therefore, this paper proposes an improved DIRECT coding mode for B-slices by efficiently exploiting some motion side information available at both the encoder and decoder. Experimental results show that the proposed improved DIRECT coding mode provides up to 8% bitrate saving or 0.46 dB PSNR improvement.
Improving SHVC performance with a joint layer coding mode

The growing need for a powerful scalable video coding engine targeting the heterogeneous landscape of network, devices, and consumption environments has led to the development of the Scalable High Efficiency Video Coding (SHVC) standard, an extension of the High Efficiency Video Coding (HEVC) standard. To improve the SHVC compression efficiency, this paper proposes a novel joint layer coding mode to be integrated in the SHVC codec. In the proposed coding mode, the base layer (BL) and enhancement layer (EL) decoded information are linearly combined at the pixel level to create an additional coding mode. To fuse the BL and EL driven predictions, a weighting term is defined to indicate the contributions of each of them for the final joint layer prediction. To reach high adaptability, these weights are computed at pixel level in the prediction unit. Moreover, to achieve the highest compression efficiency, the proposed joint layer coding mode is adaptively selected using a rate distortion optimization (RDO) mechanism. Experiments conducted for a rich set of test conditions have shown that significant compression efficiency gains can be achieved with the proposed joint layer coding mode, notably up to 4.3 % in BD-Rate savings regarding the standard SHVC quality scalable codec.
Motion compensated refinement for low complexity pixel based distributed video coding

Distributed video coding (DVC) is a new coding paradigm that enables to exploit video statistics, partially or totally at the decoder. A particular case of DVC, Wyner-Ziv coding, deals with lossy source coding with side information at the decoder and allows a shift of complexity from the encoder to the decoder, theoretically without any penalty in the coding efficiency. The Wyner-Ziv solution here described encodes each video frame independently (intraframe coding), but decodes the same frame conditionally (interframe decoding). At the decoder, and compensation tools are responsible to obtain an accurate interpolation of the original frame using previously decoded (temporally adjacent) frames. This paper proposes a novel approach to improve the performance of pixel domain Wyner-Ziv video coding by using a motion compensated refinement of the decoded frame and use it as improved side information. More precisely, upon partial decoding of each frame, the decoder refines its motion trajectories in order to achieve a better reconstruction of the decoded frame.
Refining Side Information for Improved Transform Domain Wyner-Ziv Video Coding

Wyner-Ziv (WZ) video coding is a particular case of distributed video coding, which is a recent video coding paradigm based on the Slepian-Wolf and WZ theorems. Contrary to available prediction-based standard video codecs, WZ video coding exploits the source statistics at the decoder, allowing the development of simpler encoders. Until now, WZ video coding did not reach the compression efficiency performance of conventional video coding solutions, mainly due to the poor quality of the side information, which is an estimate of the original frame created at the decoder in the most popular WZ video codecs. In this context, this paper proposes a novel side information refinement (SIR) algorithm for a transform domain WZ video codec based on a learning approach where the side information is successively improved as the decoding proceeds. The results show significant and consistent performance improvements regarding state-of-the-art WZ and standard video codecs, especially under critical conditions such as high motion content and long group of pictures sizes.
Improving scalable video coding performance with decoder side information

In a heterogeneous landscape of networks, devices and consumption environments, scalability is one of the most important video coding features. To achieve higher scalable video compression efficiency, this paper proposes a novel scalable video coding framework based on predictive video coding but also exploiting some additional decoder side information. The side information is estimated at both encoder and decoder using a motion compensated temporal interpolation technique, commonly used in distributed video coding solutions. To improve the B-slices compression efficiency, the side information independently created at each coding layer, notably the base and enhancement layers, is inserted in the corresponding layer decoded picture buffer to be exploited as an additional reference frame in the scalable predictive coding process. Experimental results have shown significant compression efficiency gains, notably up to around 3.5% in bitrate savings regarding the state-of-the-art SVC standard.
Descriptor-based adaptive tracking-by-detection for visual sensor networks

Local descriptors represent a powerful tool, which is exploited in several applications such as visual search, object recognition and visual tracking. Real-valued visual descriptors such as SIFT and SURF achieve state-of-the-art accuracy performance for a large set of visual analysis tasks. However, such algorithms are demanding in terms of computational capabilities and bandwidth, being unsuitable for scenarios where resources are constrained. In this context, binary descriptors provide an efficient alternative to real-valued descriptors, due to their low computational complexity, limited memory footprint and fast matching algorithms. In this paper, binary descriptors are used to perform visual tracking of an object along time. The proposed visual tracker performs descriptor matching between consecutive frames, applies filtering techniques to remove undesirable outliers and employs a suitable model to characterize the object appearance. In addition, techniques to code and transmit these description streams are employed, thus reducing the amount of data necessary to transmit to perform accurate object tracking. The efficiency of the proposed visual tracker is evaluated in terms of rate-accuracy, i.e. using the bitrate associated to the compressed binary descriptors and a quantitative metric to assess the accuracy of the visual tracker.
Local feature selection for efficient binary descriptor coding

In a visual sensor network, a large number of camera nodes are able to acquire and process image data locally, collaborate with other camera nodes and provide a description about the captured events. Typically, camera nodes have severe constraints in terms of energy, bandwidth resources and processing capabilities. Considering these unique characteristics, coding and transmission of the pixel-level representation of the visual scene must be avoided, due to the energy resources required. A promising approach is to extract at the camera nodes, compact visual features that are coded to meet the bandwidth and power requirements of the underlying network and devices. Since the total number of features extracted from an image may be rather significant, this paper proposes a novel method to select the most relevant features before the actual coding process. The solution relies on a score that estimates the accuracy of each local feature. Then, local features are ranked and only the most relevant features are coded and transmitted. The selected features must maximize the efficiency of the image analysis task but also minimize the required computational and transmission resources. Experimental results show that higher efficiency is achieved when compared to the previous state-of-the-art.
Boosting decoding quality performance in DASH-based streaming frameworks

Video streaming applications are increasingly popular but suffer from quality variations in best effort transmission networks. The recent DASH standard allows to dynamically adapt the transmitted rate to the network conditions under the control of the receiving client to achieve the best user experience. Naturally, the final video quality is also critically determined by the compression efficiency of the adopted coding solution. Video coding standards such as H.264/AVC and HEVC are based on a complexity allocation paradigm where complex encoders serve simpler decoders. However, some application environments can afford more complex and thus more intelligent decoders to boost the final quality. In this context, this paper proposes a decoder quality boosting algorithm where any standard compliant bitstreams are decoded with better quality in the lower rate/quality periods that succeed to higher rate/quality periods due to network bandwidth variations. The quality boosting results from a machine learning based solution, which is able to reuse already decoded frames from the previous higher quality segment to improve the lower quality segment. Thus, by increasing the decoder complexity it is possible to increase the quality of experience in DASH based streaming frameworks. Experimental results using the HEVC video coding standard show that significant quality gains may be obtained, notably achieving a graceful quality degradation in the rate dropping transitions.
Improved matching criterion for frame rate upconversion with trilateral filtering

Frame rate upconversion (FRUC) is an important post-processing technique to enhance the visual quality of low frame rate video. A major, recent advance in this area is FRUC based on trilateral filtering which novelty mainly derives from the combination of an edge-based motion estimation block matching criterion with the trilateral filter. However, there is still room for improvement, notably towards reducing the size of the uncovered regions in the initial estimated frame, this means the estimated frame before trilateral filtering. In this context, proposed is an improved motion estimation block matching criterion where a combined luminance and edge error metric is weighted according to the motion vector components, notably to regularise the motion field. Experimental results confirm that significant improvements are achieved for the final interpolated frames, reaching PSNR gains up to 2.73 dB, on average, regarding recent alternative solutions, for video content with varied motion characteristics.
Wyner-Ziv Stereo Video Coding using a Side Information Fusion Approach

Wyner-Ziv coding, also known as distributed video coding, is currently a very hot research topic in video coding due to the new opportunities it opens. This paper applies the distributed video coding principles to stereo video coding, to propose a practical solution for Wyner-Ziv stereo coding based on mask-based fusion of temporal and spatial side informations. The architecture includes a low-complexity encoder and avoids any communication between the cameras/encoders. While the rate-distortion (RD) performance strongly depends on the motion-based frame interpolation (MBFI) and disparity-based frame estimation (DBFE) solutions, first results show that the proposed approach is promising and there are still issues to address.
Adaptive Hash-Based Side Information Exploitation for Efficient Wyner-Ziv Video Coding

Wyner-Ziv video coding is a lossy source coding paradigm where the video statistics are exploited, partially or totally at the decoder. The side information represents a noisy version of the original frame and is generated at the decoder with time consuming motion estimation and compensation tools. This paper proposes a novel bidirectional hash motion estimation framework which enables the decoder to choose between past and/or future reference frames for frame interpolation. New features include the coding of DCT hash with zero-motion, combination of trajectory-based motion interpolation with hash-based motion estimation and adaptive selection of the DCT bands which are sent to the decoder in order to guide the motion estimation procedure. Gains up to 1.2 dB compared to previous motion interpolation approaches may be reached.
Evaluating multi-view plus depth coding solutions for 3D video scenarios

3D and free viewpoint video (FVV) are new media types able to provide the user an experience far beyond what is currently offered by traditional 2D video systems. Among the possible formats, the so-called multi-view plus depth format has the potential to synthesize at the decoder more views than those explicitly coded at the encoder. These additional views can be used to render arbitrary viewpoints (for FVV) or used in auto-stereoscopic displays, where a partial set of views are synthesized. Since depth provides information on the 3D scene geometry, depth coding artifacts (errors) affect the performance of the view synthesis techniques and may lead to geometric distortions in the virtual intermediate views. This paper makes a comprehensive performance evaluation of the relevant 2D texture coding schemes applied to both the texture and depth data, providing a much needed reference study. With this purpose in mind, two scenarios have been defined depending on the allowed encoder complexity. For each of these scenarios, the most relevant 2D video codecs have been evaluated, notably the low encoder complexity DISCOVER distributed video codec and the latest High Efficiency Video Coding (HEVC) standard predictive video codec. The performance of these coding solutions is evaluated through the quality of the rendered view according to precise and widely used test conditions.
Low complexity deblocking filter perceptual optimization for the HEVC codec

The compression efficiency of the state-of-art H.264/AVC video coding standard must be improved to accommodate the compression needs of high definition videos. To this end, ITU and MPEG started a new standardization project called High Efficiency Video Coding. The video codec under development still relies on transform domain quantization and includes the same in-loop deblocking filter adopted in the H.264/AVC standard to reduce quantization blocking artifacts. This deblocking filter provides two offsets to vary the amount of filtering for each image area. This paper proposes a perceptual optimization of these offsets based on a quality metric able to quantify the blocking artifacts impact on the perceived video quality. The proposed optimization involves low computational complexity and provides quality improvements with respect to a non-perceptually optimized H.264/AVC deblocking filter. Moreover, the proposed optimization allows up to 92% of complexity reduction regarding a brute force perceptual optimization which exhaustively tests all the possible offsets values.
Improving Transform Domain Wyner-Ziv Video Coding Performance

Distributed video coding (DVC) is a new video coding paradigm based on two key information theory results: the Slepian-Wolf and Wyner-Ziv theorems. A particular case of DVC, the so-called Wyner-Ziv coding, deals with lossy source coding with side information at the decoder and enables a flexible allocation of complexity between the encoder and the decoder. This paper proposes an improved transform domain Wyner-Ziv video codec including: 1) the integer block-based transform defined in the H.264/MPEG-4 AVC standard, 2) a quantizer with a symmetrical interval around zero for AC coefficients, and a quantization step size adjusted to the transform coefficient bands dynamic range, and 3) advanced frame interpolation for side information generation. The combination of these tools brings significant rate-distortion (RD) gains regarding the state-of-the-art results available in the literature
Wyner-Ziv video coding: A review of the early architectures and further developments

In 2002, the video coding community faced the emergence of a new video coding paradigm, the so-called Wyner-Ziv video coding, which was represented by two early solutions designed by the Stanford University and the University of California, Berkeley research teams. This paper intends to briefly review, and compare these two early Wyner-Ziv video coding solutions, notably from the functional point of view. Moreover, this paper reviews some important developments of the Stanford Wyner-Ziv coding architecture, which has become the most popular in the literature.