nils-herrmann/clean_scirepeval huggingface dataset:
Foveated High Efficiency Video Coding for Low Bit Rate Transmission

This work describes the design and subjective performance of Foveated High Efficiency Video Coding (FHEVC). Even though foveation has been widely used for various forms of compression since the early 1990s, we believe its use to improve HEVC is new. We consider the application of, possibly moving, foveated compression in this work and evaluate scenarios where it can be used to improve perceptual quality of videos under constrained transmission resources, e.g., bandwidth. A new method to reduce artifacts during remapping is also proposed. The preliminary implementation considers a single fovea only. Experiments summarizing user evaluations are presented to validate our implementation.
JPWL - an extension of JPEG 2000 for wireless imaging

In this paper, we present an overview of the JPWL standardization activity. JPWL is an extension of JPEG 2000 for the efficient transmission of JPEG 2000 images over an error-prone wireless network. More specifically, JPWL supports a set of tools for error protection and correction, including forward error correcting codes (FEC), unequal error protection (UEP), data partitioning and interleaving
Motion estimation techniques for digital TV: a review and a new contribution

The key to high performance in image sequence coding lies in an efficient reduction of the temporal redundancies. For this purpose, motion estimation and compensation techniques have been successfully applied. This paper studies motion estimation algorithms in the context of first generation coding techniques commonly used in digital TV. In this framework, estimating the motion in the scene is not an intrinsic goal. Motion estimation should indeed provide good temporal prediction and simultaneously require low overhead information. More specifically the aim is to minimize globally the bandwidth corresponding to both the prediction error information and the motion parameters. This paper first clarifies the notion of motion, reviews classical motion estimation techniques, and outlines new perspectives. Block matching techniques are shown to be the most appropriate in the framework of first generation coding. To overcome the drawbacks characteristic of most block matching techniques, this paper proposes a new locally adaptive multigrid block matching motion estimation technique. This algorithm has been designed taking into account the above aims. It leads to a robust motion field estimation precise prediction along moving edges and a decreased amount of side information in uniform areas. Furthermore, the algorithm controls the accuracy of the motion estimation procedure in order to optimally balance the amount of information corresponding to the prediction error and to the motion parameters. Experimental results show that the technique results in greatly enhanced visual quality and significant saving in terms of bit rate when compared to classical block matching techniques.
Comparison of DASH adaptation strategies based on bitrate and quality signalling

During a Dynamic Adaptive Streaming over HTTP (DASH) video transmission, the client can dynamically adapt the bitrate of the requested stream to changes in network conditions, by switching between different versions of the same content encoded at different bitrates, called representations. Each representation is split in smaller portions, called segments, and representation switching can be done at each segment. In this paper, we propose a comparative analysis of different adaptation strategies for representation switching, which exploit information on the bitrate and the visual quality of the different available versions of the requested content, at representation or at segment level. The results allow to identify the advantages and drawbacks related to each strategy in terms of bandwidth exploitation, maximization of visual quality and risk of buffer underflow.
Support Vector Machine based fusion for multi-view Distributed Video Coding

In multi-view Distributed Video Coding, Side Information can be computed either from previously decoded frames in the same view, or from previously decoded frames in adjacent views. In this paper, we address the problem of effectively fusing these two predictions. For this purpose, we propose two Support Vector Machine based fusion algorithms. We also identify a number of features to be used in classification. Experimental results show the efficiency of the proposed approach and its robustness over test sequences with greatly varying characteristics.
Background mosaicking for low bit rate video coding

This paper proposes a new technique to build a background memory based on mosaicking. More precisely, the technique first identifies background and foreground regions based on local motion estimates. Camera motion is then estimated on the background by applying parametric global motion estimation. Finally, after compensating for camera motion, the background content is temporally integrated in long-term memory. The method leads to high coding performances and allows for content-based functionalities.
Analytical distortion aware video coding for computer based video analysis

With the development of artificial intelligence, more and more multimedia applications for various tasks have emerged in our daily life. Meanwhile, as one of the main information sources of the applications, a huge amount of video data has been being generated by portable or mounted cameras in daily basis for varying purposes including surveillance, in which case we may need computers to "watch" videos to save labor cost. However, most video coding standards are designed for the highest human perceptual quality given a bit rate by minimizing a fidelity cost function (e.g., mean squared error, MSE), assuming the content will be consumed by human beings. In view of the above considerations, this paper proposes a new rate-analytical-distortion optimization method (RADO) for video analysis. Specifically, we consider moving object detection as the analysis task. Accordingly, we develop a novel rate analytical distortion (RAD) model for video coding, where the analytical distortion is related to the object detection performance expressed in terms of F-measure. As shown in the experimental results, the performance of the video analysis task can be significantly improved (up to 40% reduction of analytical distortion) with a slight bit rate increase.
A new two-stage global/local motion estimation based on a background/foreground segmentation

In video coding, the reduction of temporal redundancy is the key to achieving high performance. In the framework of sequence coding, motion estimation and compensation has been shown to be very efficient at removing temporal redundancy. The motion existing in a scene can be mainly seen as arising from local motions superimposed to the camera motion. A new two stage global/local motion estimation approach is presented. The global motion estimation only relies on the background information. It is based on a matching technique and the global motion model is chosen to be affine. Simulation results show significant improvements obtained with the proposed method compared to the usual methods.
Intra prediction using in-loop residual coding for the post-HEVC standard

A few years after standardization of the High Efficiency Video Coding (HEVC), now the Joint Video Exploration Team (JVET) group is exploring post-HEVC video compression technologies. In the intra prediction domain, this effort has resulted in an algorithm with 67 internal modes, new filters and tools which significantly improve HEVC. However, the improved algorithm still suffers from the long distance prediction inaccuracy problem. In this paper, we propose an In-Loop Residual coding Intra Prediction (ILR-IP) algorithm which utilizes inner-block reconstructed pixels as references to reduce the distance from predicted pixels. This is done by using the ILR signal for partially reconstructing each pixel, right after its prediction and before its block-level out-loop residual calculation. The ILR signal is decided in the rate-distortion sense, by a brute-force search on a QP-dependent finite codebook that is known to the decoder. Experiments show that the proposed ILR-IP algorithm improves the existing method in the Joint Exploration Model (JEM) up to 0.45% in terms of bit rate saving, without complexity overhead at the decoder side.
Object tracking based on temporal and spatial information

This paper addresses the problem of segmenting an image sequence in terms of multiple moving objects and tracking them through time and presents an object tracking algorithm. The objects are characterized through their temporal and spatial features so as to identify them and carry out the tracking procedure. The proposed tracking algorithm helps to detect the objects present in the current frame by supplying previous spatio-temporal information to the spatio-temporal segmentation procedure. In addition, the proposed algorithm tackles the correspondence problem. This is achieved through the use of a multiple hypotheses framework, the latter tests being based on both temporal and spatial characterizations of the objects.
Optimized tone mapping with LDR image quality constraint for backward-compatible high dynamic range image and video coding

Backward compatibility to low dynamic range (LDR) displays is an important requirement for high dynamic range (HDR) image and video coding in order to enable a successful transition to HDR technology. In a recent work , an optimized solution for tone mapping and inverse tone mapping of HDR images is achieved in terms of mean square error (MSE) of the logarithm of luminance values of HDR image pixels for backward-compatible compression. Although this pioneer optimization approach provides a well settled mathematical framework for tone mapping, one of its important shortcomings is not to take the quality of the resulting LDR images into account during the formulation. In this paper, we include the LDR image quality as a constraint to optimization problem and develop a methodology to compromise the trade-off between HDR image quality and LDR image quality during HDR image and video coding. The developed methodology is verified on HDR images by showing the increase (decrease) in the quality of generated LDR images while losing (gaining) from the rate-distortion performance of HDR image coding.
Efficient, robust, and fast global motion estimation for video coding

In this paper, we propose an efficient, robust, and fast method for the estimation of global motion from image sequences. The method is generic in that it can accommodate various global motion models, from a simple translation to an eight-parameter perspective model. The algorithm is hierarchical and consists of three stages. In the first stage, a low-pass image pyramid is built. Then, an initial translation is estimated with full-pixel precision at the top of the pyramid using a modified n-step search matching. In the third stage, a gradient descent is executed at each level of the pyramid starting from the initial translation at the coarsest level. Due to the coarse initial estimation and the hierarchical implementation, the method is very fast. To increase robustness to outliers, we replace the usual formulation based on a quadratic error criterion with a truncated quadratic function. We have applied the algorithm to various test sequences within an MPEG-4 coding system. From the experimental results we conclude that global motion estimation provides significant performance gains for video material with camera zoom and/or pan. The gains result from a reduced prediction error and a more compact representation of motion. We also conclude that the robust error criterion can introduce additional performance gains without increasing computational complexity.
A motion field segmentation to improve moving edges reconstruction in video coding

Block matching motion estimation techniques have shown their efficiency to reduce the temporal redundancy for video coding. However, they tend to introduce annoying block artifacts in the displaced frame difference due to their block-based model. In this paper, we propose a motion field refinement technique in order to overcome these artifacts. The method relies on vector quantization. The patterns to segment the motion field are derived by the LBG algorithm, the training being performed on segmented patterns obtained from natural images. Simulation results show a significant improvement due to the proposed method, in particular moving edges are much better motion compensated and the visual quality is greatly enhanced.<<ETX>>
Full-reference and reduced-reference quality metrics based on SIFT

In the last decade, an important research effort has been dedicated to implement objective image quality assessment metrics that reflect effectively human perception. Therefore, the aim of this paper is to propose new objective metrics that fulfill the demands of the image quality assessment field. For this sake, we propose two main full-reference (FR) quality metrics, and then adapt them in such a way to constitute several new reduced-reference (RR) quality metrics, for the case where the complete reference image is not available. We evaluate the influence of five types of distortion such as JPEG, JPEG2000, Gaussian Blur, AWGN, and Contrast change, on the image quality. The proposed metrics are based on the number of Scale-Invariant Feature Transform (SIFT) points, the number of SIFT matches between the unpaired and distorted images, and the Structural Similarity index (SSIM). In order to validate our proposed metrics, we compute the correlation between our metrics' scores and the subjective evaluation results. The results show a high correlation and a better quality range compared to well-known metrics, as well as a good robustness to reduced-reference situations.
Spatio-temporal segmentation based on motion and static segmentation

The problem of segmenting an image sequence in terms of regions characterized by a coherent motion is among the most challenging in image sequence analysis. This paper proposes a new technique which sequentially refines the segmentation and the motion estimation by combining static segmentation and motion information. The motion is robustly computed by a global estimation which remove the camera motion, followed by a local estimation using a matching technique and a robust estimator. Simulation results show the efficiency of the proposed technique.
Lagrangian Multiplier Adaptation for Rate-Distortion Optimization With Inter-Frame Dependency

Rate-distortion optimization (RDO) is widely used in video coding, which plays a critical role in enhancing the coding efficiency substantially. Currently, the RDO process is performed in a way that coding efficiency of each coding unit (CU) is maximized independently without considering the dependency among CUs. As we know, in the current hybrid video coding structure, spatial/temporal prediction techniques are extensively used, which introduce strong dependency among CUs. In this paper, we investigate RDO with inter-frame dependency, where the impact of coding performance of the current CU on that of the following frames is considered. Accordingly, an RDO scheme taking the inter-frame dependency into account is proposed by adapting the Lagrangian multiplier. The experimental results show that the proposed scheme can achieve about 3.22% and 3.19% BD-rate saving in average over the state-of-the-art High Efficiency Video Coding (HEVC) reference software HM15.0 in the low-delay $P$ (LDP) and low-delay $B$ (LDB) coding structures, respectively, with no extra encoding time. The proposed scheme can obtain a significantly higher coding gain than the multiple quantization parameter (MQP) (±3) optimization technique that would greatly increase the encoding time by a factor of about six. Coupled with MQP optimization, the proposed scheme can further achieve about 5.96% and 5.57% BD-rate savings in average over the HEVC and about 4.03% and 4.07% over the HEVC with MQP optimization, under the specified common test conditions for LDP and LDB coding structures, respectively.
Optimized tone mapping with flickering constraint for backward-compatible high dynamic range video coding

Backward compatibility to low dynamic range (LDR) displays is an important requirement for high dynamic range (HDR) image and video coding in order to enable a successful transition to HDR technology. In a recent work , an optimized solution for tone mapping and inverse tone mapping of HDR images and video frames is achieved in terms of the mean square error (MSE) of the logarithm of luminance values of HDR pixels for backward-compatible compression. Although this pioneer optimization approach provides a well settled mathematical framework for tone mapping, one of its important shortcomings is the flickering distortions that can occur on the resulting video due to the individual tone mapping applied to each frame. In this paper, we include the flickering distortion level as a constraint to optimization problem and develop a methodology to compromise the trade-off between the rate-distortion performance and flickering effect. The developed methodology is validated on HDR video sequences by showing the changes in rate-distortion performances with respect to the different flickering levels.
Efficient Multi-Strategy Intra Prediction for Quality Scalable High Efficiency Video Coding

As an extension of high efficiency video coding (HEVC), the scalable high efficiency video coding (SHVC) introduces multiple layers with inter-layer predictions, which greatly increases the complexity on top of the already complicated HEVC encoder. In intra prediction for quality SHVC, coding tree unit allows recursive splitting into four depth levels, which considers 35 intra prediction modes and inter-layer reference (ILR) mode to determine the best possible mode at each depth level. This achieves the highest coding efficiency but incurs a substantially high computational complexity. In this paper, we propose a novel intra prediction scheme to effectively speed up the enhancement layer intra coding in quality SHVC. The new features of the proposed framework include: first, spatial correlation and its correlation degree are combined to predict most probable depth level candidates. Second, for a given depth candidate, based on the probabilities of the ILR mode, we check the ILR mode by examining the residual distribution based on skewness and kurtosis to determine whether the residuals follow a Gaussian distribution. In that case, the intra prediction comparisons, which require a high complexity, are skipped. Third, during the intra prediction selection from 35 Intra prediction modes, spatial and inter-layer correlations are combined with the local monotonicity of the Hadamard costs associated with the modes in a small neighborhood, to examine only a portion of the intra prediction modes. Finally, a hypothesis testing on the currently selected depth level is performed to examine whether the residuals present significant differences within their block to early terminate depth selection. The proposed multi-step multi-strategy scheme aims to minimize the number of depth selections while greatly reducing the mode decision complexity for a depth candidate in a hierarchical fashion. Our experimental results demonstrate that the proposed scheme can achieve a speedup gain of more than 75% in average on the test video sequences, while maintaining almost the same coding efficiency.
JPSearch: Metadata Interoperability During Image Exchange [Standards in a Nutshell]

Billions of new images are generated daily. However, the management and exchange of the associated metadata remains cumbersome. Therefore, the Joint Photographic Experts Group (JPEG) has recently promoted a new set of technologies enabling the interoperability between image repositories and/or their clients to an international standard. JPSearch is a suite of specifications that supports the enrichment with metadata data stored in JPEG or JPEG 2000 image formats. It addresses schema and ontology building blocks, a query format, a file format for metadata embedded in image data, and a data interchange format for image repositories.
A metric approach toward point process divergence

Estimating divergence between two point processes, i.e. probability laws on the space of spike trains, is an essential tool in many computational neuroscience applications, such as change detection and neural coding. However, the problem of estimating divergence, although well studied in the Euclidean space, has seldom been addressed in a more general setting. Since the space of spike trains can be viewed as a metric space, we address the problem of estimating Jensen-Shannon divergence in a metric space using a nearest neighbor based approach. We empirically demonstrate the validity of the proposed estimator, and compare it against other available methods in the context of two-sample problem.
An Information Theoretic Algorithm for Finding Periodicities in Stellar Light Curves

We propose a new information theoretic metric for finding periodicities in stellar light curves. Light curves are astronomical time series of brightness over time, and are characterized as being noisy and unevenly sampled. The proposed metric combines correntropy (generalized correlation) with a periodic kernel to measure similarity among samples separated by a given period. The new metric provides a periodogram, called Correntropy Kernelized Periodogram (CKP), whose peaks are associated with the fundamental frequencies present in the data. The CKP does not require any resampling, slotting or folding scheme as it is computed directly from the available samples. CKP is the main part of a fully-automated pipeline for periodic light curve discrimination to be used in astronomical survey databases. We show that the CKP method outperformed the slotted correntropy, and conventional methods used in astronomy for periodicity discrimination and period estimation tasks, using a set of light curves drawn from the MACHO survey. The proposed metric achieved 97.2% of true positives with 0% of false positives at the confidence level of 99% for the periodicity discrimination task; and 88% of hits with 11.6% of multiples and 0.4% of misses in the period estimation task.
Estimation of instantaneous power in the EEG to assess brain connectivity with high temporal resolution

This paper presents advanced signal processing algorithms to quantify brain connectivity with high temporal resolution using the electroencephalogram (EEG). The experimental paradigm exploits the visual cortex response to flickering images at a given frequency. The envelope of the EEG at the flickering frequency collected by 128 electrodes over the head quantifies the communication amongst the brain areas with high temporal resolution. This work proposes to use the empirical mode decomposition to find the flickering frequency and then use the Hilbert transform to estimate the instantaneous amplitude. A video of the topographical display of the instantaneous power over the array helps us visualize the exquisite communication that occurs during the stimulus presentation.
Information Theoretic fuzzy modeling for regression

This paper presents a novel, Information Theoretic Learning (ITL) method to model a fuzzy system for regression tasks that minimizes the Renyi's entropy of the error signal. An architecture based on a generalization of the well-kwon Adaptive-Network-Based Fuzzy Inference System (ANFIS) was used to perform such a modeling. The resulting method was tested on the prediction of future values for the Mackey-Glass chaotic time series. The results show that, when using the ITL cost function, the method returns better models in comparison with a Mean Squared Error (MSE)-guided cost function.